Yocto Project 3.4 Release Test Plan: Difference between revisions
From Yocto Project
Jump to navigationJump to search
| (2 intermediate revisions by the same user not shown) | |||
| Line 418: | Line 418: | ||
= Pass / Fail Criteria = | = Pass / Fail Criteria = | ||
This criteria is defined in [[Yocto_Project_v3. | This criteria is defined in [[Yocto_Project_v3.4_Status#Yocto_Project_v3.4_Release_Criteria]] | ||
= Exit Criteria = | = Exit Criteria = | ||
| Line 433: | Line 433: | ||
= Schedule = | = Schedule = | ||
The planed schedule for 3. | The planed schedule for 3.4 can be found at [[Yocto_3.4_Schedule]] | ||
== Live Schedule & Execution History == | == Live Schedule & Execution History == | ||
| Line 449: | Line 449: | ||
| DONE | | DONE | ||
| https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults | | https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults | ||
|- bgcolor="" | |- bgcolor="93DB70" | ||
| M2 | | M2 | ||
| | | 07/26/2021 | ||
| | | 07/30/2021 | ||
| | | DONE | ||
| | | https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults | ||
|- bgcolor="" | |- bgcolor="FFD700" | ||
| M3 | | M3 | ||
| 09/06/2021 | |||
| | | | ||
| | |In Progress | ||
| | | | ||
|- bgcolor="" | |- bgcolor="" | ||
Latest revision as of 05:28, 8 September 2021
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | 6564}}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | 6565}}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:KERNEL_MANUAL | 6714}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
Revision history
| Version | Modifier | Comments |
|---|---|---|
| 1.0 | Jain Sangeeta | First draft |
Introduction
This is the the overall test plan for version 3.4 of the Yocto Project, for all the generic information this document will be pointing to QA Master Test Plan
Objectives
This test plan for Yocto Project 3.4 release supports the following objectives:
- To define the tools to be used throughout the testing process.
- To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
- To define how the tests will be conducted.
- Release Criteria
- Risks and contingencies
QA_Master_Test_Plan#Objectives_and_Tasks
Risk Analysis & Contingency Plan
QA identified risks for this release are mentioned here.
| Module | Risk Description | Risk Level | Contingency plan | Owner |
|---|---|---|---|---|
| BSP | 31 manual test cases | medium | test cases to be automated | Intel |
| Build Appliance | 3 manual test cases, not executed by any QA team | medium | test cases to be automated | |
| Compliance | 4 manual test cases, not executed by any QA team | medium | test cases to be automated | |
| Kernel dev | 9 manual test cases, not executed by any QA team | medium | test cases to be automated | |
| OE-Core | 4 manual test cases | medium | test cases to be automated | Intel |
| SDK | 1 manual test case, not executed by any QA team | medium | test case to be automated | |
| Toaster | 93 manual test cases, not executed by any QA team | medium | test cases to be automated |
Test Strategy
Is followed the general strategy defined on QA_Master_Test_Plan#Testing_Strategy and below is showed specific information applicable only for 3.4 Release
Test Items
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7 * Debian 8 * Debian 9 * Fedora 28 * Fedora 29 * Fedora 30 * OpenSuse 150 * OpenSuse 151 * OpenSuse 423 * OpenSuse Tumbleweed * ubuntu 16.04 * ubuntu 18.04
Test Cases
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/runtime
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/selftest
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/manual
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/sdk
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/sdkext
OE-CORE
| Component | Environment | Auto | Manual | Owner |
|---|---|---|---|---|
| OE-CORE | Ubuntu 20.04 | Y | Intel |
BSP/QEMU
| Component | Arch | HW | Environment | Image | Auto | Manual | pTest | Compliance | ltp |
|---|---|---|---|---|---|---|---|---|---|
| BSP | x86 | MinwTurbot 32bit | genericx86 | core-image-sato-sdk | Y | ||||
| core-image-lsb-sdk | Y | ||||||||
| Coffee Lake | genericx86-64 | core-image-sato-sdk | Y | ||||||
| core-image-lsb-sdk | Y | ||||||||
| NUC 6 | genericx86-64 | core-image-sato-sdk | Y | Y | Y | ||||
| core-image-lsb-sdk | Y | Y | |||||||
| NUC 7 | genericx86-64 | core-image-sato-sdk | Y | Y | |||||
| core-image-lsb-sdk | Y | ||||||||
| QEMU | qemux86 | core-image-sato-sdk | Y | ||||||
| core-image-lsb-sdk | Y | ||||||||
| qemux86-64 | core-image-sato-sdk | Y | |||||||
| core-image-lsb-sdk | Y | ||||||||
| non x86 | EdgeRouter | EdgeRouter | core-image-sato-sdk | Y | Y | Y | |||
| Beaglebone Black | Beaglebone | core-image-sato-sdk | Y | Y | Y | ||||
| QEMU | qemuarm | core-image-sato-sdk | Y | ||||||
| qemuarm-64 | core-image-sato-sdk | Y | Y | Y | |||||
| qemuppc | core-image-sato-sdk | Y | |||||||
| qemumips | core-image-sato-sdk | Y | |||||||
| qemumips64 | core-image-sato-sdk | Y | |||||||
| Meta-Intel | MinwTurbot 32bit | core2-32 | core-image-sato-sdk | Y | |||||
| core-image-lsb-sdk | Y | ||||||||
| Coffee Lake | corei7-64 | core-image-sato-sdk | Y | ||||||
| core-image-lsb-sdk | Y | ||||||||
| NUC 6 | corei7-64 | core-image-sato-sdk | Y | ||||||
| core-image-lsb-sdk | Y | ||||||||
| NUC 7 | corei7-64 | core-image-sato-sdk | Y | Y | |||||
| core-image-lsb-sdk | Y | ||||||||
| poky-tiny | Y |
All component to be tested on 3.4 are in #Test Items a detailed list of features per component can be found in Test Plan of the components QA_Master_Test_Plan#Component
Approach
Testing will be done as per QA process defined in main QA wiki page at https://wiki.yoctoproject.org/wiki/QA#QA_Process.
Pass / Fail Criteria
This criteria is defined in Yocto_Project_v3.4_Status#Yocto_Project_v3.4_Release_Criteria
Exit Criteria
As defined in QA_Master_Test_Plan#Entry and Exit Criteria
Test Deliverables
- Final Test results
- Final Test report
- Mail sent to the mailing list
Schedule
The planed schedule for 3.4 can be found at Yocto_3.4_Schedule
Live Schedule & Execution History
| MILESTONE | STARTED | FINISHED | STATUS | REPORT |
|---|---|---|---|---|
| M1 | 06/12/2021 | 06/18/2021 | DONE | https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults |
| M2 | 07/26/2021 | 07/30/2021 | DONE | https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults |
| M3 | 09/06/2021 | In Progress | ||
| M4 |
