Yocto Project 3.3 Release Test Plan: Difference between revisions
From Yocto Project
Jump to navigationJump to search
| Line 189: | Line 189: | ||
| core-image-sato-sdk | | core-image-sato-sdk | ||
| Y | | Y | ||
| | | | ||
| | | | ||
| | | | ||
| Line 237: | Line 237: | ||
| core-image-sato-sdk | | core-image-sato-sdk | ||
| Y | | Y | ||
| | | | ||
| | | | ||
| | | | ||
| Line 252: | Line 252: | ||
| core-image-sato-sdk | | core-image-sato-sdk | ||
| Y | | Y | ||
| | | | ||
| | | | ||
| | | | ||
| Line 267: | Line 267: | ||
| style="text-align: center;" | EdgeRouter | | style="text-align: center;" | EdgeRouter | ||
| EdgeRouter | | EdgeRouter | ||
| core-image-sato-sdk | | core-image-sato-sdk | ||
| Y | | Y | ||
| Line 296: | Line 287: | ||
| core-image-sato-sdk | | core-image-sato-sdk | ||
| Y | | Y | ||
| | | | ||
| | | | ||
| | | | ||
| Line 304: | Line 295: | ||
| core-image-sato-sdk | | core-image-sato-sdk | ||
| Y | | Y | ||
| | | | ||
| Y | | Y | ||
| | | | ||
| Line 312: | Line 303: | ||
| core-image-sato-sdk | | core-image-sato-sdk | ||
| Y | | Y | ||
| | | | ||
| | | | ||
| | | | ||
| Line 320: | Line 311: | ||
| core-image-sato-sdk | | core-image-sato-sdk | ||
| Y | | Y | ||
| | | | ||
| | | | ||
| | | | ||
| Line 328: | Line 319: | ||
| core-image-sato-sdk | | core-image-sato-sdk | ||
| Y | | Y | ||
| | | | ||
| | | | ||
| | | | ||
| Line 354: | Line 345: | ||
| core-image-sato-sdk | | core-image-sato-sdk | ||
| Y | | Y | ||
| | | | ||
| | | | ||
| | | | ||
Revision as of 07:55, 4 December 2020
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | 6564}}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | 6565}}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:KERNEL_MANUAL | 6714}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
Revision history
| Version | Modifier | Comments |
|---|---|---|
| 1.0 | Jain Sangeeta | First draft |
Introduction
This is the the overall test plan for version 3.3 of the Yocto Project, for all the generic information this document will be pointing to QA Master Test Plan
Objectives
This test plan for Yocto Project 3.3 release supports the following objectives:
- To define the tools to be used throughout the testing process.
- To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
- To define how the tests will be conducted.
- Release Criteria
- Risks and contingencies
QA_Master_Test_Plan#Objectives_and_Tasks
Risk Analysis & Contingency Plan
QA identified risks for this release are mentioned here.
| Module | Risk Description | Risk Level | Contingency plan | Owner |
|---|---|---|---|---|
| BSP | 31 manual test cases | medium | test cases to be automated | Intel |
| Build Appliance | 3 manual test cases, not executed by any QA team | medium | test cases to be automated | |
| Compliance | 4 manual test cases, not executed by any QA team | medium | test cases to be automated | |
| Kernel dev | 9 manual test cases, not executed by any QA team | medium | test cases to be automated | |
| OE-Core | 4 manual test cases | medium | test cases to be automated | Intel |
| SDK | 1 manual test case, not executed by any QA team | medium | test case to be automated | |
| Toaster | 93 manual test cases, not executed by any QA team | medium | test cases to be automated |
Test Strategy
Is followed the general strategy defined on QA_Master_Test_Plan#Testing_Strategy and below is showed specific information applicable only for 3.3 Release
Test Items
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7 * Debian 8 * Debian 9 * Fedora 28 * Fedora 29 * Fedora 30 * OpenSuse 150 * OpenSuse 151 * OpenSuse 423 * OpenSuse Tumbleweed * ubuntu 16.04 * ubuntu 18.04
Test Cases
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/runtime
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/selftest
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/manual
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/sdk
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/sdkext
OE-CORE
| Component | Environment | Auto | Manual | Owner |
|---|---|---|---|---|
| OE-CORE | Ubuntu 20.04 | Y | Intel |
BSP/QEMU
| Component | Arch | HW | Environment | Image | Auto | Manual | pTest | Compliance | ltp |
|---|---|---|---|---|---|---|---|---|---|
| BSP | x86 | MinwTurbot 32bit | genericx86 | core-image-sato-sdk | Y | ||||
| core-image-lsb-sdk | Y | ||||||||
| Coffee Lake | genericx86-64 | core-image-sato-sdk | Y | ||||||
| core-image-lsb-sdk | Y | ||||||||
| NUC 6 | genericx86-64 | core-image-sato-sdk | Y | Y | Y | ||||
| core-image-lsb-sdk | Y | Y | |||||||
| NUC 7 | genericx86-64 | core-image-sato-sdk | Y | Y | |||||
| core-image-lsb-sdk | Y | ||||||||
| QEMU | qemux86 | core-image-sato-sdk | Y | ||||||
| core-image-lsb-sdk | Y | ||||||||
| qemux86-64 | core-image-sato-sdk | Y | |||||||
| core-image-lsb-sdk | Y | ||||||||
| non x86 | EdgeRouter | EdgeRouter | core-image-sato-sdk | Y | Y | Y | |||
| Beaglebone Black | Beaglebone | core-image-sato-sdk | Y | Y | Y | ||||
| QEMU | qemuarm | core-image-sato-sdk | Y | ||||||
| qemuarm-64 | core-image-sato-sdk | Y | Y | Y | |||||
| qemuppc | core-image-sato-sdk | Y | |||||||
| qemumips | core-image-sato-sdk | Y | |||||||
| qemumips64 | core-image-sato-sdk | Y | |||||||
| Meta-Intel | MinwTurbot 32bit | core2-32 | core-image-sato-sdk | Y | |||||
| core-image-lsb-sdk | Y | ||||||||
| Coffee Lake | corei7-64 | core-image-sato-sdk | Y | ||||||
| core-image-lsb-sdk | Y | ||||||||
| NUC 6 | corei7-64 | core-image-sato-sdk | Y | ||||||
| core-image-lsb-sdk | Y | ||||||||
| NUC 7 | corei7-64 | core-image-sato-sdk | Y | Y | |||||
| core-image-lsb-sdk | Y | ||||||||
| poky-tiny | Y |
All component to be tested on 3.3 are in #Test Items a detailed list of features per component can be found in Test Plan of the components QA_Master_Test_Plan#Component
Approach
Testing will be done as per QA process defined in main QA wiki page at https://wiki.yoctoproject.org/wiki/QA#QA_Process.
Pass / Fail Criteria
This criteria is defined in Yocto_Project_v3.3_Status#Yocto_Project_v3.3_Release_Criteria
Exit Criteria
As defined in QA_Master_Test_Plan#Entry and Exit Criteria
Test Deliverables
- Final Test results
- Final Test report
- Mail sent to the mailing list
Schedule
The planed schedule for 3.3 can be found at Yocto_3.3_Schedule
Live Schedule & Execution History
| MILESTONE | STARTED | FINISHED | STATUS | REPORT |
|---|---|---|---|---|
| M1 | In Queue | |||
| M2 | In Queue | |||
| M3 | In Queue | |||
| M4 | In Queue |
