Yocto Project 3.1 Release Test Plan: Difference between revisions
From Yocto Project
Jump to navigationJump to search
No edit summary |
|||
| (2 intermediate revisions by the same user not shown) | |||
| Line 31: | Line 31: | ||
<!-- ********************* --> | <!-- ********************* --> | ||
= | = Revision history = | ||
{|class="wikitable" | {|class="wikitable" | ||
! Version || Modifier || Comments | ! Version || Modifier || Comments | ||
| Line 460: | Line 460: | ||
|- bgcolor="93DB70" | |- bgcolor="93DB70" | ||
| M2 | | M2 | ||
| | | 01/24/2020 | ||
| | | 01/31/2020 | ||
| | | DONE | ||
| https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults | | https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults | ||
|- bgcolor="93DB70" | |- bgcolor="93DB70" | ||
| M3 | | M3 | ||
| | | 03/16/2020 | ||
| | | 03/19/2020 | ||
| | | DONE | ||
| | | https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults | ||
|- bgcolor="93DB70" | |- bgcolor="93DB70" | ||
| M4 | | M4 | ||
| | | 04/08/2020 | ||
| | | 04/11/2020 | ||
| | | DONE | ||
| | | https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults | ||
|- bgcolor="93DB70" | |||
|} | |} | ||
Latest revision as of 08:41, 26 April 2020
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | 6564}}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | 6565}}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:KERNEL_MANUAL | 6714}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
Revision history
| Version | Modifier | Comments |
|---|---|---|
| 1.0 | Jain Sangeeta | First draft |
Introduction
This is the the overall test plan for version 3.1 of the Yocto Project, for all the generic information this document will be pointing to QA Master Test Plan
Objectives
This test plan for Yocto Project 3.1 release supports the following objectives:
- To define the tools to be used throughout the testing process.
- To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
- To define how the tests will be conducted.
- Release Criteria
- Risks and contingencies
QA_Master_Test_Plan#Objectives_and_Tasks
Risk Analysis & Contingency Plan
QA identified risks for this release are mentioned here.
| Module | Risk Description | Risk Level | Contingency plan | Owner |
|---|---|---|---|---|
| BSP | 46 manual test cases | medium | test cases to be automated | Intel |
| Build Appliance | 3 manual test cases, not executed by any QA team | medium | test cases to be automated | |
| Compliance | 4 manual test cases, not executed by any QA team | medium | test cases to be automated | |
| Kernel dev | 9 manual test cases, not executed by any QA team | medium | test cases to be automated | |
| OE-Core | 4 manual test cases | medium | test cases to be automated | Intel |
| SDK | 1 manual test case, not executed by any QA team | medium | test case to be automated | |
| Toaster | 93 manual test cases, not executed by any QA team | medium | test cases to be automated |
Test Strategy
Is followed the general strategy defined on QA_Master_Test_Plan#Testing_Strategy and below is showed specific information applicable only for 3.1 Release
Test Items
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7 * Debian 8 * Debian 9 * Fedora 28 * Fedora 29 * Fedora 30 * OpenSuse 150 * OpenSuse 151 * OpenSuse 423 * OpenSuse Tumbleweed * ubuntu 16.04 * ubuntu 18.04
Test Cases
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/runtime
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/selftest
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/manual
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/sdk
- http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/sdkext
OE-CORE
| Component | Environment | Auto | Manual | Owner |
|---|---|---|---|---|
| OE-CORE | Ubuntu 16.04 | Y | PG |
BSP/QEMU
| Component | Arch | HW | Environment | Image | Auto | Manual | pTest | Compliance | ltp |
|---|---|---|---|---|---|---|---|---|---|
| BSP | x86 | MinwTurbot 32bit | genericx86 | core-image-sato-sdk | Y | ||||
| core-image-lsb-sdk | Y | ||||||||
| Coffee Lake | genericx86-64 | core-image-sato-sdk | Y | Y | |||||
| core-image-lsb-sdk | Y | ||||||||
| NUC 6 | genericx86-64 | core-image-sato-sdk | Y | Y | Y | ||||
| core-image-lsb-sdk | Y | Y | |||||||
| NUC 7 | genericx86-64 | core-image-sato-sdk | Y | Y | |||||
| core-image-lsb-sdk | Y | ||||||||
| QEMU | qemux86 | core-image-sato-sdk | Y | Y | |||||
| core-image-lsb-sdk | Y | ||||||||
| qemux86-64 | core-image-sato-sdk | Y | Y | ||||||
| core-image-lsb-sdk | Y | ||||||||
| non x86 | EdgeRouter | EdgeRouter | core-image-sato-sdk | Y | Y | Y | |||
| MPC8315e-rdb | MPC8315e-rdb | core-image-sato-sdk | Y | Y | Y | ||||
| Beaglebone Black | Beaglebone | core-image-sato-sdk | Y | Y | Y | ||||
| QEMU | qemuarm | core-image-sato-sdk | Y | Y | |||||
| qemuarm-64 | core-image-sato-sdk | Y | Y | Y | Y | ||||
| qemuppc | core-image-sato-sdk | Y | Y | ||||||
| qemumips | core-image-sato-sdk | Y | Y | ||||||
| qemumips64 | core-image-sato-sdk | Y | Y | ||||||
| Meta-Intel | MinwTurbot 32bit | core2-32 | core-image-sato-sdk | Y | |||||
| core-image-lsb-sdk | Y | ||||||||
| Coffee Lake | corei7-64 | core-image-sato-sdk | Y | Y | |||||
| core-image-lsb-sdk | Y | ||||||||
| NUC 6 | corei7-64 | core-image-sato-sdk | Y | ||||||
| core-image-lsb-sdk | Y | ||||||||
| NUC 7 | corei7-64 | core-image-sato-sdk | Y | Y | |||||
| core-image-lsb-sdk | Y | ||||||||
| poky-tiny | Y |
All component to be tested on 2.8 are in #Test Items a detailed list of features per component can be found in Test Plan of the components QA_Master_Test_Plan#Component
Approach
Testing will be done as per QA process defined in main QA wiki page at https://wiki.yoctoproject.org/wiki/QA#QA_Process.
Pass / Fail Criteria
This criteria is defined in Yocto_Project_v3.1_Status#Yocto_Project_v3.1_Release_Criteria
Exit Criteria
As defined in QA_Master_Test_Plan#Entry and Exit Criteria
Test Deliverables
- Final Test results
- Final Test report
- Mail sent to the mailing list
Schedule
The planed schedule for 3.1 can be found at Yocto_3.1_Schedule
Live Schedule & Execution History
| MILESTONE | STARTED | FINISHED | STATUS | REPORT |
|---|---|---|---|---|
| M1 | 12/15/2019 | 12/18/2019 | DONE | https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults |
| M2 | 01/24/2020 | 01/31/2020 | DONE | https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults |
| M3 | 03/16/2020 | 03/19/2020 | DONE | https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults |
| M4 | 04/08/2020 | 04/11/2020 | DONE | https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults |
