Yocto Project 3.3 Release Test Plan: Difference between revisions

From Yocto Project
Jump to navigationJump to search
(Created page with "<!-- Variables for Template --> <!-- ********************** --> {| {{#vardefine:ADT_MAUAL | 6539}} {{#vardefine:ADT_AUTO | 6540}} {{#vardefine:ANYQEMU_S...")
 
 
(10 intermediate revisions by the same user not shown)
Line 42: Line 42:




This is the the overall test plan for version 3.2 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]
This is the the overall test plan for version 3.3 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]


= Objectives =
= Objectives =


This test plan for Yocto Project 3.2 release supports the following objectives:
This test plan for Yocto Project 3.3 release supports the following objectives:
*To define the tools to be used throughout the testing process.
*To define the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
Line 111: Line 111:
= Test Strategy =
= Test Strategy =


Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 3.2 Release
Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 3.3 Release


= Test Items =
= Test Items =
Line 189: Line 189:
| core-image-sato-sdk
| core-image-sato-sdk
| Y
| Y
| Y
|  
|  
|  
|  
|  
Line 237: Line 237:
| core-image-sato-sdk
| core-image-sato-sdk
| Y
| Y
| Y
|  
|  
|  
|  
|  
Line 252: Line 252:
| core-image-sato-sdk
| core-image-sato-sdk
| Y
| Y
| Y
|  
|  
|  
|  
|  
Line 264: Line 264:
|
|
|-
|-
| rowspan="8" style="text-align: center;" | non x86
| rowspan="7" style="text-align: center;" | non x86
| style="text-align: center;" | EdgeRouter
| style="text-align: center;" | EdgeRouter
| EdgeRouter
| EdgeRouter
| core-image-sato-sdk
| Y
| Y
|
| Y
|
|-
| style="text-align: center;" | MPC8315e-rdb
| MPC8315e-rdb
| core-image-sato-sdk
| core-image-sato-sdk
| Y
| Y
Line 296: Line 287:
| core-image-sato-sdk
| core-image-sato-sdk
| Y
| Y
| Y
|  
|  
|  
|  
|  
Line 304: Line 295:
| core-image-sato-sdk
| core-image-sato-sdk
| Y
| Y
| Y
|  
| Y
| Y
|  
|  
Line 312: Line 303:
| core-image-sato-sdk
| core-image-sato-sdk
| Y
| Y
| Y
|  
|  
|  
|  
|  
Line 320: Line 311:
| core-image-sato-sdk
| core-image-sato-sdk
| Y
| Y
| Y
|  
|  
|  
|  
|  
Line 328: Line 319:
| core-image-sato-sdk
| core-image-sato-sdk
| Y
| Y
| Y
|  
|  
|  
|  
|  
Line 354: Line 345:
| core-image-sato-sdk
| core-image-sato-sdk
| Y
| Y
| Y
|  
|  
|  
|  
|  
Line 452: Line 443:
! STATUS
! STATUS
! REPORT
! REPORT
|- bgcolor="FFD700"
|- bgcolor="93DB70"
| M1  
| M1  
|  
| 12/11/2020
|  
| 12/15/2020
| In Queue
| DONE
|  
| https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults
|- bgcolor="FFD700"
|- bgcolor="93DB70"
| M2  
| M2  
|  
| 01/21/2021
|  
| 01/27/2021
|  In Queue
|  DONE
|  
| https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults
|- bgcolor="FFD700"
|- bgcolor="93DB70"
| M3  
| M3  
|  
| 03/16/2021
|  
| 03/26/2021
| In Queue
| DONE
|  
| https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults
|- bgcolor="FFD700"
|- bgcolor="93DB70"
| M4  
| M4  
|  
| 04/05/2021
|  
| 04/20/2021
|  In Queue
|  DONE
|  
| https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults
|- bgcolor="FFD700"
|- bgcolor="FFD700"
|}
|}

Latest revision as of 09:52, 13 July 2021

{{#vardefine:ADT_MAUAL | 6539}} {{#vardefine:ADT_AUTO | 6540}} {{#vardefine:ANYQEMU_SATO_MANUAL | 6563}} {{#vardefine:ANYQEMU_SATO_AUTO | 6564}} {{#vardefine:ANYBSP_SATO_AUTO | 6558}} {{#vardefine:ANYBSP_LSB_AUTO | 6561}} {{#vardefine:ANYBSP_WIC_AUTO | 6560}} {{#vardefine:MMAX_CORE2_MANUAL | 6554}} {{#vardefine:MMAX_I7_MANUAL | 6557}} {{#vardefine:NUC_I7_MANUAL | 6556}} {{#vardefine:NUC_GEX86-64_MANUAL | 6555}} {{#vardefine:MMAX_GEX86-32_MANUAL | 6553}} {{#vardefine:GAL_SATO_AUTO | 6565}} {{#vardefine:CROPS_AUTO | 6541}} {{#vardefine:CROPS_MANUAL | 6542}} {{#vardefine:ECLIPSE_MANUAL | 6543}} {{#vardefine:KERNEL_MANUAL | 6714}} {{#vardefine:METAYOCTO_MANUAL | 6544}} {{#vardefine:TOASTER_AUTO | 6545}} {{#vardefine:TOASTER_MANUAL | 6546}} {{#vardefine:ESDK_MANUAL | 6547}} {{#vardefine:ESDK_AUTO | 6548}} {{#vardefine:OE-CORE_AUTO | 6549}} {{#vardefine:BITBAKE_AUTO | 6550}} {{#vardefine:RUNTIME_AUTO | 6551}} {{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}

Revision history

Version Modifier Comments
1.0 Jain Sangeeta First draft

Introduction

This is the the overall test plan for version 3.3 of the Yocto Project, for all the generic information this document will be pointing to QA Master Test Plan

Objectives

This test plan for Yocto Project 3.3 release supports the following objectives:

  • To define the tools to be used throughout the testing process.
  • To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
  • To define how the tests will be conducted.
  • Release Criteria
  • Risks and contingencies

QA_Master_Test_Plan#Objectives_and_Tasks

Risk Analysis & Contingency Plan

QA identified risks for this release are mentioned here.

Module Risk Description Risk Level Contingency plan Owner
BSP 31 manual test cases medium test cases to be automated Intel
Build Appliance 3 manual test cases, not executed by any QA team medium test cases to be automated
Compliance 4 manual test cases, not executed by any QA team medium test cases to be automated
Kernel dev 9 manual test cases, not executed by any QA team medium test cases to be automated
OE-Core 4 manual test cases medium test cases to be automated Intel
SDK 1 manual test case, not executed by any QA team medium test case to be automated
Toaster 93 manual test cases, not executed by any QA team medium test cases to be automated

Test Strategy

Is followed the general strategy defined on QA_Master_Test_Plan#Testing_Strategy and below is showed specific information applicable only for 3.3 Release

Test Items

The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases. 

* centos 7 
* Debian 8
* Debian 9
* Fedora 28
* Fedora 29
* Fedora 30
* OpenSuse 150
* OpenSuse 151
* OpenSuse 423
* OpenSuse Tumbleweed
* ubuntu 16.04 
* ubuntu 18.04

Test Cases

OE-CORE

Component Environment Auto Manual Owner
OE-CORE Ubuntu 20.04 Y Intel

BSP/QEMU

Component Arch HW Environment Image Auto Manual pTest Compliance ltp
BSP x86 MinwTurbot 32bit genericx86 core-image-sato-sdk Y
core-image-lsb-sdk Y
Coffee Lake genericx86-64 core-image-sato-sdk Y
core-image-lsb-sdk Y
NUC 6 genericx86-64 core-image-sato-sdk Y Y Y
core-image-lsb-sdk Y Y
NUC 7 genericx86-64 core-image-sato-sdk Y Y
core-image-lsb-sdk Y
QEMU qemux86 core-image-sato-sdk Y
core-image-lsb-sdk Y
qemux86-64 core-image-sato-sdk Y
core-image-lsb-sdk Y
non x86 EdgeRouter EdgeRouter core-image-sato-sdk Y Y Y
Beaglebone Black Beaglebone core-image-sato-sdk Y Y Y
QEMU qemuarm core-image-sato-sdk Y
qemuarm-64 core-image-sato-sdk Y Y Y
qemuppc core-image-sato-sdk Y
qemumips core-image-sato-sdk Y
qemumips64 core-image-sato-sdk Y
Meta-Intel MinwTurbot 32bit core2-32 core-image-sato-sdk Y
core-image-lsb-sdk Y
Coffee Lake corei7-64 core-image-sato-sdk Y
core-image-lsb-sdk Y
NUC 6 corei7-64 core-image-sato-sdk Y
core-image-lsb-sdk Y
NUC 7 corei7-64 core-image-sato-sdk Y Y
core-image-lsb-sdk Y
poky-tiny Y

All component to be tested on 3.3 are in #Test Items a detailed list of features per component can be found in Test Plan of the components QA_Master_Test_Plan#Component

Approach

Testing will be done as per QA process defined in main QA wiki page at https://wiki.yoctoproject.org/wiki/QA#QA_Process.

Pass / Fail Criteria

This criteria is defined in Yocto_Project_v3.3_Status#Yocto_Project_v3.3_Release_Criteria

Exit Criteria

As defined in QA_Master_Test_Plan#Entry and Exit Criteria

Test Deliverables

  • Final Test results
  • Final Test report
  • Mail sent to the mailing list

Schedule

The planed schedule for 3.3 can be found at Yocto_3.3_Schedule

Live Schedule & Execution History

MILESTONE STARTED FINISHED STATUS REPORT
M1 12/11/2020 12/15/2020 DONE https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults
M2 01/21/2021 01/27/2021 DONE https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults
M3 03/16/2021 03/26/2021 DONE https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults
M4 04/05/2021 04/20/2021 DONE https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/log/?h=intel-yocto-testresults
Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Yocto_Project_3.3_Release_Test_Plan&oldid=84822"