Yocto Project - User contributions [en]

Yocto Project 2.3 Release Test Plan

2016-12-21T19:55:47Z

Sonia Delgado: /* Features pending to be tested */

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | 6564}}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | 6565}}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This is the the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]

= Objectives =

This test plan for Yocto Project 2.3 release supports the following objectives:
*To define the the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7
* Debian 8
* Fedora 23
* Fedora 24
* OpenSuse132
* OpenSuse421
* ubuntu 14.04
* ubuntu 16.04

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Components to be tested =

The components to be tested for version 2.3 of the {{ns:4}} against the different milestones are included in this section:

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.
{| class="wikitable"
!colspan="2"|Features
|-
! style="border-style: solid; border-width: 0 1px 1px 0"| Bug ID
! style="border-style: solid; border-width: 0 0 1px 0"| Description
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10723 Bug 10723]
|Build-Appliance - Run builds in the build-appliance image
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify if -c menuconfig work
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify if -c devshell work
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10725 Bug 10725]
|Verify if BBPOSTCONF and BBPRECONF work in bitbakek
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10726 Bug 10726]
|Verify if oe-pkgdata-util print help when no parameters specified
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10727 Bug 10727]
|Verify if buildhistory_analsysis.py is tested
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10728 Bug 10728]
|Verify combo-layer
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=6926 Bug 6926]
|Verify yocto-bsp/yocto-layer
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify bitbake -u depexp -g XXX
|}
The complete List of QA Implementations for YP 2.3 can be found at [https://wiki.yoctoproject.org/wiki/2.3_QA_OWNED_BUGS Features Implementation by QA Team]

= Features Not to be tested =
*

=Approach=
Tests will be conducted per the documented test cases stored in Testopia. The test manager will create test runs for each tester. The tester will execute the tests in Testopia and mark each case as Pass / Fail / Skip /Blocked.
The tester should leave notes on actual results and any other relevant details when possible.

Once complete, the test manager should review the test run reports in Testopia and report back to the team by mail and in the wiki as a test report.

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =
== M1 (Oct. 31, 2016 to Dec. 23, 2016 -- 9 weeks: planning done in week 1, development week 2-6, stabilization week 7-8, release week 9) ==
=== M1 Feature/Tasks Development / Cutoff is Dec. 12, 2016 ===

{{#bugzilla:
|columns=id,from,to,estimated,summary,severity,priority,whiteboard,status
|total=estimated
|milestone=2.3 M1
|severity=enhancement
|sort=to,priority
|to=corneliux.stoicescu@intel.com,alexandru.c.georgescu@intel.com,ionutx.chisanovici@intel.com,georgex.l.musat@intel.com,stanciux.mihail@intel.com,andrax.g.necula@intel.com,costin.c.constantin@intel.com,daniel.alexandrux.istrate@intel.com,alexandru.costinx.roman@intel.com,valentinx.hangan@intel.com,libertad.gonzalez.de.la.cruz@intel.com,francisco.j.pedraza.gonzalez@intel.com,jose.perez.carranza@intel.com,humberto.ibarra.lopez@intel.com,david.israelx.rodriguez.castellanos@intel.com,davidx.lopez.barriba@intel.com,juan.fernandox.ramos.frayle@intel.com,sonia.delgado@intel.com,jair.de.jesus.gonzalez.plascencia@intel.com
}}

=== M1 Bugs ===

{{#bugzilla:
|columns=id,from,to,estimated,summary,severity,priority,whiteboard,status
|total=estimated
|milestone=2.3 M1
|severity=!enhancement
|sort=to,priority
|to=corneliux.stoicescu@intel.com,alexandru.c.georgescu@intel.com,ionutx.chisanovici@intel.com,georgex.l.musat@intel.com,stanciux.mihail@intel.com,andrax.g.necula@intel.com,costin.c.constantin@intel.com,daniel.alexandrux.istrate@intel.com,alexandru.costinx.roman@intel.com,valentinx.hangan@intel.com,libertad.gonzalez.de.la.cruz@intel.com,francisco.j.pedraza.gonzalez@intel.com,jose.perez.carranza@intel.com,humberto.ibarra.lopez@intel.com,david.israelx.rodriguez.castellanos@intel.com,davidx.lopez.barriba@intel.com,juan.fernandox.ramos.frayle@intel.com,sonia.delgado@intel.com,jair.de.jesus.gonzalez.plascencia@intel.com
}}

=== M1 Stabilize schedule ===
'''Release Candidate and Milestone Release schedule'''
* RC1: 12/12/16
* M1 release: 12/23/16

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
| 12/19/2016
|
| [https://wiki.yoctoproject.org/wiki/2.3_QA_Status Live Status]
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-12-21T19:54:59Z

Sonia Delgado: /* Features pending to be tested */

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | 6564}}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | 6565}}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This is the the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]

= Objectives =

This test plan for Yocto Project 2.3 release supports the following objectives:
*To define the the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7
* Debian 8
* Fedora 23
* Fedora 24
* OpenSuse132
* OpenSuse421
* ubuntu 14.04
* ubuntu 16.04

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Components to be tested =

The components to be tested for version 2.3 of the {{ns:4}} against the different milestones are included in this section:

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.
{| class="wikitable"
!colspan="2"|Features
|-
! style="border-style: solid; border-width: 0 1px 1px 0"| Bug ID
! style="border-style: solid; border-width: 0 0 1px 0"| Description
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10723 Bug 10723]
|Build-Appliance - Run builds in the build-appliance image
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify if -c menuconfig work
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify if -c devshell work
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10725 Bug 10725]
|Verify if BBPOSTCONF and BBPRECONF work in bitbakek
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10726 Bug 10726]
|Verify if oe-pkgdata-util print help when no parameters specified
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10727 Bug 10727]
|Verify if buildhistory_analsysis.py is tested
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10728 Bug 10728]
|Verify combo-layer
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=6926 Bug 6926]
|Verify yocto-bsp/yocto-layer
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify bitbake -u depexp -g XXX
|}

The complete List of QA Implementations for YP 2.3 can be found at [https://wiki.yoctoproject.org/wiki/2.3_QA_OWNED_BUGS Features Implementation by QA Team]

= Features Not to be tested =
*

=Approach=
Tests will be conducted per the documented test cases stored in Testopia. The test manager will create test runs for each tester. The tester will execute the tests in Testopia and mark each case as Pass / Fail / Skip /Blocked.
The tester should leave notes on actual results and any other relevant details when possible.

Once complete, the test manager should review the test run reports in Testopia and report back to the team by mail and in the wiki as a test report.

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =
== M1 (Oct. 31, 2016 to Dec. 23, 2016 -- 9 weeks: planning done in week 1, development week 2-6, stabilization week 7-8, release week 9) ==
=== M1 Feature/Tasks Development / Cutoff is Dec. 12, 2016 ===

{{#bugzilla:
|columns=id,from,to,estimated,summary,severity,priority,whiteboard,status
|total=estimated
|milestone=2.3 M1
|severity=enhancement
|sort=to,priority
|to=corneliux.stoicescu@intel.com,alexandru.c.georgescu@intel.com,ionutx.chisanovici@intel.com,georgex.l.musat@intel.com,stanciux.mihail@intel.com,andrax.g.necula@intel.com,costin.c.constantin@intel.com,daniel.alexandrux.istrate@intel.com,alexandru.costinx.roman@intel.com,valentinx.hangan@intel.com,libertad.gonzalez.de.la.cruz@intel.com,francisco.j.pedraza.gonzalez@intel.com,jose.perez.carranza@intel.com,humberto.ibarra.lopez@intel.com,david.israelx.rodriguez.castellanos@intel.com,davidx.lopez.barriba@intel.com,juan.fernandox.ramos.frayle@intel.com,sonia.delgado@intel.com,jair.de.jesus.gonzalez.plascencia@intel.com
}}

=== M1 Bugs ===

{{#bugzilla:
|columns=id,from,to,estimated,summary,severity,priority,whiteboard,status
|total=estimated
|milestone=2.3 M1
|severity=!enhancement
|sort=to,priority
|to=corneliux.stoicescu@intel.com,alexandru.c.georgescu@intel.com,ionutx.chisanovici@intel.com,georgex.l.musat@intel.com,stanciux.mihail@intel.com,andrax.g.necula@intel.com,costin.c.constantin@intel.com,daniel.alexandrux.istrate@intel.com,alexandru.costinx.roman@intel.com,valentinx.hangan@intel.com,libertad.gonzalez.de.la.cruz@intel.com,francisco.j.pedraza.gonzalez@intel.com,jose.perez.carranza@intel.com,humberto.ibarra.lopez@intel.com,david.israelx.rodriguez.castellanos@intel.com,davidx.lopez.barriba@intel.com,juan.fernandox.ramos.frayle@intel.com,sonia.delgado@intel.com,jair.de.jesus.gonzalez.plascencia@intel.com
}}

=== M1 Stabilize schedule ===
'''Release Candidate and Milestone Release schedule'''
* RC1: 12/12/16
* M1 release: 12/23/16

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
| 12/19/2016
|
| [https://wiki.yoctoproject.org/wiki/2.3_QA_Status Live Status]
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-12-21T19:33:37Z

Sonia Delgado: /* Features to be tested */

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | 6564}}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | 6565}}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This is the the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]

= Objectives =

This test plan for Yocto Project 2.3 release supports the following objectives:
*To define the the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7
* Debian 8
* Fedora 23
* Fedora 24
* OpenSuse132
* OpenSuse421
* ubuntu 14.04
* ubuntu 16.04

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Components to be tested =

The components to be tested for version 2.3 of the {{ns:4}} against the different milestones are included in this section:

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.
{| class="wikitable"
!colspan="2"|Features
|-
! style="border-style: solid; border-width: 0 1px 1px 0"| Bug ID
! style="border-style: solid; border-width: 0 0 1px 0"| Description
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10723 Bug 10723]
|Build-Appliance - Run builds in the build-appliance image
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify if -c menuconfig work
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify if -c devshell work
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10725 Bug 10725]
|Verify if BBPOSTCONF and BBPRECONF work in bitbakek
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10726 Bug 10726]
|Verify if oe-pkgdata-util print help when no parameters specified
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10727 Bug 10727]
|Verify if buildhistory_analsysis.py is tested
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10728 Bug 10728]
|Verify combo-layer
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=6926 Bug 6926]
|Verify yocto-bsp/yocto-layer
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify bitbake -u depexp -g XXX
|}

= Features Not to be tested =
*

=Approach=
Tests will be conducted per the documented test cases stored in Testopia. The test manager will create test runs for each tester. The tester will execute the tests in Testopia and mark each case as Pass / Fail / Skip /Blocked.
The tester should leave notes on actual results and any other relevant details when possible.

Once complete, the test manager should review the test run reports in Testopia and report back to the team by mail and in the wiki as a test report.

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =
== M1 (Oct. 31, 2016 to Dec. 23, 2016 -- 9 weeks: planning done in week 1, development week 2-6, stabilization week 7-8, release week 9) ==
=== M1 Feature/Tasks Development / Cutoff is Dec. 12, 2016 ===

{{#bugzilla:
|columns=id,from,to,estimated,summary,severity,priority,whiteboard,status
|total=estimated
|milestone=2.3 M1
|severity=enhancement
|sort=to,priority
|to=corneliux.stoicescu@intel.com,alexandru.c.georgescu@intel.com,ionutx.chisanovici@intel.com,georgex.l.musat@intel.com,stanciux.mihail@intel.com,andrax.g.necula@intel.com,costin.c.constantin@intel.com,daniel.alexandrux.istrate@intel.com,alexandru.costinx.roman@intel.com,valentinx.hangan@intel.com,libertad.gonzalez.de.la.cruz@intel.com,francisco.j.pedraza.gonzalez@intel.com,jose.perez.carranza@intel.com,humberto.ibarra.lopez@intel.com,david.israelx.rodriguez.castellanos@intel.com,davidx.lopez.barriba@intel.com,juan.fernandox.ramos.frayle@intel.com,sonia.delgado@intel.com,jair.de.jesus.gonzalez.plascencia@intel.com
}}

=== M1 Bugs ===

{{#bugzilla:
|columns=id,from,to,estimated,summary,severity,priority,whiteboard,status
|total=estimated
|milestone=2.3 M1
|severity=!enhancement
|sort=to,priority
|to=corneliux.stoicescu@intel.com,alexandru.c.georgescu@intel.com,ionutx.chisanovici@intel.com,georgex.l.musat@intel.com,stanciux.mihail@intel.com,andrax.g.necula@intel.com,costin.c.constantin@intel.com,daniel.alexandrux.istrate@intel.com,alexandru.costinx.roman@intel.com,valentinx.hangan@intel.com,libertad.gonzalez.de.la.cruz@intel.com,francisco.j.pedraza.gonzalez@intel.com,jose.perez.carranza@intel.com,humberto.ibarra.lopez@intel.com,david.israelx.rodriguez.castellanos@intel.com,davidx.lopez.barriba@intel.com,juan.fernandox.ramos.frayle@intel.com,sonia.delgado@intel.com,jair.de.jesus.gonzalez.plascencia@intel.com
}}

=== M1 Stabilize schedule ===
'''Release Candidate and Milestone Release schedule'''
* RC1: 12/12/16
* M1 release: 12/23/16

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
| 12/19/2016
|
| [https://wiki.yoctoproject.org/wiki/2.3_QA_Status Live Status]
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-12-21T19:31:20Z

Sonia Delgado: /* Schedule */

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | 6564}}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | 6565}}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This is the the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]

= Objectives =

This test plan for Yocto Project 2.3 release supports the following objectives:
*To define the the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7
* Debian 8
* Fedora 23
* Fedora 24
* OpenSuse132
* OpenSuse421
* ubuntu 14.04
* ubuntu 16.04

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

The components to be tested for version 2.3 of the {{ns:4}} against the different milestones are included in this section:

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.
{| class="wikitable"
!colspan="2"|Features
|-
! style="border-style: solid; border-width: 0 1px 1px 0"| Bug ID
! style="border-style: solid; border-width: 0 0 1px 0"| Description
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10723 Bug 10723]
|Build-Appliance - Run builds in the build-appliance image
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify if -c menuconfig work
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify if -c devshell work
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10725 Bug 10725]
|Verify if BBPOSTCONF and BBPRECONF work in bitbakek
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10726 Bug 10726]
|Verify if oe-pkgdata-util print help when no parameters specified
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10727 Bug 10727]
|Verify if buildhistory_analsysis.py is tested
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10728 Bug 10728]
|Verify combo-layer
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=6926 Bug 6926]
|Verify yocto-bsp/yocto-layer
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify bitbake -u depexp -g XXX
|}

= Features Not to be tested =
*

=Approach=
Tests will be conducted per the documented test cases stored in Testopia. The test manager will create test runs for each tester. The tester will execute the tests in Testopia and mark each case as Pass / Fail / Skip /Blocked.
The tester should leave notes on actual results and any other relevant details when possible.

Once complete, the test manager should review the test run reports in Testopia and report back to the team by mail and in the wiki as a test report.

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =
== M1 (Oct. 31, 2016 to Dec. 23, 2016 -- 9 weeks: planning done in week 1, development week 2-6, stabilization week 7-8, release week 9) ==
=== M1 Feature/Tasks Development / Cutoff is Dec. 12, 2016 ===

{{#bugzilla:
|columns=id,from,to,estimated,summary,severity,priority,whiteboard,status
|total=estimated
|milestone=2.3 M1
|severity=enhancement
|sort=to,priority
|to=corneliux.stoicescu@intel.com,alexandru.c.georgescu@intel.com,ionutx.chisanovici@intel.com,georgex.l.musat@intel.com,stanciux.mihail@intel.com,andrax.g.necula@intel.com,costin.c.constantin@intel.com,daniel.alexandrux.istrate@intel.com,alexandru.costinx.roman@intel.com,valentinx.hangan@intel.com,libertad.gonzalez.de.la.cruz@intel.com,francisco.j.pedraza.gonzalez@intel.com,jose.perez.carranza@intel.com,humberto.ibarra.lopez@intel.com,david.israelx.rodriguez.castellanos@intel.com,davidx.lopez.barriba@intel.com,juan.fernandox.ramos.frayle@intel.com,sonia.delgado@intel.com,jair.de.jesus.gonzalez.plascencia@intel.com
}}

=== M1 Bugs ===

{{#bugzilla:
|columns=id,from,to,estimated,summary,severity,priority,whiteboard,status
|total=estimated
|milestone=2.3 M1
|severity=!enhancement
|sort=to,priority
|to=corneliux.stoicescu@intel.com,alexandru.c.georgescu@intel.com,ionutx.chisanovici@intel.com,georgex.l.musat@intel.com,stanciux.mihail@intel.com,andrax.g.necula@intel.com,costin.c.constantin@intel.com,daniel.alexandrux.istrate@intel.com,alexandru.costinx.roman@intel.com,valentinx.hangan@intel.com,libertad.gonzalez.de.la.cruz@intel.com,francisco.j.pedraza.gonzalez@intel.com,jose.perez.carranza@intel.com,humberto.ibarra.lopez@intel.com,david.israelx.rodriguez.castellanos@intel.com,davidx.lopez.barriba@intel.com,juan.fernandox.ramos.frayle@intel.com,sonia.delgado@intel.com,jair.de.jesus.gonzalez.plascencia@intel.com
}}

=== M1 Stabilize schedule ===
'''Release Candidate and Milestone Release schedule'''
* RC1: 12/12/16
* M1 release: 12/23/16

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
| 12/19/2016
|
| [https://wiki.yoctoproject.org/wiki/2.3_QA_Status Live Status]
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

2.3 QA Status

2016-12-21T19:15:43Z

Sonia Delgado: /* Testing data */

{|
{{#vardefine:build| }}
{{#vardefine:test_date| max }} 
{{#vardefine:test_type| }} 
{{#vardefine:testplan| master }} 


{{#vardefine:bitbake| . }}
{{#vardefine:meta-yocto| . }}
{{#vardefine:oe-core| . }}
{{#vardefine:eSDK| . }}
{{#vardefine:minnowboard| . }}
{{#vardefine:nuc| . }}
{{#vardefine:emenlow| . }}
{{#vardefine:sugarbay| . }}
{{#vardefine:jasperforest| . }}
{{#vardefine:crownbay| . }}
{{#vardefine:fri2| . }}
{{#vardefine:beagleboard| . }}
{{#vardefine:routerstationpro| . }}
{{#vardefine:mpc8315e-rdb| . }}
{{#vardefine:p1022ds-rdb| . }}
{{#vardefine:qemux86| . }}
{{#vardefine:qemux86-64| . }}
{{#vardefine:genericx86| . }}
{{#vardefine:genericx86-64| . }}
{{#vardefine:qemuarm| . }}
{{#vardefine:qemumips| . }}
{{#vardefine:qemuppc| . }}
{{#vardefine:runtime| . }}
{{#vardefine:build-appliance| . }}
{{#vardefine:adt_fedora17_i686| . }}
{{#vardefine:adt_fedora17_x86-64| . }}
{{#vardefine:adt_ubuntu_1204_x86-64|. }}
{{#vardefine:adt_opensuse_122_x86-64| . }}
{{#vardefine:adt_opensuse_123_x86-64| . }}
{{#vardefine:eclipseplugin| . }}
{{#vardefine:adt_ubuntu_1210_x86-64|. }}
{{#vardefine:adt_fedora18_i686| . }}
{{#vardefine:adt_fedora19_i686| . }}
{{#vardefine:adt_ubuntu_1304_x86-64|. }}
|}

=Testing data=

This is a live status report. The information contained is relevant to the latest test suite performed for Yocto 2.3.x .

=Summary Report=
{|
|{{#testopia:
|run_id=-1
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=concatstats}}
|{{#testopia:
|run_id=-1
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|charttype=pie3
|report_id=status}}
|}

{{#testopia:
|run_id=-1
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus
|charttype=bar
|sortable=true
|chartpos=top}}

=Detailed Report=
{{#if: {{#var:bitbake}}{{#var:oe-core}}{{#var:meta-yocto}} |
==Core Build System==
{{#if: {{#var:bitbake}} |
===Bitbake===
Status
{{#testopia:
|run_id=-1
|product=bitbake
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=bitbake
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=bitbake
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true}}
| }}
{{#if: {{#var:meta-yocto}} |
===Meta-Yocto===
Status
{{#testopia:
|run_id=-1
|product=meta-yocto
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=meta-yocto
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=meta-yocto
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:oe-core}} |
===OE-Core===
Status
{{#testopia:
|run_id=-1
|product=oe-core
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=oe-core
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=oe-core
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
| }}
{{#if: {{#var:eSDK}} |
==eSDK==
Status
{{#testopia:
|run_id=-1
|product=eSDK
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=eSDK
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=eSDK
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:minnowboard}}{{#var:nuc}}{{#var:emenlow}}{{#var:sugarbay}}{{#var:jasperforest}}{{#var:crownbay}}{{#var:fri2}}{{#var:beagleboard}}{{#var:routerstationpro}}{{#var:mpc8315e-rdb}}{{#var:p1022ds-rdb}} |
==BSP==

{{#if: {{#var:genericx86}} |
===Genericx86: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=generic-x86 on AtomPC
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=generic-x86 on AtomPC
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=generic-x86 on AtomPC
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:genericx86-64}} |
===Genericx86-64: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=genericx86-64 on Sugarbay
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=genericx86-64 on Sugarbay
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=genericx86-64 on Sugarbay
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:minnowboard}} |
===MinnowBoard: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=MinnowBoard
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=MinnowBoard
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=MinnowBoard
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:nuc}} |
===NUC: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=nuc
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=nuc
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=nuc
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:emenlow}} |
===Emenlow: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=emenlow
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=emenlow
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=emenlow
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:sugarbay}} |
===Sugarbay: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=sugarbay
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=sugarbay
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=sugarbay
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:jasperforest}} |
===Jasperforest: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=jasperforest
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=jasperforest
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=jasperforest
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:crownbay}} |
===Crownbay: core-image-sato-sdk-noemgd===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=crownbay
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=crownbay
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=crownbay
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:fri2}} |
===FRI2: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=Fish River Island 2 (FRI2)
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=Fish River Island 2 (FRI2)
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=Fish River Island 2 (FRI2)
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:beagleboard}} |
===Beagleboard: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=beagleboard
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=beagleboard
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=beagleboard
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:routerstationpro}} |
===Routerstationpro: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=routerstationpro
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=routerstationpro
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=routerstationpro
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:mpc8315e-rdb}} |
===MPC8315e-rdb: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=MPC8315e-rdb
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=MPC8315e-rdb
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=MPC8315e-rdb
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:p1022ds-rdb}} |
===P1022ds-rdb: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=P1022ds
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=P1022ds
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=P1022ds
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
| }}
{{#if: {{#var:qemux86}}{{#var:qemux86-64}}{{#var:qemuarm}}{{#var:qemumips}}{{#var:qemuppc}} |
==Qemu==
{{#if: {{#var:qemux86}} |
===qemux86: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemu-x86
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemu-x86
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemu-x86
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:qemux86-64}} |
===qemux86-64: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemux86-64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemux86-64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemux86-64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:qemuarm}} |
===qemuarm: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemuarm
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemuarm
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemuarm
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:qemumips}} |
===qemumips: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemumips
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemumips
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemumips
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:qemuppc}} |
===qemuppc: core-image-sato-sdk===
Status
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemuppc
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemuppc
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=BSPs
|environment=qemuppc
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
| }}
{{#if: {{#var:runtime}} |
==Performance Testing==

Status
{{#testopia:
|run_id=-1
|product=runtime
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=runtime
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=runtime
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}

'''Performance Metrics''' <br />

Detailed result at: [https://wiki.yoctoproject.org/wiki/Performance_Test Build performance metrics]
| }}
{{#if: {{#var:build-appliance}} |
==Build-appliance==

Status
{{#testopia:
|run_id=-1
|product=build appliance
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=build appliance
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=build appliance
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:adt_fedora17_i686}}{{#var:adt_fedora17_x86-64}}{{#var:adt_ubuntu_1204_x86-64}}{{#var:adt_opensuse_122_x86-64}}{{#var:adt_opensuse_123_x86-64}}{{#var:adt_fedora18_i686}}{{#var:adt_fedora19_i686}}{{#var:adt_ubuntu_1210_x86-64}}{{#var:adt_ubuntu_1304_x86-64}}{{#var:eclipseplugin}} |
==ADT==
{{#if: {{#var:adt_fedora17_i686}} |
===ADT on: Fedora 17 i686===
Status
{{#testopia:
|run_id=-1
|product=adt
|environment=fedora 17 i686
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=adt
|environment=fedora 17 i686
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=adt
|environment=fedora 17 i686
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:adt_fedora18_i686}} |
===ADT on: Fedora 18 i686===
Status
{{#testopia:
|run_id=-1
|product=adt
|environment=fedora 18 i686
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=adt
|environment=fedora 18 i686
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=adt
|environment=fedora 18 i686
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:adt_fedora19_i686}} |
===ADT on: Fedora 19 i686===
Status
{{#testopia:
|run_id=-1
|product=adt
|environment=fedora 19 i686
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=adt
|environment=fedora 19 i686
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=adt
|environment=fedora 19 i686
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:adt_fedora17_x86-64}} |
===ADT on: Fedora 17 x86-64===
Status
{{#testopia:
|run_id=-1
|product=adt
|environment=fedora 17 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=adt
|environment=fedora 17 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=adt
|environment=fedora 17 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:adt_ubuntu_1204_x86-64}} |
===ADT on: Ubuntu 12.04 x86-64===
Status
{{#testopia:
|run_id=-1
|product=adt
|environment=ubuntu 12.04 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=adt
|environment=ubuntu 12.04 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=adt
|environment=ubuntu 12.04 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
{{#if: {{#var:adt_ubuntu_1210_x86-64}} |
===ADT on: Ubuntu 12.10 x86-64===
Status
{{#testopia:
|run_id=-1
|product=adt
|environment=ubuntu 12.10 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=adt
|environment=ubuntu 12.10 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=adt
|environment=ubuntu 12.10 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}

{{#if: {{#var:adt_ubuntu_1304_x86-64}} |
===ADT on: Ubuntu 13.04 x86-64===
Status
{{#testopia:
|run_id=-1
|product=adt
|environment=ubuntu 13.04 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=adt
|environment=ubuntu 13.04 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=adt
|environment=ubuntu 13.04 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}

{{#if: {{#var:adt_opensuse_122_x86-64}} |
===ADT on: OpenSUSE 12.2 x86-64===
Status
{{#testopia:
|run_id=-1
|product=adt
|environment=opensuse 12.2 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=adt
|environment=opensuse 12.2 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=adt
|environment=opensuse 12.2 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}

{{#if: {{#var:adt_opensuse_123_x86-64}} |
===ADT on: OpenSUSE 12.3 x86-64===
Status
{{#testopia:
|run_id=-1
|product=adt
|environment=opensuse 12.3 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=adt
|environment=opensuse 12.3 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=adt
|environment=opensuse 12.3 x86_64
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}
| }}
{{#if: {{#var:eclipseplugin}} |
==Eclipse Plugin==

Status
{{#testopia:
|run_id=-1
|product=eclipse plugin
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|showhide=true
|hidden=true
|report_id=status
|charttype=pie3}}

Bugs
{{#testopia:
|run_id=-1
|product=eclipse plugin
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=bugs
|chart=bstatus,prio
|charttype=pie,pie
|showhide=true
|hidden=true}}

Agenda
{{#testopia:
|run_id=-1
|product=eclipse plugin
|testplan={{#var:testplan}}
|build={{#var:build}}
|insummary_all={{#var:test_type}} |insummary_any={{#var:test_date}}
|report_id=agenda
|showhide=true
|hidden=true
|reportoptions=withdependency=false}}
| }}

Yocto Project 2.3 Release Test Plan

2016-12-21T19:08:07Z

Sonia Delgado: /* Features pending to be tested */

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | 6564}}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | 6565}}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This is the the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]

= Objectives =

This test plan for Yocto Project 2.3 release supports the following objectives:
*To define the the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7
* Debian 8
* Fedora 23
* Fedora 24
* OpenSuse132
* OpenSuse421
* ubuntu 14.04
* ubuntu 16.04

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

The components to be tested for version 2.3 of the {{ns:4}} against the different milestones are included in this section:

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.
{| class="wikitable"
!colspan="2"|Features
|-
! style="border-style: solid; border-width: 0 1px 1px 0"| Bug ID
! style="border-style: solid; border-width: 0 0 1px 0"| Description
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10723 Bug 10723]
|Build-Appliance - Run builds in the build-appliance image
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify if -c menuconfig work
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify if -c devshell work
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10725 Bug 10725]
|Verify if BBPOSTCONF and BBPRECONF work in bitbakek
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10726 Bug 10726]
|Verify if oe-pkgdata-util print help when no parameters specified
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10727 Bug 10727]
|Verify if buildhistory_analsysis.py is tested
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10728 Bug 10728]
|Verify combo-layer
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=6926 Bug 6926]
|Verify yocto-bsp/yocto-layer
|-
|[https://bugzilla.yoctoproject.org/show_bug.cgi?id=10724 Bug 10724]
|Verify bitbake -u depexp -g XXX
|}

= Features Not to be tested =
*

=Approach=
Tests will be conducted per the documented test cases stored in Testopia. The test manager will create test runs for each tester. The tester will execute the tests in Testopia and mark each case as Pass / Fail / Skip /Blocked.
The tester should leave notes on actual results and any other relevant details when possible.

Once complete, the test manager should review the test run reports in Testopia and report back to the team by mail and in the wiki as a test report.

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-12-21T18:21:14Z

Sonia Delgado: /* Features pending to be tested */

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | 6564}}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | 6565}}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This is the the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]

= Objectives =

This test plan for Yocto Project 2.3 release supports the following objectives:
*To define the the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7
* Debian 8
* Fedora 23
* Fedora 24
* OpenSuse132
* OpenSuse421
* ubuntu 14.04
* ubuntu 16.04

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

The components to be tested for version 2.3 of the {{ns:4}} against the different milestones are included in this section:

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image? [https://bugzilla.yoctoproject.org/show_bug.cgi?id=10723]
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Features Not to be tested =
*

=Approach=
Tests will be conducted per the documented test cases stored in Testopia. The test manager will create test runs for each tester. The tester will execute the tests in Testopia and mark each case as Pass / Fail / Skip /Blocked.
The tester should leave notes on actual results and any other relevant details when possible.

Once complete, the test manager should review the test run reports in Testopia and report back to the team by mail and in the wiki as a test report.

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-11-24T20:43:37Z

Sonia Delgado: /* Approach */

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | 6564}}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | }}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This is the the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]

= Objectives =

This test plan for Yocto Project 2.3 release supports the following objectives:
*To define the the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7
* Debian 8
* Fedora 23
* Fedora 24
* OpenSuse132
* OpenSuse421
* ubuntu 14.04
* ubuntu 16.04

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

The components to be tested for version 2.3 of the {{ns:4}} against the different milestones are included in this section:

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image?
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Features Not to be tested =
*

=Approach=
Tests will be conducted per the documented test cases stored in Testopia. The test manager will create test runs for each tester. The tester will execute the tests in Testopia and mark each case as Pass / Fail / Skip /Blocked.
The tester should leave notes on actual results and any other relevant details when possible.

Once complete, the test manager should review the test run reports in Testopia and report back to the team by mail and in the wiki as a test report.

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-11-24T19:26:36Z

Sonia Delgado:

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | }}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | }}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This is the the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]

= Objectives =

This test plan for Yocto Project 2.3 release supports the following objectives:
*To define the the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7
* Debian 8
* Fedora 23
* Fedora 24
* OpenSuse132
* OpenSuse421
* ubuntu 14.04
* ubuntu 16.04

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

The components to be tested for version 2.3 of the {{ns:4}} against the different milestones are included in this section:

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image?
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Features Not to be tested =
*

=Approach=
Tests will be conducted per the documented test cases stored in Testopia. The test manager will create test runs for each tester. The tester will execute the tests in Testopia and mark each case as Pass / Fail / Skip /Blocked
The tester should leave notes on actual results and any other relevant details when possible.

Once complete, the test manager should review the test run reports in Testopia and report back to the team by mail and in the wiki test report.

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-11-24T19:03:48Z

Sonia Delgado:

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | }}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | }}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This is the the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]

= Objectives =

This test plan for Yocto Project 2.3 release supports the following objectives:
*To define the the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7
* Debian 8
* Fedora 23
* Fedora 24
* OpenSuse132
* OpenSuse421
* ubuntu 14.04
* ubuntu 16.04

= Features to be tested =

The components to be tested for version 2.3 of the {{ns:4}} against the different milestones are included in this section:

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]
= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image?
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-11-24T19:00:59Z

Sonia Delgado:

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | }}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | }}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This is the the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]

= Objectives =

This test plan for Yocto Project 2.3 release supports the following objectives:
*To define the the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7
* Debian 8
* Fedora 23
* Fedora 24
* OpenSuse132
* OpenSuse421
* ubuntu 14.04
* ubuntu 16.04

= Features to be tested =

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image?
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-11-24T18:59:14Z

Sonia Delgado: /* Test Items */

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | }}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | }}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This is the the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]

= Objectives =

This test plan for Yocto Project 2.3 release supports the following objectives:
*To define the the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7
* Debian 8
* Fedora 23
* Fedora 24
* OpenSuse132
* OpenSuse421
* ubuntu 14.04
* ubuntu 16.04

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image?
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-11-24T18:45:19Z

Sonia Delgado:

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | }}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | }}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This is the the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing to [[QA Master Test Plan]]

= Objectives =

This test plan for Yocto Project 2.3 release supports the following objectives:
*To define the the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image?
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-11-24T18:42:00Z

Sonia Delgado:

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | }}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | }}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This test plan for Yocto Project 2.3 release supports the following objectives:
*To define the the tools to be used throughout the testing process.
*To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
*To define how the tests will be conducted.
*Release Criteria
*Risks and contingencies

The overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing [[QA Master Test Plan]]

= Objectives =

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image?
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-11-24T18:32:08Z

Sonia Delgado:

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | 6563}}
{{#vardefine:ANYQEMU_SATO_AUTO | }}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | }}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

=References=
Documents that support this test plan

= Introduction =

This article is the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing [[QA Master Test Plan]]

= Objectives =

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image?
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-11-24T17:19:37Z

Sonia Delgado: /* Test Items */

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | }}
{{#vardefine:ANYQEMU_SATO_AUTO | }}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | }}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

= Introduction =
This article is the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing [[QA Master Test Plan]]

= Objectives =

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|
|-
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|
|-
|}

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image?
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-11-24T17:12:05Z

Sonia Delgado: /* Test Items */

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | }}
{{#vardefine:ANYQEMU_SATO_AUTO | }}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | }}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

= Introduction =
This article is the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing [[QA Master Test Plan]]

= Objectives =

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|-
| bgcolor="ffbf00" style="text-align: center;" | CROPS
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|-
|}

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image?
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-11-22T21:47:07Z

Sonia Delgado:

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | }}
{{#vardefine:ANYQEMU_SATO_AUTO | }}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | }}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

= Introduction =
This article is the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing [[QA Master Test Plan]]

= Objectives =

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|-
| bgcolor="ffbf00" style="text-align: center;" | CROPS
|-
| bgcolor="ffbf00" style="text-align: center;" | Meta-SwUpd
|-
|}

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image?
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-11-22T21:42:24Z

Sonia Delgado:

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | }}
{{#vardefine:ANYQEMU_SATO_AUTO | }}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | }}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

= Introduction =
This article is the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing [[QA Master Test Plan]]

= Objectives =

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|-
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|}

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image?
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}

Yocto Project 2.3 Release Test Plan

2016-11-22T21:41:38Z

Sonia Delgado:

{|
{{#vardefine:ADT_MAUAL | 6539}}
{{#vardefine:ADT_AUTO | 6540}}
{{#vardefine:ANYQEMU_SATO_MANUAL | }}
{{#vardefine:ANYQEMU_SATO_AUTO | }}
{{#vardefine:ANYBSP_SATO_AUTO | 6558}}
{{#vardefine:ANYBSP_LSB_AUTO | 6561}}
{{#vardefine:ANYBSP_WIC_AUTO | 6560}}
{{#vardefine:MMAX_CORE2_MANUAL | 6554}}
{{#vardefine:MMAX_I7_MANUAL | 6557}}
{{#vardefine:NUC_I7_MANUAL | 6556}}
{{#vardefine:NUC_GEX86-64_MANUAL | 6555}}
{{#vardefine:MMAX_GEX86-32_MANUAL | 6553}}
{{#vardefine:GAL_SATO_AUTO | }}
{{#vardefine:CROPS_AUTO | 6541}}
{{#vardefine:CROPS_MANUAL | 6542}}
{{#vardefine:ECLIPSE_MANUAL | 6543}}
{{#vardefine:METAYOCTO_MANUAL | 6544}}
{{#vardefine:TOASTER_AUTO | 6545}}
{{#vardefine:TOASTER_MANUAL | 6546}}
{{#vardefine:ESDK_MANUAL | 6547}}
{{#vardefine:ESDK_AUTO | 6548}}
{{#vardefine:OE-CORE_AUTO | 6549}}
{{#vardefine:BITBAKE_AUTO | 6550}}
{{#vardefine:RUNTIME_AUTO | 6551}}
{{#vardefine:BUILD-APPLIANCE_AUTO | 6552}}
|}


= Reversion history =
{|class="wikitable"
! Version || Modifier || Comments
|-
| 1.0 || Jose Perez Carranza || First draft
|-
|}

= Introduction =
This article is the overall test plan for version 2.3 of the {{ns:4}}, for all the generic information this document will be pointing [[QA Master Test Plan]]

= Objectives =

[[QA_Master_Test_Plan#Objectives_and_Tasks]]

= Test Strategy =

Is followed the general strategy defined on [[QA_Master_Test_Plan#Testing_Strategy]] and below is showed specific information applicable only for 2.3 Release

== Bugs Tracking ==

*Bugs that need to be implemented by QA Team
**[[2.3_QA_OWNED_BUGS | 2.3 QA Assigned Bugs]]

*Bugs that need to be verified by QA Team
**[[2.3_QA_Bugs_To_Verify]]


*Old Bugs that need to be Verified
**[[Old resolved bugs and features]]

= Test Items =

{| class="wikitable"
! style="text-align: center;" | Code of Colors
|-bgcolor="93DB70"
| style="text-align: center;" | Active
Tracked on Testpoia
|-bgcolor="67E6EC"
| style="text-align: center;" | Active
Non Tracked on Testopia
|-bgcolor="ffbf00"
| style="text-align: center;" | In Design
Will be tracked on Tesopia
|- bgcolor="gray"
| style="text-align: center;" | Inactive
|}

{| class="wikitable"
! COMPONENT
! style="text-align: center;" | TEMPLATE
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | ADT
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_MAUAL}} TRTEMP_2.3_MANUAL_ADT]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ADT_AUTO}} TRTEMP_2.3_AUTO_ADT]
|-
| bgcolor="93DB70" rowspan="11" style="text-align: center;" | BSP
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_LSB_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYBSP_WIC_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk-wic_ANYBSP]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:GAL_SATO_AUTO}} TRTEMP_2.3_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ANYQEMU_SATO_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_CORE2_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_core2-32_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_MinnowMax]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_I7_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_corei7-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:NUC_GEX86-64_MANUAL }} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86-64_NUC]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:MMAX_GEX86-32_MANUAL}} TRTEMP_2.3_MANUAL_BSP/QEMU_GenericX86_MMAX32]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | CROPS
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_AUTO}} TRTEMP_2.3_AUTO_CROPS]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:CROPS_MANUAL}} TRTEMP_2.3_MANUAL_CROPS]
|-
| bgcolor="93DB70" style="text-align: center;" | Eclipse-Plugin
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ECLIPSE_MANUAL}} TRTEMP_2.3._MANUAL_Eclipse-Plugin]
|-
| bgcolor="93DB70" style="text-align: center;" | Meta-Yocto
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:METAYOCTO_MANUAL}} TRTEMP_2.3_MANUAL_Meta-yocto]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | Toaster
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_AUTO}} TRTEMP_2.3_AUTO_TOASTER_UI]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:TOASTER_MANUAL}} TRTEMP_2.3_MANUAL_TOASTER_UI-managed-mode]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | eSDK
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_MANUAL}} TRTEMP_2.3_AUTO_eSDK]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:ESDK_AUTO}} TRTEMP_2.3_MANUAL_eSDK]
|-
| bgcolor="93DB70" rowspan="2" style="text-align: center;" | OE-CORE / BitBake
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:OE-CORE_AUTO}} TRTEMP_2.3_AUTO_OE-Core]
|-
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BITBAKE_AUTO}} TRTEMP_2.3_MANUAL_BitBake]
|-
| bgcolor="93DB70" style="text-align: center;" | Runtime (Compliance)
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:RUNTIME_AUTO}} TRTEMP_2.3_AUTO_Runtime]
|-
| bgcolor="93DB70" style="text-align: center;" | Build Apliance
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id={{#var:BUILD-APPLIANCE_AUTO}} TRTEMP_2.3_MANUAL_Build-Appliance]
|-
| bgcolor="67E6EC" style="text-align: center;" | pTest
|
|-
| bgcolor="67E6EC" style="text-align: center;" | Performance
| [https://wiki.yoctoproject.org/wiki/Performance_Test Performance Test]
|-
| bgcolor="ffbf00" style="text-align: center;" | WUpd (Software Updater)
|
|-
| bgcolor="ffbf00" style="text-align: center;" | Distributed Builds
|
| bgcolor="ffbf00" style="text-align: center;" | MRAA
|}

= Test Cases =

*[[Yocto 2.3 Test Cases]]

= Features to be tested =

All component to be tested on 2.3 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]]

= Features pending to be tested =

Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.

* build-appliance - can we run builds in the build-appliance image?
* Verify if -c menuconfig work?
* Verify if -c devshell work?
* Verify if BBPOSTCONF and BBPRECONF work in bitbake?
* Verify if oe-pkgdata-util print help when no parameters specified?
* Verify if parsing time is improved with a hot codeparser cache (build metrics did catch a regression here but we had to manually spot it).
* Verify Bitbake sane error if a layer doesn't exist? (no backtrace?) send-error-report - no unittesting, only caught python3 issues when it failed to send errors
* Verify if buildhistory_analsysis.py is tested
* Verify if compression image types is tested
* Verify combo-layer
* Verify yocto-bsp/yocto-layer
* Verify oe-selftest coverage
* Verify bitbake -u depexp -g XXX
* rpm to be replaced
* <ADD MORE>

= Pass / Fail Criteria =

This criteria is defined in [[Yocto_Project_v2.3_Status#Yocto_Project_v2.3_Release_Criteria]]

= Exit Criteria =

As defined in [[QA_Master_Test_Plan#Entry and Exit Criteria]]

= Test Deliverables =

* Live Status [[2.3_QA_Status]]
* Final execution Report [[#Execution History]]
* Mail sent to the mailing list

= Test Environment =

There are different environments and tools for every type of testing refer to [[QA_Master_Test_Plan#Test_Areas]] for more detailed information.

= Schedule =

The planed schedule for 2.3 can be found at [[Yocto_2.3_Schedule]]

== Live Schedule ==

{| class="wikitable"
! MILESTONE
! STARTED
! FINISHED
! STATUS
|- bgcolor="Gray"
| M1
|
|
|
|- bgcolor="Gray"
| M2
|
|
|
|- bgcolor="Gray"
| M3
|
|
|
|- bgcolor="Gray"
| M4
|
|
|
|- bgcolor="Gray"
|
|
|
|
|- bgcolor="Gray"
|
|
|
|
|}

= Execution History =

{| class="wikitable"
! MILESTONE
! REPORT
|-
| M1
|
|-
| M2
|
|-
| M3
|
|-
| M4
|
|-
|
|
|-
|
|
|}