Yocto Project 2.4 Release Test Plan: Difference between revisions
No edit summary |
|||
(71 intermediate revisions by 6 users not shown) | |||
Line 84: | Line 84: | ||
* centos 7 | * centos 7 | ||
* Debian 8 | * Debian 8 | ||
* Fedora | * Fedora 24 | ||
* Fedora | * Fedora 25 | ||
* OpenSuse132 | * OpenSuse132 | ||
* | * OpenSuse422 | ||
* ubuntu | * ubuntu 16.04 | ||
* ubuntu 16. | * ubuntu 16.10 | ||
= Test Cases = | = Test Cases = | ||
Line 96: | Line 96: | ||
= | = Test Matrix = | ||
The components to be tested for version 2.4 of the {{ns:4}} against the different milestones are included in this section: | The components to be tested for version 2.4 of the {{ns:4}} against the different milestones are included in this section: | ||
Line 115: | Line 115: | ||
|} | |} | ||
== ADT == | |||
{| class="wikitable" | |||
! style="text-align: center; font-weight: bold;" | Component | |||
! style="text-align: center; font-weight: bold;" | Environment | |||
! style="text-align: center; font-weight: bold;" | Auto | |||
! style="text-align: center; font-weight: bold;" | Manual | |||
! style="text-align: center; font-weight: bold;" | Owner | |||
|- | |||
| rowspan="2" style="text-align: center;" | ADT | |||
| Fedora 25 i686 | |||
| Y | |||
| Y | |||
| WR | |||
|- | |||
| Ubuntu 16.04 x86_64 | |||
| Y | |||
| Y | |||
| WR | |||
|} | |||
== Eclipse Plugin == | |||
{| class="wikitable" | |||
! style="text-align: center;" | Component | |||
! style="text-align: center;" | Environment | |||
! style="text-align: center;" | Auto | |||
! style="text-align: center;" | Manual | |||
! style="text-align: center;" | Owner | |||
|- | |||
| rowspan="3" style="text-align: center;" | Eclipse Plugin | |||
| Mars + Any Distro | |||
| | |||
| Y | |||
| GDC | |||
|- | |||
| Neon + Any Distro | |||
| | |||
| Y | |||
| GDC | |||
|- | |||
| Oxygen + Any Distro | |||
| | |||
| Y | |||
| GDC | |||
|} | |||
== Kernel == | |||
{| class="wikitable" | |||
! style="text-align: center;" | Component | |||
! style="text-align: center;" | Environment | |||
! style="text-align: center;" | Auto | |||
! style="text-align: center;" | Manual | |||
! style="text-align: center;" | Owner | |||
|- | |||
| Kernel | |||
| Any Distro | |||
| | |||
| Y | |||
| GDC | |||
|} | |||
== Meta-Yocto == | |||
{| class="wikitable" | |||
! style="text-align: center;" | Component | |||
! style="text-align: center;" | Environment | |||
! style="text-align: center;" | Auto | |||
! style="text-align: center;" | Manual | |||
! style="text-align: center;" | Owner | |||
|- | |||
| Meta-Yocto | |||
| Any Distro | |||
| | |||
| Y | |||
| WR | |||
|} | |||
== Toaster == | |||
{| class="wikitable" | |||
! style="text-align: center;" | Component | |||
! style="text-align: center;" | Environment | |||
! style="text-align: center;" | Auto | |||
! style="text-align: center;" | Manual | |||
! style="text-align: center;" | Owner | |||
|- | |||
| Toaster | |||
| Any Distro | |||
| Y | |||
| Y | |||
| GDC | |||
|} | |||
== eSDK == | |||
{| class="wikitable" | {| class="wikitable" | ||
! | ! style="text-align: center;" | Component | ||
! style="text-align: center;" | | ! style="text-align: center;" | Environment | ||
! style="text-align: center;" | Auto | |||
! style="text-align: center;" | Manual | |||
! style="text-align: center;" | Owner | |||
|- | |||
| rowspan="2" style="text-align: center;" | eSDK | |||
| Any Distro + runtime | |||
| Y | |||
| | |||
| GDC | |||
|- | |- | ||
| | | Any Distro + selftest | ||
| | | Y | ||
|- | | | ||
| | | GDC | ||
|} | |||
== Selftest == | |||
{| class="wikitable" | |||
! style="text-align: center;" | Component | |||
! style="text-align: center;" | Environment | |||
! style="text-align: center;" | Auto | |||
! style="text-align: center;" | Manual | |||
! style="text-align: center;" | Owner | |||
|- | |||
| rowspan="8" style="text-align: center;" | OE-CORE | |||
| Ubuntu 16.10 | |||
| Y | |||
| | |||
| GDC | |||
|- | |||
| Ubuntu 16.04 | |||
| Y | |||
| | |||
| GDC | |||
|- | |||
| OpenSuse 42.2 | |||
| Y | |||
| | |||
| GDC | |||
|- | |||
| Opensuse 13.2 | |||
| Y | |||
| | |||
| GDC | |||
|- | |||
| Debian 8 | |||
| Y | |||
| | |||
| GDC | |||
|- | |||
| Centos 7 | |||
| Y | |||
| | |||
| GDC | |||
|- | |||
| Fedora 24 | |||
| Y | |||
| | |||
| GDC | |||
|- | |||
| Fedora 25 | |||
| Y | |||
| | |||
| GDC | |||
|} | |||
== Bitbake == | |||
{| class="wikitable" | |||
! style="text-align: center;" | Component | |||
! style="text-align: center;" | Environment | |||
! style="text-align: center;" | Auto | |||
! style="text-align: center;" | Manual | |||
! style="text-align: center;" | Owner | |||
|- | |||
| Bitbake | |||
| Any Distro | |||
| | |||
| Y | |||
| GDC | |||
|} | |||
== Build - Apliance == | |||
{| class="wikitable" | |||
! style="text-align: center;" | Component | |||
! style="text-align: center;" | Environment | |||
! style="text-align: center;" | Auto | |||
! style="text-align: center;" | Manual | |||
! style="text-align: center;" | Owner | |||
|- | |||
| Build - Apliance | |||
| VM - Any Distro | |||
| | |||
| Y | |||
| GDC | |||
|} | |||
== Package Manager == | |||
{| class="wikitable" | |||
! style="text-align: center;" | Component | |||
! style="text-align: center;" | Environment | |||
! style="text-align: center;" | Auto | |||
! style="text-align: center;" | Manual | |||
! style="text-align: center;" | Owner | |||
|- | |||
| Package Manager | |||
| Any Distro | |||
| | |||
| Y | |||
| GDC | |||
|} | |||
== Performance == | |||
{| class="wikitable" | |||
! style="text-align: center;" | Component | |||
! style="text-align: center;" | Environment | |||
! style="text-align: center;" | Auto | |||
! style="text-align: center;" | Manual | |||
! style="text-align: center;" | Owner | |||
|- | |||
| rowspan="2" style="text-align: center;" | Performance | |||
| Fedora 23 | |||
| Y | |||
| | |||
| GDC | |||
|- | |||
| Ubuntu 15 | |||
| Y | |||
| | |||
| GDC | |||
|} | |||
== BSP/QEMU == | |||
{| class="wikitable" | |||
! Component | |||
! Arch | |||
! HW | |||
! Environment | |||
! Image | |||
! Auto | |||
! Manual | |||
! pTest | |||
! Compliance | |||
|- | |||
| rowspan="29" style="text-align: center;" | BSP | |||
| rowspan="11" style="text-align: center;" | x86 | |||
| rowspan="2" style="text-align: center;" | MinwMax 32bit | |||
| rowspan="2" style="text-align: center;" | genericx86 | |||
| core-image-sato-sdk | |||
| Y | |||
| Y | |||
| | |||
| | |||
|- | |- | ||
| | | core-image-lsb-sdk | ||
| | | Y | ||
| | |||
| | |||
| | |||
|- | |- | ||
| | | rowspan="3" style="text-align: center;" | MinwMax 64bit | ||
| rowspan="3" style="text-align: center;" | genericx86-64 | |||
| core-image-sato-sdk | |||
| Y | |||
| | |||
| | |||
| | |||
|- | |- | ||
| | | core-image-lsb-sdk | ||
| Y | |||
| | |||
| | |||
| | |||
|- | |- | ||
| | | core-image-tiny-initramfs | ||
| Y | |||
| | |||
| | |||
| | |||
|- | |- | ||
| | | rowspan="2" style="text-align: center;" | NUC 64 | ||
| style="text-align: center;" | genericx86-64 | |||
| core-image-sato-sdk | |||
| Y | |||
| Y | |||
| Y | |||
| | |||
|- | |- | ||
| | | | ||
| core-image-lsb-sdk | |||
| Y | |||
| | |||
| | |||
| Y | |||
|- | |- | ||
| | | rowspan="4" style="text-align: center;" | QEMU | ||
| rowspan="2" style="text-align: center;" | qemux86 | |||
| core-image-sato-sdk | |||
| Y | |||
| Y | |||
| | |||
| | |||
|- | |- | ||
| | | core-image-lsb-sdk | ||
| Y | |||
| | |||
| | |||
| | |||
|- | |- | ||
| | | rowspan="2" style="text-align: center;" | qemux86-64 | ||
| core-image-sato-sdk | |||
| Y | |||
| Y | |||
| | |||
| | |||
|- | |- | ||
| | | core-image-lsb-sdk | ||
| Y | |||
| | |||
| | |||
| | |||
|- | |- | ||
| | | rowspan="8" style="text-align: center;" | non x86 | ||
| style="text-align: center;" | EdgeRouter | |||
| EdgeRouter | |||
| core-image-sato-sdk | |||
| Y | |||
| Y | |||
| | |||
| | |||
|- | |- | ||
| | | style="text-align: center;" | MPC8315e-rdb | ||
| | | MPC8315e-rdb | ||
| core-image-sato-sdk | |||
| Y | |||
| Y | |||
| | |||
| | |||
|- | |- | ||
| | | style="text-align: center;" | Beaglebone Black | ||
| Beaglebone | |||
| core-image-sato-sdk | |||
| Y | |||
| Y | |||
| | |||
| | |||
|- | |- | ||
| | | rowspan="5" style="text-align: center;" | QEMU | ||
| | | qemuarm | ||
| core-image-sato-sdk | |||
| Y | |||
| Y | |||
| | |||
| | |||
|- | |- | ||
| | | qemuarm-64 | ||
| | | core-image-sato-sdk | ||
| Y | |||
| Y | |||
| | |||
| | |||
|- | |- | ||
| | | qemuppc | ||
| | | core-image-sato-sdk | ||
| Y | |||
| Y | |||
| | |||
| | |||
|- | |- | ||
| | | qemumips | ||
| | | core-image-sato-sdk | ||
| Y | |||
| Y | |||
| | |||
| | |||
|- | |- | ||
| | | qemumips64 | ||
| core-image-sato-sdk | |||
| Y | |||
| Y | |||
| | |||
| | |||
|- | |- | ||
| | | rowspan="10" style="text-align: center;" | Meta-Intel | ||
| | | rowspan="2" style="text-align: center;" | MinnowMax 32bit | ||
| rowspan="2" style="text-align: center;" | core2-32 | |||
| core-image-sato-sdk | |||
| Y | |||
| Y | |||
| | |||
| | |||
|- | |- | ||
| | | core-image-lsb-sdk | ||
| Y | |||
| | |||
| | |||
| | |||
|- | |- | ||
| | | rowspan="2" style="text-align: center;" | MinnowMax 64bit | ||
| | | rowspan="2" style="text-align: center;" | corei7-64 | ||
| core-image-sato-sdk | |||
| Y | |||
| | |||
| | |||
| | |||
|- | |- | ||
| | | core-image-lsb-sdk | ||
| Y | |||
| | |||
| | |||
| | |||
|- | |- | ||
| | | rowspan="2" style="text-align: center;" | NUC | ||
| | | rowspan="2" style="text-align: center;" | corei7-64 | ||
| core-image-sato-sdk | |||
| Y | |||
| Y | |||
| | |||
| | |||
|- | |- | ||
| | | core-image-lsb-sdk | ||
| | | Y | ||
| | |||
| | |||
| | |||
|- | |- | ||
| | | rowspan="2" style="text-align: center;" | CherryHill | ||
| | | rowspan="2" style="text-align: center;" | corei7-64 | ||
| core-image-sato-sdk | |||
| Y | |||
| | |||
| | |||
| | |||
|- | |- | ||
| | | core-image-lsb-sdk | ||
| | | Y | ||
| | |||
| | |||
| | |||
|- | |- | ||
| | | style="text-align: center;" | Joule | ||
| | | style="text-align: center;" | corei7-64 | ||
| core-image-sato-sdk | |||
| Y | |||
| | |||
| | |||
| | |||
|- | |- | ||
| | | style="text-align: center;" | meta-intel-quark 32bit | ||
| style="text-align: center;" | intel-quark | |||
| Core-image-sato-sdk | |||
| | | Y | ||
| | |||
|- | | | ||
| | | | ||
| | |||
| | |||
| | |||
|} | |} | ||
All component to be tested on 2.4 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]] | All component to be tested on 2.4 are in [[#Test Items]] a detailed list of features per component can be found in Test Plan of the components [[QA_Master_Test_Plan#Component]] | ||
= | = Testopia Templates = | ||
{| class="wikitable" | {| class="wikitable" | ||
! | ! style="font-weight: bold;" | COMPONENT | ||
! style="font-weight: bold;" | TEMPLATE | |||
|- | |||
| BitBake | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7403 TRTEMP_2.4_MANUAL_BitBake] | |||
|- | |||
| rowspan="15" | BSP/QEMU | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7404 TRTEMP_2.4_MANUAL_BSP/QEMU_core-image-sato-sdk_Genericx86_MMAX32] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7405 TRTEMP_2.4_MANUAL_BSP/QEMU_core-image-sato-sdk_core2-32_MMAX32] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7406 TRTEMP_2.4_MANUAL_BSP/QEMU_core-image-sato-sdk_Genericx86-64_NUC] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7407 TRTEMP_2.4_MANUAL_BSP/QEMU_core-image-sato-sdk_corei7-64_NUC] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7408 TRTEMP_2.4_MANUAL_BSP/QEMU_core-image-sato-sdk_corei7-64_MMAX64] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7409 TRTEMP_2.4_AUTO_BSP/QEMU_core-image-sato-sdk_ANYBSP] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7410 TRTEMP_2.4_AUTO_BSP/QEMU_core-image-sato-sdk_wic_ANYBSP] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7411 TRTEMP_2.4_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYBSP] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7412 TRTEMP_2.4_MANUAL_BSP/QEMU_core-image-sato-sdk_ANYQEMU] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7413 TRTEMP_2.4_AUTO_BSP/QEMU_core-image-sato-sdk_ANYQEMU] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7414 TRTEMP_2.4_AUTO_BSP/QEMU_core-image-sato-sdk_intel-quark] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7415 TRTEMP_2.4_MANUAL_BSP/QEMU_core-image-sato-sdk_Beaglebone-Black] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7416 TRTEMP_2.4_MANUAL_BSP/QEMU_core-image-sato-sdk_EdgeRouter] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7417 TRTEMP_2.4_MANUAL_BSP/QEMU_core-image-sato-sdk_MPC8315e-rdb] | |||
|- | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7418 TRTEMP_2.4_AUTO_BSP/QEMU_core-image-lsb-sdk_ANYQEMU] | |||
|- | |||
| Meta-Yocto | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7419 TRTEMP_2.4_MANUAL_Meta-yocto] | |||
|- | |||
| OE-CORE | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7420 TRTEMP_2.4_AUTO_OE-Core] | |||
|- | |||
| Build-Appliance | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7421 TRTEMP_2.4_MANUAL_Build-Appliance] | |||
|- | |||
| Runtime | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7422 TRTEMP_2.4_AUTO_Runtime] | |||
|- | |||
| rowspan="2" | Toaster | |||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7423 TRTEMP_2.4_AUTO_Toaster_UI] | |||
|- | |- | ||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7424 TRTEMP_2.4_MANUAL_Toaster_UI-managed-mode] | |||
|- | |- | ||
|[https://bugzilla.yoctoproject.org/ | | rowspan="2" | ADT | ||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7425 TRTEMP_2.4_MANUAL_ADT] | |||
|- | |- | ||
|[https://bugzilla.yoctoproject.org/ | | [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7426 TRTEMP_2.4_AUTO_ADT] | ||
|- | |- | ||
|[https://bugzilla.yoctoproject.org/ | | rowspan="2" | CROPS | ||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7430 TRTEMP_2.4_AUTO_CROPS_Poky] | |||
|- | |- | ||
|[https://bugzilla.yoctoproject.org/ | | [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7431 TRTEMP_2.4_MANUAL_CROPS_eSDK] | ||
|- | |- | ||
|[https://bugzilla.yoctoproject.org/ | | Eclipse | ||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7432 TRTEMP_2.4_MANUAL_Eclipse] | |||
|- | |- | ||
|[https://bugzilla.yoctoproject.org/ | | rowspan="2" | eSDK | ||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7433 TRTEMP_2.4_AUTO_eSDK_sdkext] | |||
|- | |- | ||
|[https://bugzilla.yoctoproject.org/ | | [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7434 TRTEMP_2.4_AUTO_eSDK_oeselftest] | ||
|- | |- | ||
|[https://bugzilla.yoctoproject.org/ | | Kernel | ||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7435 TRTEMP_2.4_MANUAL_Kernel] | |||
|- | |- | ||
|[https://bugzilla.yoctoproject.org/ | | Package Management | ||
| [https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7489 TRTEMP_2.4_MANUAL_Package-Management-Updates] | |||
|} | |} | ||
= Features pending to be tested = | |||
Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component. | |||
'''ADD FEATURES !!!!!!!!!!''' | |||
The complete List of QA Implementations for YP 2.4 can be found at [https://wiki.yoctoproject.org/wiki/2.4_QA_OWNED_BUGS Features Implementation by QA Team] | The complete List of QA Implementations for YP 2.4 can be found at [https://wiki.yoctoproject.org/wiki/2.4_QA_OWNED_BUGS Features Implementation by QA Team] | ||
Line 273: | Line 700: | ||
= Schedule = | = Schedule = | ||
== M1 ( | == M1 (May 8, 2017 to Jun. 23, 2017 == | ||
=== M1 Feature/Tasks Development / Cutoff is | === M1 Feature/Tasks Development / Cutoff is Jun. 12, 2017 === | ||
{{#bugzilla: | {{#bugzilla: | ||
Line 282: | Line 709: | ||
|severity=enhancement | |severity=enhancement | ||
|sort=to,priority | |sort=to,priority | ||
|to=corneliux.stoicescu@intel.com,alexandru.c.georgescu@intel.com,ionutx.chisanovici@intel.com,georgex.l.musat@intel.com,stanciux.mihail@intel.com,andrax.g.necula@intel.com,costin.c.constantin@intel.com,daniel.alexandrux.istrate@intel.com,alexandru.costinx.roman@intel.com,valentinx.hangan@intel.com,libertad.gonzalez.de.la.cruz@intel.com,francisco.j.pedraza.gonzalez@intel.com,jose.perez.carranza@intel.com,humberto.ibarra.lopez | |to=corneliux.stoicescu@intel.com,alexandru.c.georgescu@intel.com,ionutx.chisanovici@intel.com,georgex.l.musat@intel.com,stanciux.mihail@intel.com,andrax.g.necula@intel.com,costin.c.constantin@intel.com,daniel.alexandrux.istrate@intel.com,alexandru.costinx.roman@intel.com,valentinx.hangan@intel.com,libertad.gonzalez.de.la.cruz@intel.com,francisco.j.pedraza.gonzalez@intel.com,jose.perez.carranza@intel.com,humberto.ibarra.lopez@intel.com,davidx.lopez.barriba@intel.com,juan.fernandox.ramos.frayle@intel.com,jair.de.jesus.gonzalez.plascencia@intel.com,sonia.leon@intel.com | ||
}} | }} | ||
Line 296: | Line 723: | ||
}} | }} | ||
The planed schedule for 2.4 can be found at [[Yocto_2.4_Schedule]] | The planed schedule for 2.4 can be found at [[Yocto_2.4_Schedule]] | ||
Line 312: | Line 735: | ||
|- bgcolor="00FF00" | |- bgcolor="00FF00" | ||
| M1 rc1 | | M1 rc1 | ||
| | | 06/15/2017 | ||
| | | 06/22/2017 | ||
| DONE | |||
|- bgcolor="00FF00" | |||
| M1 rc1 meta-intel | |||
| 06/23/2017 | |||
| 06/27/2017 | |||
| DONE | | DONE | ||
|- bgcolor="00FF00" | |- bgcolor="00FF00" | ||
| M2 | | M2 rc2 | ||
| | | 07/26/2017 | ||
| | | 08/02/2017 | ||
| DONE | | DONE | ||
|- bgcolor="00FF00" | |- bgcolor="00FF00" | ||
| M2 rc3 | | M2 rc3 | ||
| | | 08/03/2017 | ||
| | | 08/10/2017 | ||
| DONE | |||
|- bgcolor="00FF00" | |||
| M2 rc2 meta-intel | |||
| 08/11/2017 | |||
| 08/16/2017 | |||
| DONE | |||
|- bgcolor="00FF00" | |||
| M3 | |||
| 08/23/2017 | |||
| 08/30/2017 | |||
| DONE | | DONE | ||
|- bgcolor="00FF00" | |- bgcolor="00FF00" | ||
| M3 rc1 | | M3 rc1 meta-intel | ||
| | | 09/01/2017 | ||
| | | 09/05/2017 | ||
| DONE | | DONE | ||
|- bgcolor="00FF00" | |- bgcolor="00FF00" | ||
| | | M4 RC1 | ||
| | | 09/26/2017 | ||
| | | 10/06/2017 | ||
| DONE | | DONE | ||
|- bgcolor="00FF00" | |||
| M4 RC2 | |||
| 10/09/2017 | |||
| 10/13/2017 | |||
| DONE | |||
|- bgcolor="00FF00" | |- bgcolor="00FF00" | ||
| | | M4 RC3 | ||
| | | 10/17/2017 | ||
| | | 10/23/2017 | ||
| DONE | | DONE | ||
|- bgcolor="00FF00" | |- bgcolor="00FF00" | ||
| M4 rc2 | | M4 rc1 meta-intel | ||
| | | 10/26/2017 | ||
| | | 10/31/2017 | ||
| DONE | |||
|- bgcolor="00FF00" | |||
| M4 rc2 meta-intel | |||
| 11/03/2017 | |||
| 11/06/2017 | |||
| DONE | |||
|- bgcolor="00FF00" | |||
| 2.4.1.rc1 | |||
| 12/08/2017 | |||
| 12/22/2017 | |||
| DONE | |||
|- bgcolor="00FF00" | |||
| 2.4.2.rc2 | |||
| 2/21/2018 | |||
| 3/5/2018 | |||
| DONE | | DONE | ||
|- bgcolor="00FF00" | |- bgcolor="00FF00" | ||
| | | 2.4.2.rc2 meta-intel | ||
| | | 3/5/2018 | ||
| | | 3/15/2018 | ||
| DONE | | DONE | ||
|- bgcolor="00FF00" | |- bgcolor="00FF00" | ||
| 2.4 | | 2.4.3.rc2 | ||
| | | 5/25/2018 | ||
| | | 6/12/2018 | ||
| DONE | | DONE | ||
|- bgcolor="00FF00" | |- bgcolor="00FF00" | ||
| 2.4 | | 2.4.4.rc1 | ||
| | | 11/12/2018 | ||
| | | 11/23/2018 | ||
| DONE | | DONE | ||
|- bgcolor="grey" | |||
|} | |} | ||
Line 369: | Line 828: | ||
! REPORT | ! REPORT | ||
|- | |- | ||
| M1 | | M1 | ||
| | |https://wiki.yoctoproject.org/wiki/WW25_-_2017-06-22_-_Full_Test_Cycle_2.4_M1_rc1 | ||
|- | |- | ||
| M2 | | M1 meta-intel | ||
| | |https://wiki.yoctoproject.org/wiki/WW26_-_2017-06-27-_Full_Test_Cycle_-_2.4_M1_rc1_-_meta-intel | ||
|- | |- | ||
| M2 rc3 | | M2 | ||
| | |https://wiki.yoctoproject.org/wiki/WW31_-_2017-07-31_-_Full_Test_Cycle_2.4_M2_rc2 | ||
|- | |||
| M2 - rc3 | |||
|https://wiki.yoctoproject.org/wiki/WW32_-_2017-08-09_-_Full_Test_Cycle_2.4_M2_rc3 | |||
|- | |||
| M2 - rc2 meta-intel | |||
|https://wiki.yoctoproject.org/wiki/WW33_-_2017-08-15-_Full_Test_Cycle_-_2.4_M2_rc2_-_meta-intel | |||
|- | |||
| M3 | |||
|https://wiki.yoctoproject.org/wiki/WW35_-_2017-08-30_-_Full_Test_Cycle_2.4_M3 | |||
|- | |||
| M3 - rc1 meta-intel | |||
|https://wiki.yoctoproject.org/wiki/WW36_-_2017-09-05-_Full_Test_Cycle_-_2.4_M3_rc1_-_meta-intel | |||
|- | |||
| M4 - rc1 | |||
|https://wiki.yoctoproject.org/wiki/WW40_-_2017-10-06_-_Full_Test_Cycle_2.4_RC1 | |||
|- | |||
| M4 - rc2 | |||
|https://wiki.yoctoproject.org/wiki/WW41_-_2017-10-13_-_Full_Test_Cycle_2.4_RC2 | |||
|- | |||
| M4 - rc3 | |||
|https://wiki.yoctoproject.org/wiki/WW43_-_2017-10-23_-_Full_Test_Cycle_2.4_RC3 | |||
|- | |||
| M4 - rc1 meta-intel | |||
|https://wiki.yoctoproject.org/wiki/WW44_-_2017-10-31-_Full_Test_Cycle_-_2.4_M4_rc1_-_meta-intel | |||
|- | |- | ||
| | | M4 - rc2 meta-intel | ||
| | |https://wiki.yoctoproject.org/wiki/WW45_-_2017-11-06-_Full_Test_Cycle_-_2.4_M4_rc2_-_meta-intel | ||
|- | |- | ||
| | | 2.4.1.rc1 | ||
| | |https://wiki.yoctoproject.org/wiki/WW51_-_2017-12-21-_Full_Test_Cycle_-_2.4.1_rc1 | ||
|- | |- | ||
| | | 2.4.2.rc2 | ||
| | |https://wiki.yoctoproject.org/wiki/WW10_-_2018-03-05-_Full_Test_Cycle_-_2.4.2_rc2 | ||
|- | |- | ||
| | | 2.4.2.rc2 meta-intel | ||
| | |https://wiki.yoctoproject.org/wiki/WW11_-_2018-3-15-_Full_Test_Cycle_-_2.4.2_rc2_-_meta-intel | ||
|- | |- | ||
| | | 2.4.3.rc2 | ||
| | |https://wiki.yoctoproject.org/wiki/WW24_-_2018-06-12-_Full_Test_Cycle_-_2.4.3_rc2 | ||
|- | |- | ||
| | | 2.4.4.rc1 | ||
| | |https://wiki.yoctoproject.org/wiki/WW46_-_2018-11-14_-_Full_Test_Cycle_2.4.4_RC1 | ||
|- | |- | ||
|} | |} |
Latest revision as of 03:18, 23 November 2018
Reversion history
Version | Modifier | Comments |
---|---|---|
1.0 | Jose Perez Carranza | First draft |
References
Documents that support this test plan
Introduction
This is the the overall test plan for version 2.4 of the Yocto Project, for all the generic information this document will be pointing to QA Master Test Plan
Objectives
This test plan for Yocto Project 2.4 release supports the following objectives:
- To define the the tools to be used throughout the testing process.
- To communicate to the responsible parties the items to be tested, set expectations around schedule, and define environmental needs.
- To define how the tests will be conducted.
- Release Criteria
- Risks and contingencies
QA_Master_Test_Plan#Objectives_and_Tasks
Test Strategy
Is followed the general strategy defined on QA_Master_Test_Plan#Testing_Strategy and below is showed specific information applicable only for 2.4 Release
Bugs Tracking
- Bugs that need to be implemented by QA Team
- Bugs that need to be verified by QA Team
- Old Bugs that need to be Verified
Test Items
The Yocto Project QA team verifies each release against recent versions of the most popular Linux distributions that provide stable releases.
* centos 7 * Debian 8 * Fedora 24 * Fedora 25 * OpenSuse132 * OpenSuse422 * ubuntu 16.04 * ubuntu 16.10
Test Cases
Test Matrix
The components to be tested for version 2.4 of the Yocto Project against the different milestones are included in this section:
Code of Colors |
---|
Active
Tracked on Testpoia |
Active
Non Tracked on Testopia |
In Design
Will be tracked on Tesopia |
Inactive |
ADT
Component | Environment | Auto | Manual | Owner |
---|---|---|---|---|
ADT | Fedora 25 i686 | Y | Y | WR |
Ubuntu 16.04 x86_64 | Y | Y | WR |
Eclipse Plugin
Component | Environment | Auto | Manual | Owner |
---|---|---|---|---|
Eclipse Plugin | Mars + Any Distro | Y | GDC | |
Neon + Any Distro | Y | GDC | ||
Oxygen + Any Distro | Y | GDC |
Kernel
Component | Environment | Auto | Manual | Owner |
---|---|---|---|---|
Kernel | Any Distro | Y | GDC |
Meta-Yocto
Component | Environment | Auto | Manual | Owner |
---|---|---|---|---|
Meta-Yocto | Any Distro | Y | WR |
Toaster
Component | Environment | Auto | Manual | Owner |
---|---|---|---|---|
Toaster | Any Distro | Y | Y | GDC |
eSDK
Component | Environment | Auto | Manual | Owner |
---|---|---|---|---|
eSDK | Any Distro + runtime | Y | GDC | |
Any Distro + selftest | Y | GDC |
Selftest
Component | Environment | Auto | Manual | Owner |
---|---|---|---|---|
OE-CORE | Ubuntu 16.10 | Y | GDC | |
Ubuntu 16.04 | Y | GDC | ||
OpenSuse 42.2 | Y | GDC | ||
Opensuse 13.2 | Y | GDC | ||
Debian 8 | Y | GDC | ||
Centos 7 | Y | GDC | ||
Fedora 24 | Y | GDC | ||
Fedora 25 | Y | GDC |
Bitbake
Component | Environment | Auto | Manual | Owner |
---|---|---|---|---|
Bitbake | Any Distro | Y | GDC |
Build - Apliance
Component | Environment | Auto | Manual | Owner |
---|---|---|---|---|
Build - Apliance | VM - Any Distro | Y | GDC |
Package Manager
Component | Environment | Auto | Manual | Owner |
---|---|---|---|---|
Package Manager | Any Distro | Y | GDC |
Performance
Component | Environment | Auto | Manual | Owner |
---|---|---|---|---|
Performance | Fedora 23 | Y | GDC | |
Ubuntu 15 | Y | GDC |
BSP/QEMU
Component | Arch | HW | Environment | Image | Auto | Manual | pTest | Compliance |
---|---|---|---|---|---|---|---|---|
BSP | x86 | MinwMax 32bit | genericx86 | core-image-sato-sdk | Y | Y | ||
core-image-lsb-sdk | Y | |||||||
MinwMax 64bit | genericx86-64 | core-image-sato-sdk | Y | |||||
core-image-lsb-sdk | Y | |||||||
core-image-tiny-initramfs | Y | |||||||
NUC 64 | genericx86-64 | core-image-sato-sdk | Y | Y | Y | |||
core-image-lsb-sdk | Y | Y | ||||||
QEMU | qemux86 | core-image-sato-sdk | Y | Y | ||||
core-image-lsb-sdk | Y | |||||||
qemux86-64 | core-image-sato-sdk | Y | Y | |||||
core-image-lsb-sdk | Y | |||||||
non x86 | EdgeRouter | EdgeRouter | core-image-sato-sdk | Y | Y | |||
MPC8315e-rdb | MPC8315e-rdb | core-image-sato-sdk | Y | Y | ||||
Beaglebone Black | Beaglebone | core-image-sato-sdk | Y | Y | ||||
QEMU | qemuarm | core-image-sato-sdk | Y | Y | ||||
qemuarm-64 | core-image-sato-sdk | Y | Y | |||||
qemuppc | core-image-sato-sdk | Y | Y | |||||
qemumips | core-image-sato-sdk | Y | Y | |||||
qemumips64 | core-image-sato-sdk | Y | Y | |||||
Meta-Intel | MinnowMax 32bit | core2-32 | core-image-sato-sdk | Y | Y | |||
core-image-lsb-sdk | Y | |||||||
MinnowMax 64bit | corei7-64 | core-image-sato-sdk | Y | |||||
core-image-lsb-sdk | Y | |||||||
NUC | corei7-64 | core-image-sato-sdk | Y | Y | ||||
core-image-lsb-sdk | Y | |||||||
CherryHill | corei7-64 | core-image-sato-sdk | Y | |||||
core-image-lsb-sdk | Y | |||||||
Joule | corei7-64 | core-image-sato-sdk | Y | |||||
meta-intel-quark 32bit | intel-quark | Core-image-sato-sdk | Y |
All component to be tested on 2.4 are in #Test Items a detailed list of features per component can be found in Test Plan of the components QA_Master_Test_Plan#Component
Testopia Templates
Features pending to be tested
Here is a list of areas that are not yet covered and need some work, implement test cases and add it to a component that belongs or create a new test plan for new component.
ADD FEATURES !!!!!!!!!!
The complete List of QA Implementations for YP 2.4 can be found at Features Implementation by QA Team
Features Not to be tested
Approach
Tests will be conducted per the documented test cases stored in Testopia. The test manager will create test runs for each tester. The tester will execute the tests in Testopia and mark each case as Pass / Fail / Skip /Blocked. The tester should leave notes on actual results and any other relevant details when possible.
Once complete, the test manager should review the test run reports in Testopia and report back to the team by mail and in the wiki as a test report.
Pass / Fail Criteria
This criteria is defined in Yocto_Project_v2.4_Status#Yocto_Project_v2.4_Release_Criteria
Exit Criteria
As defined in QA_Master_Test_Plan#Entry and Exit Criteria
Test Deliverables
- Live Status 2.4_QA_Status
- Final execution Report #Execution History
- Mail sent to the mailing list
Test Environment
There are different environments and tools for every type of testing refer to QA_Master_Test_Plan#Test_Areas for more detailed information.
Schedule
M1 (May 8, 2017 to Jun. 23, 2017
M1 Feature/Tasks Development / Cutoff is Jun. 12, 2017
ID | Requester | Assignee | E | Summary (4 tasks) ⇒ | Severity | P | Whiteboard | Status |
---|---|---|---|---|---|---|---|---|
11244 | Jair Gonzalez | Jair Gonzalez | 2 | QA: Validate Reference kit has BT audio support on gateway profile | enhancement | Medium | RESOLVED | |
11246 | Jair Gonzalez | Jair Gonzalez | 5 | QA: Validate Reference kit supports image installer | enhancement | Medium | RESOLVED | |
10817 | Paul Eggleton | Jose Perez C | 4 | oe-selftest: add a test for devtool modify on the kernel | enhancement | Medium+ | RESOLVED | |
11550 | Jose Perez C | Juan Ramos | poky-tiny: Add tests for poky-tiny images | enhancement | Medium | VERIFIED | ||
11 |
M1 Bugs
ID | Requester | Assignee | E | Summary (4 tasks) ⇒ | Severity | P | Whiteboard | Status |
---|---|---|---|---|---|---|---|---|
11436 | Mariano Lopez | Humberto Ibarra | 3 | selftest: runtime-test install test is testing two different features | normal | Medium+ | RESOLVED | |
10848 | Jose Perez C | Jose Perez C | 2 | Add support for testing wic images on qemux86* | normal | Medium+ | RESOLVED | |
11300 | Jose Perez C | Jose Perez C | 2 | selftest: do_populate_sdk_ext task failing on eSDK test on GDC ABs | normal | Medium+ | VERIFIED | |
10947 | Libertad | Libertad | 3 | Reintegrate fully functional and corrected toaster automation testcases from toaster_automation_tests.py | normal | Medium | RESOLVED | |
10 |
The planed schedule for 2.4 can be found at Yocto_2.4_Schedule
Live Schedule
MILESTONE | STARTED | FINISHED | STATUS |
---|---|---|---|
M1 rc1 | 06/15/2017 | 06/22/2017 | DONE |
M1 rc1 meta-intel | 06/23/2017 | 06/27/2017 | DONE |
M2 rc2 | 07/26/2017 | 08/02/2017 | DONE |
M2 rc3 | 08/03/2017 | 08/10/2017 | DONE |
M2 rc2 meta-intel | 08/11/2017 | 08/16/2017 | DONE |
M3 | 08/23/2017 | 08/30/2017 | DONE |
M3 rc1 meta-intel | 09/01/2017 | 09/05/2017 | DONE |
M4 RC1 | 09/26/2017 | 10/06/2017 | DONE |
M4 RC2 | 10/09/2017 | 10/13/2017 | DONE |
M4 RC3 | 10/17/2017 | 10/23/2017 | DONE |
M4 rc1 meta-intel | 10/26/2017 | 10/31/2017 | DONE |
M4 rc2 meta-intel | 11/03/2017 | 11/06/2017 | DONE |
2.4.1.rc1 | 12/08/2017 | 12/22/2017 | DONE |
2.4.2.rc2 | 2/21/2018 | 3/5/2018 | DONE |
2.4.2.rc2 meta-intel | 3/5/2018 | 3/15/2018 | DONE |
2.4.3.rc2 | 5/25/2018 | 6/12/2018 | DONE |
2.4.4.rc1 | 11/12/2018 | 11/23/2018 | DONE |