WW32 - 2017-08-09 - Full Test Cycle 2.4 M2 rc3: Difference between revisions
No edit summary |
|||
(12 intermediate revisions by 2 users not shown) | |||
Line 65: | Line 65: | ||
=Testing data= | =Testing data= | ||
<nowiki> | <nowiki> | ||
Test type: | Test type: Full Pass Test | ||
Branch: | Branch: master | ||
Build name: | Build name: 2.4 M2 RC3 | ||
poky commit: | poky commit: 9ed748a542b520c1cb763d981969233c0f5efd4e | ||
Autobuilder images repository: | Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.4_M2.rc3 | ||
Distributions tested: | Distributions tested: | ||
CentOS 7.1 | |||
Debian 8 | |||
Fedora 24 | |||
Fedora 25 | |||
OpenSuse 422 | |||
Ubuntu 16.04 | |||
Ubuntu 17.04 | |||
</nowiki> | </nowiki> | ||
{| | {| | ||
Line 96: | Line 104: | ||
|report_id=status}} | |report_id=status}} | ||
|} | |} | ||
= Bugs Found during QA Test = | = Bugs Found during QA Test = | ||
== <span style="color:blue; "> New issues Found during QA Test & Community </span>== | == <span style="color:blue; "> New issues Found during QA Test & Community </span>== | ||
Line 103: | Line 112: | ||
|sort=ID | |sort=ID | ||
|group=product | |group=product | ||
|id= | |id=11917, 11923 | ||
}} | }} | ||
<br> | <br> | ||
Line 113: | Line 122: | ||
|sort=priority | |sort=priority | ||
|group=product | |group=product | ||
|id= | |id=11879, 11833 | ||
}}<br> | }}<br> | ||
Line 122: | Line 131: | ||
|sort=priority | |sort=priority | ||
|group=cf_regression_type,product | |group=cf_regression_type,product | ||
|id= | |id=9267, 10455, 10477 | ||
}} | }} | ||
Line 137: | Line 146: | ||
|total=summary | |total=summary | ||
|status=!(RESOLVED,VERIFIED) | |status=!(RESOLVED,VERIFIED) | ||
|milestone=2.4 | |milestone=2.4, 2.4 M1, 2.4 M2, 2.4 M3 | ||
}} | }} | ||
Line 144: | Line 153: | ||
= Build Performance Test = | = Build Performance Test = | ||
Detailed description on how performance test | Detailed description on how performance test are run can be found on the [[Performance Test]] wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.4 M1. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and another one started on 2.1 from GDC. There are differences between HW used in both locations, hence the numbers are different. | ||
The | |||
* [https://wiki.yoctoproject.org/charts/perf_milestone/performance_test.html RO Charts] | * [https://wiki.yoctoproject.org/charts/perf_milestone/performance_test.html RO Charts] | ||
* [https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.html GDC Charts] | * [https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.html GDC Charts] | ||
Below | Below charts are showing the latest performance results taken from the current milestone. | ||
== Performance Charts== | == Performance Charts== | ||
Table | |||
[[File:Ww32-2k17_-_table.png | 500px]] | |||
For Bitbake core-image-sato withrm_work | For Bitbake core-image-sato withrm_work | ||
[[File: | [[File:Ww32-2k17 - rmwork.png | 1280px | link=https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-core-image-sato-rmwork.html ]] | ||
Building the Virtual kernel | Building the Virtual kernel | ||
[[File: | [[File:Ww32-2k17 - kernel.png | 1280px | link=https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-virtual-kernel.html]] | ||
Building the rootfs | Building the rootfs | ||
[[File: | [[File:Ww32-2k17_-_rootfs.png | 1280px | link=https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-core-image-sato-rootfs.html]] | ||
Building core-image-sato | Building core-image-sato | ||
[[File: | [[File:Ww32-2k17_-_sato.png | 1280px | link=https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-core-image-sato.html]] | ||
eSDK Deploy time | eSDK Deploy time | ||
[[File: | [[File:Ww32-2k17_-_esdk.png | 1280px | link=https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-eSDK.html]] | ||
=pTest for genericx86-64 on NUC= | =pTest for genericx86-64 on NUC= | ||
Line 183: | Line 196: | ||
|sort=status | |sort=status | ||
|group=product | |group=product | ||
|id= | |id=11596, 10266, 11602, 11668, 6707, 5698, 8832, 9042, 8471,8828, 5697, 10953,8650,11024,5695,5605,4437,4436,6737,11674,10841,8064,11667,11666,11547,11812,11813}} | ||
}} | |||
== results == | == results == | ||
{{:Ptest | {{:Ptest 9ed748a542b520c1cb763d981969233c0f5efd4e}} | ||
=Compliance Report= | |||
==POSIX results== | |||
General results for POSIX execution can be found [https://wiki.yoctoproject.org/wiki/Posix_result POSIX_result] | |||
and greater details on each architecture can be found [https://wiki.yoctoproject.org/wiki/POSIX-results POSIX_details] | |||
Bellow you will see the last two execution results for comparison: | |||
===Yocto 2.4 M2 RC3 build=== | |||
{|border="1" | |||
|| '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb''' | |||
|- | |||
|| failures || 46 || 45 || 46 || 45 | |||
|} | |||
===Yocto 2.4 M2 RC2 build=== | |||
{|border="1" | |||
|| '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb''' | |||
|- | |||
|| failures || 46 || 45 || 46 || 45 | |||
|} | |||
===Yocto 2.4 M1 RC1 build=== | |||
{|border="1" | |||
|| '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb''' | |||
|- | |||
|| failures || 45 || 45 || 46 || 45 | |||
|} | |||
==LTP results== | |||
General results for LTP can be found here [https://wiki.yoctoproject.org/wiki/LTP_result LTP_results] | |||
and for greater information click on each architecture to be redirected to their details. There is currently an infrastructure issue that is blocking the beblebone and edgerouter testing for LTP, results are not shown in the tables below. | |||
===Yocto 2.4 M2 RC3 build=== | |||
{|border="1" | |||
|| '''Status''' || '''[[genericx86-64-ltp|genericx86-64]]''' || '''[[beaglebone-ltp|beaglebone]]''' || '''[[edgerouter-ltp|edgerouter]]''' || '''[[mpc8315e-rdb-ltp|mpc8315e-rdb]]''' | |||
|- | |||
|| total || 1384 || 1346 || || 1346 | |||
|- | |||
|| failures || 23 || 15 || || 27 | |||
|} | |||
===Yocto 2.4 M2 RC2 build=== | |||
{|border="1" | |||
|| '''Status''' || '''[[genericx86-64-ltp|genericx86-64]]''' || '''[[beaglebone-ltp|beaglebone]]''' || '''[[edgerouter-ltp|edgerouter]]''' || '''[[mpc8315e-rdb-ltp|mpc8315e-rdb]]''' | |||
|- | |||
|| total || 1384 || || || 1348 | |||
|- | |||
|| failures || 24 || || || 30 | |||
|} | |||
===Yocto 2.4 M1 RC1 build=== | |||
{|border="1" | |||
|| '''Status''' || '''[[genericx86-64-ltp|genericx86-64]]''' || '''[[beaglebone-ltp|beaglebone]]''' || '''[[edgerouter-ltp|edgerouter]]''' || '''[[mpc8315e-rdb-ltp|mpc8315e-rdb]]''' | |||
|- | |||
|| total || 1384 || || || 1348 | |||
|- | |||
|| failures || 22 || || || 30 | |||
|} | |||
=Detailed Report= | =Detailed Report= | ||
Latest revision as of 15:58, 10 August 2017
Testing data
Test type: Full Pass Test Branch: master Build name: 2.4 M2 RC3 poky commit: 9ed748a542b520c1cb763d981969233c0f5efd4e Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.4_M2.rc3 Distributions tested: CentOS 7.1 Debian 8 Fedora 24 Fedora 25 OpenSuse 422 Ubuntu 16.04 Ubuntu 17.04
{{#testopia: | run_id=0, {{#var:1_ADT_Fedora_26_i686}}, {{#var:2_ADT_Fedora_26_i686}}, {{#var:3_ADT_Ubuntu_16.04_x86_64}}, {{#var:4_ADT_Ubuntu_16.04_x86_64}}, {{#var:5_BitBake_Fedora_24_x86_64}}, {{#var:6_BSPs_Genericx86_MMAX32}}, {{#var:7_BSPs_Genericx86-64_NUC}}, {{#var:8_BSPs_Beaglebone-Black}}, {{#var:9_BSPs_EdgeRouter}}, {{#var:10_BSPs_MPC8315e-rdb}}, {{#var:11_BSPs_qemu-x86}}, {{#var:12_BSPs_qemuarm}}, {{#var:13_BSPs_qemuarm64}}, {{#var:14_BSPs_qemumips}}, {{#var:15_BSPs_qemumips64}}, {{#var:16_BSPs_qemuppc}}, {{#var:17_BSPs_qemux86-64}}, {{#var:18_BSPs_qemu-x86}}, {{#var:19_BSPs_qemuarm}}, {{#var:20_BSPs_qemuarm64}}, {{#var:21_BSPs_qemumips}}, {{#var:22_BSPs_qemumips64}}, {{#var:23_BSPs_qemuppc}}, {{#var:24_BSPs_qemux86-64,}}, {{#var:25_BSPs_qemux86-64-WIC,}}, {{#var:26_BSPs_qemux86-WIC}}, {{#var:27_BSPs_qemux86-lsb}}, {{#var:28_BSPs_qemux86_64-lsb}}, {{#var:29_BSPs_genericx86_-_on_MMAX32bit}}, {{#var:30_BSPs_genericx86-64_on_MMAX64}}, {{#var:31_BSPs_genericx86-64_on_NUC}}, {{#var:32_BSPs_genericx86-64-WIC_on_MMAX64}}, {{#var:33_BSPs_genericx86-64-WIC_on_NUC}}, {{#var:34_BSPs_genericx86-WIC_on_MMAX32}}, {{#var:35_BSPs_MPC8315e-rdb}}, {{#var:36_BSPs_Beaglebone_Black}}, {{#var:37_BSPs_EdgeRouter}}, {{#var:38_BSPs_genericx86-64-lsb_on_NUC}}, {{#var:39_BSPs_genericx86-64-lsb_on_MMAX64}}, {{#var:40_BSPs_genericx86-lsb_on_MMAX32bit}}, {{#var:41_CROPS_Centos_7.1}}, {{#var:42_Eclipse Plugin_Mars_on_Fedora_22_x86_64,}}, {{#var:43_Eclipse Plugin_Neon_on_Fedora_22_x86_64}}, {{#var:44_eSDK_Centos_7.1}}, {{#var:45_eSDK_Centos_7.1}}, {{#var:46_General Runtime_user-preference_(any_supported_environment)}}, {{#var:47_Kernel_Centos_7}}, {{#var:48_Meta-yocto_Ubuntu_14.04_x86_64}}, {{#var:49_OE-Core_CentOS_7.1_x86_64}}, {{#var:50_OE-Core_Debian_8_x86_64}}, {{#var:51_OE-Core_Fedora_25_x86_64}}, {{#var:52_OE-Core_Fedora_24_x86_64}}, {{#var:53_OE-Core_OpenSuse421_x86_64}}, {{#var:54_OE-Core_ubuntu_16.04_x86_64}}, {{#var:55_OE-Core_Ubuntu_17.04_x86-64}}, {{#var:56_Runtime_VMWare_Player_on_Ubuntu}}, {{#var:57_Toaster_Debian_4.3.0-0.bpo.1-amd64}}, {{#var:58_Toaster_Debian_4.3.0-0.bpo.1-amd64}}, {{#var:59_Package Management Issues_Any_Supported_Distro}}, {{#var:60_BSPs_Any_QEMU}}, {{#var:61_BSPs_Any_QEMU}} | report_id=concatstats}}
|