WW41 - 2017-10-13 - Full Test Cycle 2.4 RC2: Difference between revisions
(Created page with "{| {{#vardefine:1_ADT_Fedora_26_i686| 8285}} {{#vardefine:2_ADT_Fedora_26_i686| 8286}} {{#vardefine:3_ADT_Ubuntu_16.04_x86_64| 8287}} {{#vardefine:4_ADT_Ubuntu_16.04_x86_64...") |
|||
(14 intermediate revisions by the same user not shown) | |||
Line 62: | Line 62: | ||
=Testing data= | =Testing data= | ||
<nowiki> | <nowiki> | ||
Test type: | Test type: Full Pass Test | ||
Branch: | Branch: rocko | ||
Build name: | Build name: yocto-2.4.rc2 | ||
poky commit: | poky commit: da05a01423a60a23cd1073eb0ad3776e5bf4f2a8 | ||
Autobuilder images repository: | Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.4.rc2/ | ||
Distributions tested: | Distributions tested: | ||
CentOs 7.1 | |||
Debian 8 | |||
Fedora 25 | |||
Fedora 26 | |||
OpenSuse 422 | |||
Ubuntu 16.04 | |||
Ubuntu 17.04 | |||
</nowiki> | </nowiki> | ||
{| | {| | ||
Line 93: | Line 102: | ||
|report_id=status}} | |report_id=status}} | ||
|} | |} | ||
= Bugs Found during QA Test = | = Bugs Found during QA Test = | ||
== <span style="color:blue; "> New issues Found during QA Test & Community </span>== | == <span style="color:blue; "> New issues Found during QA Test & Community </span>== | ||
Line 100: | Line 110: | ||
|sort=ID | |sort=ID | ||
|group=product | |group=product | ||
|id= | |id=12213, 12214, 12218, 12220, 12215 | ||
}} | }} | ||
<br> | <br> | ||
Line 110: | Line 120: | ||
|sort=priority | |sort=priority | ||
|group=product | |group=product | ||
|id= | |id=11994, 12131 | ||
}}<br> | }}<br> | ||
Line 119: | Line 129: | ||
|sort=priority | |sort=priority | ||
|group=cf_regression_type,product | |group=cf_regression_type,product | ||
|id= | |id=10455, 12187 | ||
}} | }} | ||
Line 134: | Line 144: | ||
|total=summary | |total=summary | ||
|status=!(RESOLVED,VERIFIED) | |status=!(RESOLVED,VERIFIED) | ||
|milestone=2.4 | |milestone=2.4, 2.4 M1, 2.4 M2, 2.4 M3, 2.4 M4 | ||
}} | }} | ||
Line 141: | Line 151: | ||
= Build Performance Test = | = Build Performance Test = | ||
Detailed description on how performance test is run | Detailed description on how performance test is run can be found on the [[Performance Test]] wiki. | ||
The | The below charts are displaying the build performance trend for 2.1M4 to 2.4 RC1. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and executed from 2.1 to 2.4 in GDC and are differences between HW used in both locations, hence the numbers are different. | ||
* [https://wiki.yoctoproject.org/charts/perf_milestone/performance_test.html RO Charts] | * [https://wiki.yoctoproject.org/charts/perf_milestone/performance_test.html RO Charts] | ||
Line 153: | Line 163: | ||
For Bitbake core-image-sato withrm_work | For Bitbake core-image-sato withrm_work | ||
[[File: | [[File:Ww41-2k17 - rmwork.png | 1280px | link=https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-core-image-sato-rmwork.html ]] | ||
Building the Virtual kernel | Building the Virtual kernel | ||
[[File: | [[File:Ww41-2k17 - kernel.png | 1280px | link=https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-virtual-kernel.html]] | ||
Building the rootfs | Building the rootfs | ||
[[File: | [[File:Ww41-2k17_-_rootfs.png | 1280px | link=https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-core-image-sato-rootfs.html]] | ||
Building core-image-sato | Building core-image-sato | ||
[[File: | [[File:Ww41-2k17_-_sato.png | 1280px | link=https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-core-image-sato.html]] | ||
eSDK Deploy time | eSDK Deploy time | ||
[[File: | [[File:Ww41-2k17_-_esdk.png | 1280px | link=https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-eSDK.html]] | ||
=pTest for genericx86-64 on NUC= | =pTest for genericx86-64 on NUC= | ||
Line 180: | Line 190: | ||
|sort=status | |sort=status | ||
|group=product | |group=product | ||
|id= | |id=12094, 6707, 11942, 12181, 4436, 4437, 5697, 5698, 6737, 8064, 8471, 8650, 8828, 8832, 9042, 10266, 10841, 10953, 11024,11674, 11811, 11812, 11813, 11997, 11998, 5605, 5695,12215 | ||
}} | }} | ||
== results == | == results == | ||
{{:Ptest | {{:Ptest da05a01423a60a23cd1073eb0ad3776e5bf4f2a8}} | ||
=Compliance Report= | |||
==POSIX results== | |||
General results for POSIX execution can be found [https://wiki.yoctoproject.org/wiki/Posix_result POSIX_result] | |||
and greater details on each architecture can be found [https://wiki.yoctoproject.org/wiki/POSIX-results POSIX_details] | |||
Bellow you will see the last three execution results for comparison: | |||
===Yocto 2.4 RC2 build=== | |||
{|border="1" | |||
|| '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb''' | |||
|- | |||
|| failures || 46 || 45 || || 45 | |||
|} | |||
===Yocto 2.4 RC1 build=== | |||
{|border="1" | |||
|| '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb''' | |||
|- | |||
|| failures || 46 || 45 || 47 || 45 | |||
|} | |||
==LTP results== | |||
General results for LTP can be found here [https://wiki.yoctoproject.org/wiki/LTP_result LTP_results] | |||
and for greater information click on each architecture to be redirected to their details. | |||
===Yocto 2.4 RC2 build=== | |||
{|border="1" | |||
|| '''Status''' || '''[[genericx86-64-ltp|genericx86-64]]''' || '''[[beaglebone-ltp|beaglebone]]''' || '''[[edgerouter-ltp|edgerouter]]''' || '''[[mpc8315e-rdb-ltp|mpc8315e-rdb]]''' | |||
|- | |||
|| total || 1391 || 1353 || 1353 || 1353 | |||
|- | |||
|| failures || 27 || 17 || || 30 | |||
|} | |||
===Yocto 2.4 RC1 build=== | |||
{|border="1" | |||
|| '''Status''' || '''[[genericx86-64-ltp|genericx86-64]]''' || '''[[beaglebone-ltp|beaglebone]]''' || '''[[edgerouter-ltp|edgerouter]]''' || '''[[mpc8315e-rdb-ltp|mpc8315e-rdb]]''' | |||
|- | |||
|| total || 1391 || 1353 || 1353 || 1353 | |||
|- | |||
|| failures || 27 || 18 || 34 || 31 | |||
|} | |||
=Detailed Report= | =Detailed Report= | ||
Latest revision as of 19:02, 24 October 2017
Testing data
Test type: Full Pass Test Branch: rocko Build name: yocto-2.4.rc2 poky commit: da05a01423a60a23cd1073eb0ad3776e5bf4f2a8 Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.4.rc2/ Distributions tested: CentOs 7.1 Debian 8 Fedora 25 Fedora 26 OpenSuse 422 Ubuntu 16.04 Ubuntu 17.04
{{#testopia: | run_id=0, {{#var:1_ADT_Fedora_26_i686}}, {{#var:2_ADT_Fedora_26_i686}}, {{#var:3_ADT_Ubuntu_16.04_x86_64}}, {{#var:4_ADT_Ubuntu_16.04_x86_64}}, {{#var:5_BitBake_Ubuntu_16.04_x86_64}}, {{#var:6_BSPs_Genericx86_MMAX32}}, {{#var:7_BSPs_Genericx86-64_NUC}}, {{#var:8_BSPs_Beaglebone_Black}}, {{#var:9_BSPs_EdgeRouter}}, {{#var:10_BSPs_MPC8315e-rdb}}, {{#var:11_BSPs_qemu-x86}}, {{#var:12_BSPs_qemuarm}}, {{#var:13_BSPs_qemuarm64}}, {{#var:14_BSPs_qemumips}}, {{#var:15_BSPs_qemumips64}}, {{#var:16_BSPs_qemuppc}}, {{#var:17_BSPs_qemux86-64}}, {{#var:18_BSPs_qemu-x86}}, {{#var:19_BSPs_qemuarm}}, {{#var:20_BSPs_qemuarm64}}, {{#var:21_BSPs_qemumips}}, {{#var:22_BSPs_qemumips64}}, {{#var:23_BSPs_qemuppc}}, {{#var:24_BSPs_qemux86-64,}}, {{#var:25_BSPs_qemux86-64-WIC,}}, {{#var:26_BSPs_qemux86-WIC}}, {{#var:27_BSPs_qemux86-lsb}}, {{#var:28_BSPs_qemux86_64-lsb}}, {{#var:29_BSPs_genericx86_-_on_MMAX32bit}}, {{#var:30_BSPs_genericx86-64_on_MMAX64}}, {{#var:31_BSPs_genericx86-64_on_NUC}}, {{#var:32_BSPs_genericx86-64-WIC_on_MMAX64}}, {{#var:33_BSPs_genericx86-64-WIC_on_NUC}}, {{#var:34_BSPs_genericx86-WIC_on_MMAX32}}, {{#var:35_BSPs_MPC8315e-rdb}}, {{#var:36_BSPs_Beaglebone_Black}}, {{#var:37_BSPs_EdgeRouter}}, {{#var:38_BSPs_genericx86-64-lsb_on_NUC}}, {{#var:39_BSPs_genericx86-64-lsb_on_MMAX64}}, {{#var:40_BSPs_genericx86-lsb_on_MMAX32bit}}, {{#var:41_CROPS_user-preference_(any_supported_environment)}}, {{#var:42_Eclipse Plugin_Fedora_26}}, {{#var:43_Eclipse Plugin_Fedora_26}}, {{#var:44_eSDK_Ubuntu_16.04}}, {{#var:45_eSDK_Ubuntu_16.04}}, {{#var:46_General Runtime_VMWare_Player_on_Ubuntu}}, {{#var:47_Kernel_SuSE_42.2}}, {{#var:48_Meta-yocto_Ubuntu_14.04_x86_64}}, {{#var:49_OE-Core_CentOS_7.1_x86_64}}, {{#var:50_OE-Core_Debian_8_x86_64}}, {{#var:51_OE-Core_Fedora_25_x86_64}}, {{#var:52_OE-Core_OpenSuse_422_x86_64}}, {{#var:53_OE-Core_ubuntu_16.04_x86_64}}, {{#var:54_OE-Core_Ubuntu_17.04_x86-64}}, {{#var:55_OE-Core_Fedora_26_x86_64}}, {{#var:56_Runtime_user-preference}}, {{#var:57_Toaster_Debian_4.3.0-0.bpo.1-amd64}}, {{#var:58_CROPS_user-preference_(any_supported_environment)}} | report_id=concatstats}}
|