WW31 - 2017-07-31 - Full Test Cycle 2.4 M2 rc2: Difference between revisions
(36 intermediate revisions by the same user not shown) | |||
Line 65: | Line 65: | ||
=Testing data= | =Testing data= | ||
<nowiki> | <nowiki> | ||
Test type: | Test type: Full Pass Test | ||
Branch: | Branch: master | ||
Build name: | Build name: 2.4 M2 RC2 | ||
poky commit: | poky commit: 8e15e9b6e478f6368034519b2a8fd3c7ea71d23b | ||
Autobuilder images repository: | Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.4_M2.rc2 | ||
Distributions tested: | Distributions tested: | ||
CentOS 7.1 | |||
Debian 8 | |||
Fedora 24 | |||
Fedora 25 | |||
OpenSuse 42.3 | |||
Ubuntu 16.04 | |||
Ubuntu 17.04 | |||
</nowiki> | </nowiki> | ||
{| | {| | ||
Line 96: | Line 103: | ||
|report_id=status}} | |report_id=status}} | ||
|} | |} | ||
= Bugs Found during QA Test = | = Bugs Found during QA Test = | ||
== <span style="color:blue; "> New issues Found during QA Test & Community </span>== | == <span style="color:blue; "> New issues Found during QA Test & Community </span>== | ||
Line 103: | Line 111: | ||
|sort=ID | |sort=ID | ||
|group=product | |group=product | ||
|id= | |id=11855, 11859, 11868, 11869, 11871,11879, 11873, 11850, 11889 | ||
}} | }} | ||
<br> | <br> | ||
== <span style="color: | == <span style="color:green; "> New issues Found and solved during QA Test cycle but commit not integrated in 2.4 M2 RC2 </span>== | ||
{{#bugzilla: | {{#bugzilla: | ||
|columns=id,summary,priority,to,qa,milestone,status,resolution,cf_regression_type,created | |columns=id,summary,priority,to,qa,milestone,status,resolution,cf_regression_type,created | ||
|noresultsmessage="None" | |noresultsmessage="None" | ||
|sort= | |sort=ID | ||
|group=product | |group=product | ||
|id= | |id=11834, 11857 | ||
}}<br> | }} | ||
<br> | |||
== Other Bugs found during QA Test (NOT NEW) == | == Other Bugs found during QA Test (NOT NEW) == | ||
Line 122: | Line 131: | ||
|sort=priority | |sort=priority | ||
|group=cf_regression_type,product | |group=cf_regression_type,product | ||
|id= | |id=9267, 10455, 10477, 11812 | ||
}} | }} | ||
Line 137: | Line 146: | ||
|total=summary | |total=summary | ||
|status=!(RESOLVED,VERIFIED) | |status=!(RESOLVED,VERIFIED) | ||
|milestone=2.4 | |milestone=2.4, 2.4 M1, 2.4 M2 | ||
}} | }} | ||
Line 144: | Line 153: | ||
= Build Performance Test = | = Build Performance Test = | ||
Detailed description on how performance test | Detailed description on how performance test are run can be found on the [[Performance Test]] wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.4 M1. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and another one started on 2.1 from GDC. There are differences between HW used in both locations, hence the numbers are different. | ||
The | |||
* [https://wiki.yoctoproject.org/charts/perf_milestone/performance_test.html RO Charts] | * [https://wiki.yoctoproject.org/charts/perf_milestone/performance_test.html RO Charts] | ||
* [https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.html GDC Charts] | * [https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.html GDC Charts] | ||
Below | Below charts are showing the latest performance results taken from the current milestone. | ||
== Performance Charts== | == Performance Charts== | ||
Line 187: | Line 195: | ||
|sort=status | |sort=status | ||
|group=product | |group=product | ||
|id= | |id=11596, 10266, 11602, 11668, 6707, 5698, 8832, 9042, 8471,8828, 5697, 10953,8650,11024,5695,5605,4437,4436,6737,11674,10841,8064,11667,11666,11547,1181,11812,11813}} | ||
}} | |||
== results == | == results == | ||
{{:Ptest | {{:Ptest 8e15e9b6e478f6368034519b2a8fd3c7ea71d23b}} | ||
=Compliance Report= | |||
==POSIX results== | |||
General results for POSIX execution can be found [https://wiki.yoctoproject.org/wiki/Posix_result POSIX_result] | |||
and greater details on each architecture can be found [https://wiki.yoctoproject.org/wiki/POSIX-results POSIX_details] | |||
Bellow you will see the last two execution results for comparison: | |||
===Yocto 2.4 M2 RC2 build=== | |||
{|border="1" | |||
|| '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb''' | |||
|- | |||
|| failures || 46 || 45 || 46 || 45 | |||
|} | |||
===Yocto 2.4 M1 RC1 build=== | |||
{|border="1" | |||
|| '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb''' | |||
|- | |||
|| failures || 45 || 45 || 46 || 45 | |||
|} | |||
==LTP results== | |||
General results for LTP can be found here [https://wiki.yoctoproject.org/wiki/LTP_result LTP_results] | |||
and for greater information click on each architecture to be redirected to their details. There is currently an infrastructure issue that is blocking the beblebone and edgerouter testing for LTP, results are not shown in the tables below. | |||
===Yocto 2.4 M2 RC2 build=== | |||
{|border="1" | |||
|| '''Status''' || '''[[genericx86-64-ltp|genericx86-64]]''' || '''[[beaglebone-ltp|beaglebone]]''' || '''[[edgerouter-ltp|edgerouter]]''' || '''[[mpc8315e-rdb-ltp|mpc8315e-rdb]]''' | |||
|- | |||
|| total || 1384 || || || 1348 | |||
|- | |||
|| failures || 24 || || || 30 | |||
|} | |||
===Yocto 2.4 M1 RC1 build=== | |||
{|border="1" | |||
|| '''Status''' || '''[[genericx86-64-ltp|genericx86-64]]''' || '''[[beaglebone-ltp|beaglebone]]''' || '''[[edgerouter-ltp|edgerouter]]''' || '''[[mpc8315e-rdb-ltp|mpc8315e-rdb]]''' | |||
|- | |||
|| total || 1384 || || || 1348 | |||
|- | |||
|| failures || 22 || || || 30 | |||
|} | |||
=Detailed Report= | =Detailed Report= | ||
Latest revision as of 23:50, 2 August 2017
Testing data
Test type: Full Pass Test Branch: master Build name: 2.4 M2 RC2 poky commit: 8e15e9b6e478f6368034519b2a8fd3c7ea71d23b Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.4_M2.rc2 Distributions tested: CentOS 7.1 Debian 8 Fedora 24 Fedora 25 OpenSuse 42.3 Ubuntu 16.04 Ubuntu 17.04
{{#testopia: | run_id=0, {{#var:1_ADT_Fedora_25_i686}}, {{#var:2_ADT_Fedora_26_i686}}, {{#var:3_ADT_Ubuntu_16.04_x86_64}}, {{#var:4_ADT_Ubuntu_16.04_x86_64}}, {{#var:5_BitBake_Fedora_24_x86_64}}, {{#var:6_BSPs_Genericx86_MMAX32}}, {{#var:7_BSPs_Genericx86-64_NUC}}, {{#var:8_BSPs_Beaglebone-Black}}, {{#var:9_BSPs_EdgeRouter}}, {{#var:10_BSPs_MPC8315e-rdb}}, {{#var:11_BSPs_qemu-x86}}, {{#var:12_BSPs_qemuarm}}, {{#var:13_BSPs_qemuarm64}}, {{#var:14_BSPs_qemumips}}, {{#var:15_BSPs_qemumips64}}, {{#var:16_BSPs_qemuppc}}, {{#var:17_BSPs_qemux86-64}}, {{#var:18_BSPs_qemu-x86}}, {{#var:19_BSPs_qemuarm}}, {{#var:20_BSPs_qemuarm64}}, {{#var:21_BSPs_qemumips}}, {{#var:22_BSPs_qemumips64}}, {{#var:23_BSPs_qemuppc}}, {{#var:24_BSPs_qemux86-64,}}, {{#var:25_BSPs_qemux86-64-WIC,}}, {{#var:26_BSPs_qemux86-WIC}}, {{#var:27_BSPs_qemux86-lsb}}, {{#var:28_BSPs_qemux86_64-lsb}}, {{#var:29_BSPs_genericx86_-_on_MMAX32bit}}, {{#var:30_BSPs_genericx86-64_on_MMAX64}}, {{#var:31_BSPs_genericx86-64_on_NUC}}, {{#var:32_BSPs_genericx86-64-WIC_on_MMAX64}}, {{#var:33_BSPs_genericx86-64-WIC_on_NUC}}, {{#var:34_BSPs_genericx86-WIC_on_MMAX32}}, {{#var:35_BSPs_MPC8315e-rdb}}, {{#var:36_BSPs_Beaglebone_Black}}, {{#var:37_BSPs_EdgeRouter}}, {{#var:38_BSPs_genericx86-64-lsb_on_NUC}}, {{#var:39_BSPs_genericx86-64-lsb_on_MMAX64}}, {{#var:40_BSPs_genericx86-lsb_on_MMAX32bit}}, {{#var:41_CROPS_Centos_7.1}}, {{#var:42_Eclipse Plugin_Mars_on_Fedora_22_x86_64,}}, {{#var:43_Eclipse Plugin_Neon_on_Fedora_22_x86_64}}, {{#var:44_eSDK_Centos_7.1}}, {{#var:45_eSDK_Centos_7.1}}, {{#var:46_General Runtime_VMWare_Player_on_Ubuntu}}, {{#var:47_Kernel_Centos_7}}, {{#var:48_Meta-yocto_Ubuntu_14.04_x86_64}}, {{#var:49_OE-Core_CentOS_7.1_x86_64}}, {{#var:50_OE-Core_Debian_8_x86_64}}, {{#var:51_OE-Core_Fedora_25_x86_64}}, {{#var:52_OE-Core_Fedora_24_x86_64}}, {{#var:53_OE-Core_OpenSuse_42.3_x86_64}}, {{#var:54_OE-Core_OpenSuse421_x86_64}}, {{#var:55_OE-Core_ubuntu_16.04_x86_64}}, {{#var:56_OE-Core_Ubuntu_17.04_x86-64}}, {{#var:57_OE-Core_Fedora_26_x86_64}}, {{#var:58_Runtime_user-preference}}, {{#var:59_Toaster_Debian_4.3.0-0.bpo.1-amd64}}, {{#var:60_Toaster_Debian_4.3.0-0.bpo.1-amd64}}, {{#var:61_Package Management Issues_Any_Supported_Distro}} | report_id=concatstats}}
|