WW40 - 2017-10-06 - Full Test Cycle 2.4 RC1: Difference between revisions
(21 intermediate revisions by 2 users not shown) | |||
Line 68: | Line 68: | ||
=Testing data= | =Testing data= | ||
<nowiki> | <nowiki> | ||
Test type: | Test type: Full Pass Test Cycle | ||
Branch: | Branch: master | ||
Build name: | Build name: yocto-2.4.rc1 | ||
poky commit: | poky commit: 592c3449697ce39bf07864fafd7f0a61ca230561 | ||
Autobuilder images repository: | Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.4.rc1 | ||
Distributions tested: | Distributions tested: | ||
CentOS 7.1 | |||
Debian 8 | |||
Debian 9 | |||
Fedora 24 | |||
Fedora 25 | |||
Ubuntu 16.04 | |||
Ubuntu 17.04 | |||
OpenSuse 422 | |||
</nowiki> | </nowiki> | ||
{| | {| | ||
Line 99: | Line 108: | ||
|report_id=status}} | |report_id=status}} | ||
|} | |} | ||
= Bugs Found during QA Test = | = Bugs Found during QA Test = | ||
== <span style="color:blue; "> New issues Found during QA Test & Community </span>== | == <span style="color:blue; "> New issues Found during QA Test & Community </span>== | ||
Line 106: | Line 116: | ||
|sort=ID | |sort=ID | ||
|group=product | |group=product | ||
|id=12131, 12133, 12138, 12140, 12142, 12150, 12164, 12168, 12170, 12171, 12175, 12178 | |id=12131, 12133, 12138, 12140, 12142, 12150, 12164, 12168, 12170, 12171, 12175, 12178, 12143, 12144, 12145, 12146, 12194, 12200, 12201 | ||
}} | }} | ||
<br> | <br> | ||
== <span style="color: red; "> High and M+ | == <span style="color: red; "> High and M+ issues found during QA test (NOT NEW) </span>== | ||
{{#bugzilla: | {{#bugzilla: | ||
|columns=id,summary,priority,to,qa,milestone,status,resolution,cf_regression_type,created | |columns=id,summary,priority,to,qa,milestone,status,resolution,cf_regression_type,created | ||
Line 129: | Line 139: | ||
= Current community Open Bugs - Medium+/High = | = Current community Open Bugs - Medium+/High = | ||
Overview of the open bugs opened against current | Overview of the open bugs opened against current milestone or release which are Medium+ or High and are not targeted for the next milestones | ||
{{#bugzilla: | {{#bugzilla: | ||
Line 147: | Line 157: | ||
= Build Performance Test = | = Build Performance Test = | ||
Detailed description on how performance test is run | Detailed description on how performance test is run can be found on the [[Performance Test]] wiki. | ||
The | The below charts are displaying the build performance trend for 2.1M4 to 2.4 RC1. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and executed from 2.1 to 2.4 in GDC and are differences between HW used in both locations, hence the numbers are different. | ||
* [https://wiki.yoctoproject.org/charts/perf_milestone/performance_test.html RO Charts] | * [https://wiki.yoctoproject.org/charts/perf_milestone/performance_test.html RO Charts] | ||
* [https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.html GDC Charts] | * [https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.html GDC Charts] | ||
Below chars are showing the | Below chars are showing the latest performance results taken from the current milestone. | ||
== Performance Charts== | == Performance Charts== | ||
Line 190: | Line 200: | ||
|sort=status | |sort=status | ||
|group=product | |group=product | ||
|id= | |id=12094, 12143,12144, 12145, 12146, 6707, 11942, 12181, 4436, 4437, 5697, 5698, 6737, 8064, 8471, 8650, 8828, 8832, 9042, 10266, 10841, 10953, 11024,11674, 11811, 11812, 11813, 11997, 11998, 5605, 5695 | ||
}} | }} | ||
== results == | == results == | ||
{{:Ptest | {{:Ptest 592c3449697ce39bf07864fafd7f0a61ca230561}} | ||
=Compliance Report= | |||
==POSIX results== | |||
General results for POSIX execution can be found [https://wiki.yoctoproject.org/wiki/Posix_result POSIX_result] | |||
and greater details on each architecture can be found [https://wiki.yoctoproject.org/wiki/POSIX-results POSIX_details] | |||
Bellow you will see the last three execution results for comparison: | |||
===Yocto 2.4 RC1 build=== | |||
{|border="1" | |||
|| '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb''' | |||
|- | |||
|| failures || 46 || 45 || 47 || 45 | |||
|} | |||
===Yocto 2.4 M3 RC1 build=== | |||
{|border="1" | |||
|| '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb''' | |||
|- | |||
|| failures || 46 || 45 || 46 || 45 | |||
|} | |||
===Yocto 2.4 M2 RC3 build=== | |||
{|border="1" | |||
|| '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb''' | |||
|- | |||
|| failures || 46 || 45 || 46 || 45 | |||
|} | |||
==LTP results== | |||
General results for LTP can be found here [https://wiki.yoctoproject.org/wiki/LTP_result LTP_results] | |||
and for greater information click on each architecture to be redirected to their details. | |||
===Yocto 2.4 RC1 build=== | |||
{|border="1" | |||
|| '''Status''' || '''[[genericx86-64-ltp|genericx86-64]]''' || '''[[beaglebone-ltp|beaglebone]]''' || '''[[edgerouter-ltp|edgerouter]]''' || '''[[mpc8315e-rdb-ltp|mpc8315e-rdb]]''' | |||
|- | |||
|| total || 1391 || 1353 || 1353 || 1353 | |||
|- | |||
|| failures || 27 || 18 || 34 || 31 | |||
|} | |||
===Yocto 2.4 M3 RC1 build=== | |||
{|border="1" | |||
|| '''Status''' || '''[[genericx86-64-ltp|genericx86-64]]''' || '''[[beaglebone-ltp|beaglebone]]''' || '''[[edgerouter-ltp|edgerouter]]''' || '''[[mpc8315e-rdb-ltp|mpc8315e-rdb]]''' | |||
|- | |||
|| total || 1391 || 1353 || 1353 || 1353 | |||
|- | |||
|| failures || 26 || 17 || 32 || 31 | |||
|} | |||
===Yocto 2.4 M2 RC3 build=== | |||
{|border="1" | |||
|| '''Status''' || '''[[genericx86-64-ltp|genericx86-64]]''' || '''[[beaglebone-ltp|beaglebone]]''' || '''[[edgerouter-ltp|edgerouter]]''' || '''[[mpc8315e-rdb-ltp|mpc8315e-rdb]]''' | |||
|- | |||
|| total || 1384 || 1346 || || 1346 | |||
|- | |||
|| failures || 23 || 15 || || 27 | |||
|} | |||
=Detailed Report= | =Detailed Report= | ||
Latest revision as of 23:40, 6 October 2017
Testing data
Test type: Full Pass Test Cycle Branch: master Build name: yocto-2.4.rc1 poky commit: 592c3449697ce39bf07864fafd7f0a61ca230561 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.4.rc1 Distributions tested: CentOS 7.1 Debian 8 Debian 9 Fedora 24 Fedora 25 Ubuntu 16.04 Ubuntu 17.04 OpenSuse 422
{{#testopia: | run_id=0, {{#var:1_ADT_Fedora_26_i686}}, {{#var:2_ADT_Fedora_26_i686}}, {{#var:3_ADT_Ubuntu_16.04_x86_64}}, {{#var:4_ADT_Ubuntu_16.04_x86_64}}, {{#var:5_BitBake_Ubuntu_16.04_x86_64}}, {{#var:6_BSPs_Genericx86_MMAX32}}, {{#var:7_BSPs_Genericx86-64_NUC}}, {{#var:8_BSPs_Beaglebone_Black}}, {{#var:9_BSPs_EdgeRouter}}, {{#var:10_BSPs_MPC8315e-rdb}}, {{#var:11_BSPs_qemu-x86}}, {{#var:12_BSPs_qemuarm}}, {{#var:13_BSPs_qemuarm64}}, {{#var:14_BSPs_qemumips}}, {{#var:15_BSPs_qemumips64}}, {{#var:16_BSPs_qemuppc}}, {{#var:17_BSPs_qemux86-64}}, {{#var:18_BSPs_qemu-x86}}, {{#var:19_BSPs_qemuarm}}, {{#var:20_BSPs_qemuarm64}}, {{#var:21_BSPs_qemumips}}, {{#var:22_BSPs_qemumips64}}, {{#var:23_BSPs_qemuppc}}, {{#var:24_BSPs_qemux86-64,}}, {{#var:25_BSPs_qemux86-64-WIC,}}, {{#var:26_BSPs_qemux86-WIC}}, {{#var:27_BSPs_qemux86-lsb}}, {{#var:28_BSPs_qemux86_64-lsb}}, {{#var:29_BSPs_genericx86_-_on_MMAX32bit}}, {{#var:30_BSPs_genericx86-64_on_MMAX64}}, {{#var:31_BSPs_genericx86-64_on_NUC}}, {{#var:32_BSPs_genericx86-64-WIC_on_MMAX64}}, {{#var:33_BSPs_genericx86-64-WIC_on_NUC}}, {{#var:34_BSPs_genericx86-WIC_on_MMAX32}}, {{#var:35_BSPs_MPC8315e-rdb}}, {{#var:36_BSPs_Beaglebone_Black}}, {{#var:37_BSPs_EdgeRouter}}, {{#var:38_BSPs_genericx86-64-lsb_on_NUC}}, {{#var:39_BSPs_genericx86-64-lsb_on_MMAX64}}, {{#var:40_BSPs_genericx86-lsb_on_MMAX32bit}}, {{#var:41_CROPS_Fedora_26}}, {{#var:42_Eclipse Plugin_Mars_on_Fedora_22_x86_64,}}, {{#var:43_Eclipse Plugin_Neon_on_Fedora_22_x86_64}}, {{#var:44_eSDK_Ubuntu_16.04}}, {{#var:45_eSDK_Ubuntu_16.04}}, {{#var:46_General Runtime_VMWare_Player_on_Ubuntu}}, {{#var:47_Kernel_Fedora_24}}, {{#var:48_Meta-yocto_Ubuntu_16.04_x86_64}}, {{#var:49_OE-Core_CentOS_7.1_x86_64}}, {{#var:50_OE-Core_Debian_8_x86_64}}, {{#var:51_OE-Core_Fedora_25_x86_64}}, {{#var:52_OE-Core_Fedora_24_x86_64}}, {{#var:53_OE-Core_OpenSuse_422_x86_64}}, {{#var:54_OE-Core_ubuntu_16.04_x86_64}}, {{#var:55_OE-Core_Ubuntu_17.04_x86-64}}, {{#var:56_Runtime_VMWare_Player_on_Ubuntu}}, {{#var:57_Toaster_Fedora_26}}, {{#var:58_Toaster_Fedora_26}}, {{#var:59_Package Management Issues_Any_Supported_Distro}}, {{#var:60_OE-Core_Debian_9_x86_64}}, {{#var:61_BSPs_corei7_on_NUC}}, {{#var:62_Eclipse Plugin_Oxygen_on_ubuntu_1604}}, {{#var:63_CROPS_Fedora_26}}, {{#var:64_CROPS_user-preference_(any_supported_environment)}} | report_id=concatstats}}
|