WW17 - 2017-04-21 - Full Test Cycle 2.3 rc2
Testing data
Test type: Full Test Cycle Branch: master Build name: 2.3 rc2 poky commit: 7a0e795373653886452a7a2992ced10080711c26 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3.rc2/ Distributions tested: -Ubuntu 16.10 -Ubuntu 16.04 -OpenSuse 42.2 -Opensuse 13.2 -Debian 8 -Centos 7 -Fedora 24 -Fedora 25
Test Run Summary Report
AutBuild-Testing
{{#testopia: | run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:kernel}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:adt_ubuntu_1310_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}}, {{#var:AutBuild-Testing}}, {{#var:CROPS}} | report_id=concatstats}}
|
chart=bstatus | report_id=bugs | charttype=bar | sortable=true | nudechart=true | chartpos=top
}}
|
run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:kernel}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}}, {{#var:AutBuild-Testing}}, {{#var:CROPS}} | charttype=bar | report_id=status}} |
Bugs Found during QA Test
New issues Found during QA Test & Community
ID | Summary (3 tasks) ⇒ | P | Assignee | QA | Milestone | Status | Resolution | cf_regression_type | Created |
---|---|---|---|---|---|---|---|---|---|
CROPS | |||||||||
11421 | Crops-ESDK: devtool build v4l2loopback-driver, fails when trying to access /lib/modules/<laptop_kernel_version> | Medium | Todor Minchev | Libertad | 2.3 M4 | RESOLVED | FIXED | --- | 2017-04-26 |
11431 | Crops-ESDK: devtool is unable to add recipe for nodejs-native recipe | Medium | Anuj Mittal | nagarajagopal | 2.7 | RESOLVED | NOTABUG | --- | 2017-04-27 |
OE-Core | |||||||||
11409 | runqemu-extract-sdk doesn't work in toolchain for 2.3 rc2 build | Medium+ | Robert Yang | Yi Zhao | 2.3 M4 | VERIFIED | FIXED | Regression (Used to work) | 2017-04-25 |
High and M+ issues Found during QA Test (NOT NEW)
eSDK | |||||||||
11300 | selftest: do_populate_sdk_ext task failing on eSDK test on GDC ABs | Medium+ | Jose Perez C | oscar lopez arandas | 2.4 M1 | VERIFIED | FIXED | --- | 2017-04-05 |
Other Bugs found during QA Test (NOT NEW)
ID | Summary (3 tasks) ⇒ | P | Assignee | QA | Milestone | Status | Resolution | cf_regression_type | Created |
---|---|---|---|---|---|---|---|---|---|
--- | |||||||||
10455 | [morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0 | Medium | California Sullivan | yeoh ee peng | 2.5 M3 | RESOLVED | FIXED | --- | 2016-10-18 |
Regression (Used to work) | |||||||||
10025 | Missing 'Connection Manager' icon for qemumips and qemumips64 | Medium | Jussi Kukkonen | Yi Zhao | 2.4 | RESOLVED | WORKSFORME | Regression (Used to work) | 2016-07-26 |
General Runtime | |||||||||
10858 | Can not stop tcf-agent daemon | Medium | brian avery | Yi Zhao | 2.4 M1 | VERIFIED | FIXED | Regression (Used to work) | 2016-12-26 |
Current community Open Bugs - Medium+/High
Overview of the open bugs opened against current milestone or release which are Medium+ or High and are not targeted for the next milestones
ID | P | Summary | Assignee | QA | Milestone | Status | cf_regression_type | Created |
---|---|---|---|---|---|---|---|---|
0 | ||||||||
BitBake | ||||||||
10984 | Medium+ | [Test Case 1058] bitbake can use alternate download methods than git | Richard Purdie | oscar lopez arandas | 2.3 M3 | CLOSED | --- | 2017-01-27 |
1 |
Bug Trend
An overview of this week's Yocto Bug Trend charts can be accessed on the following page
Build Performance Test
Detailed description on how performance test is run can be found on the Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.3 M3. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and another one started on 2.1 from GDC. There are differences between HW used in both locations, hence the numbers are different.
Below chars are showing the latest performance results taken from the current milestone.
Performance Charts
NOTE: During the 2.3 M2 QA cycle period, the Fedora machine was under debugging activities to investigate more on bug 10884, hence only results for the Ubuntu machine are displayed on the charts.
Performance Charts
For Bitbake core-image-sato with rm_work
Building the Virtual kernel
Building the rootfs
Building core-image-sato
eSDK Deploy time
pTest for genericx86-64 on NUC
open bugs
ID | Summary (12 tasks) ⇒ | P | Assignee | QA | Milestone | Status | Resolution | cf_regression_type | Created |
---|---|---|---|---|---|---|---|---|---|
OE-Core | |||||||||
5697 | busybox: ptest cases failed | Medium | Changqing Li | 2.7 M2 | RESOLVED | FIXED | --- | 2013-12-31 | |
5695 | python: ptest cases failed | Medium | Ross Burton | 2.6 | RESOLVED | FIXED | --- | 2013-12-31 | |
10841 | pTest : Various test cases are failing on "dbus-test" module | Medium | Ross Burton | 2.5 | RESOLVED | DUPLICATE | --- | 2016-12-21 | |
10840 | pTest- openssl: test cases "test_bn" and "test_cms" are failing | Medium+ | Alexander Kanavin | 2.4 M1 | VERIFIED | FIXED | --- | 2016-12-21 | |
10842 | testrunner test timeout can lead to garbage output | Medium | Aníbal Limón | 2.3 M4 | VERIFIED | FIXED | --- | 2016-12-21 | |
Runtime Testing | |||||||||
8831 | pTest - busybox tests fail for 2.1_M1.rc1 | Medium | dengke | Future | RESOLVED | DUPLICATE | --- | 2015-12-17 | |
8828 | pTest - lttng-tools tests fail | Medium | Richard Purdie | yeoh ee peng | 4.99 | RESOLVED | FIXED | Regression (Used to work) | 2015-12-17 |
9372 | Ptest for acl hang because of lack of two user bin and daemon | Medium | Chen Qi | oscar lopez arandas | 2.4 | RESOLVED | FIXED | --- | 2016-04-01 |
8471 | ptest - some valgrind ptests fail on 2.4 M1.rc1 | Medium | Maxin B. John | 4.99 | RESOLVED | DUPLICATE | Regression (Used to work) | 2015-10-09 | |
9055 | pTest - acl (NUC) failed tested on NUC 5425WYK for YP 2.1_M2.rc1 | Medium | Robert Yang | Francisco Pedraza | 2.4 | RESOLVED | FIXED | Regression (Used to work) | 2016-02-04 |
9042 | pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1 | Medium | Changqing Li | 2.7 M2 | RESOLVED | FIXED | Regression (Used to work) | 2016-02-03 | |
8832 | pTest - bash tests fail for 2.1_M1.rc1 | Medium | Changqing Li | 2.7 M2 | RESOLVED | WORKSFORME | --- | 2015-12-17 |
results
< Back
Ran on a NUC and compared with commit Ptest 415b72ffcbd26e5f3664370d8b2a9b8105fb6342
Failures |
% increased |
% decreased |
Package Name | Tests ran | Passed | Failed | Passrate | Last Passrate |
acl | 380 | 380 | 0 | 100.00% | 100.00% |
attr | 143 | 143 | 0 | 100.00% | 100.00% |
bash | 76 | 72 | 4 | 94.70% | 94.70% |
bluez5 | 25 | 25 | 0 | 100.00% | 100.00% |
busybox | 486 | 478 | 8 | 98.40% | 98.40% |
bzip2 | 6 | 6 | 0 | 100.00% | 100.00% |
dbus-test | 15 | 4 | 11 | 26.70% | 26.70% |
diffstat | 440 | 440 | 0 | 100.00% | 100.00% |
diffutils | 17 | 17 | 0 | 100.00% | 100.00% |
e2fsprogs | 1 | 1 | 0 | 100.00% | 100.00% |
flex | 114 | 114 | 0 | 100.00% | 100.00% |
gawk | 298 | 297 | 1 | 99.70% | 99.70% |
gdbm | 22 | 22 | 0 | 100.00% | 100.00% |
gdk-pixbuf | 20 | 20 | 0 | 100.00% | 100.00% |
glib-2.0 | 284 | 277 | 7 | 97.50% | 97.50% |
kbd | 7 | 7 | 0 | 100.00% | 100.00% |
libpcre | 3 | 3 | 0 | 100.00% | 100.00% |
libxml2 | 85 | 82 | 3 | 96.50% | 96.50% |
lttng-tools | 2057 | 1988 | 69 | 96.60% | 96.60% |
nettle | 88 | 87 | 1 | 98.90% | 98.90% |
openssh | 34 | 33 | 1 | 97.10% | 97.10% |
openssl | 52 | 52 | 0 | 100.00% | 96.20% |
pango | 12 | 12 | 0 | 100.00% | 100.00% |
perl | 2286 | 2285 | 1 | 100.00% | 100.00% |
perl5 | 2286 | 2285 | 1 | 100.00% | 100.00% |
python | 32230 | 32212 | 118 | 99.60% | 99.60% |
quilt | 57 | 57 | 0 | 100.00 | 33.30% |
strace | 335 | 334 | 1 | 99.70% | 99.70% |
tcl | 148 | 146 | 2 | 98.60% | 98.60% |
util-linux | 108 | 106 | 2 | 98.10% | 98.10% |
valgrind | 553 | 296 | 257 | 53.50% | 53.50% |
zlib | 2 | 2 | 0 | 100.00% | 100.00% |
Detailed Report
Distro Testing
Status {{#testopia: |run_id={{#var:distro}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:distro}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:distro}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
Core Build System
Bitbake
Status {{#testopia: |run_id={{#var:bitbake}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:bitbake}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:bitbake}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
Meta-Yocyo
Status {{#testopia: |run_id={{#var:meta-yocto}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:meta-yocto}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:meta-yocto}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
OE-Core
Status {{#testopia: |run_id={{#var:oe-core}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:oe-core}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:oe-core}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
eSDK
Status {{#testopia: |run_id={{#var:eSDK}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:eSDK}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:eSDK}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
BSP
genericx86: core-image-sato-sdk
Status {{#testopia: |run_id={{#var:genericx86}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:genericx86}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:genericx86}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
genericx86_64: core-image-sato-sdk
Status {{#testopia: |run_id={{#var:genericx86_64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:genericx86_64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:genericx86_64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
Beaglebone Black: core-image-sato-sdk
Status {{#testopia: |run_id={{#var:beaglebone_black}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:beaglebone_black}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:beaglebone_black}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
EdgeRouter: core-image-sato-sdk
Status {{#testopia: |run_id={{#var:edgerouter}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:edgerouter}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:edgerouter}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
MPC8315e-rdb: core-image-sato-sdk
Status {{#testopia: |run_id={{#var:mpc8315e-rdb}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:mpc8315e-rdb}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:mpc8315e-rdb}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
P1022ds-rdb: core-image-sato-sdk
Status {{#testopia: |run_id={{#var:p1022ds-rdb}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:p1022ds-rdb}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:p1022ds-rdb}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
Qemu
qemux86: core-image-sato-sdk
Status {{#testopia: |run_id={{#var:qemux86}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:qemux86}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:qemux86}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
qemux86-64: core-image-sato-sdk
Status {{#testopia: |run_id={{#var:qemux86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:qemux86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:qemux86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
qemuarm: core-image-sato-sdk
Status {{#testopia: |run_id={{#var:qemuarm}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:qemuarm}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:qemuarm}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
qemumips: core-image-sato-sdk
Status {{#testopia: |run_id={{#var:qemumips}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:qemumips}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:qemumips}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
qemuppc: core-image-sato-sdk
Status {{#testopia: |run_id={{#var:qemuppc}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:qemuppc}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:qemuppc}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
Toaster
Status {{#testopia: |run_id={{#var:toaster}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:toaster}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:toaster}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
Runtime Testing
Status {{#testopia: |run_id={{#var:runtime}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:runtime}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:runtime}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
Detailed result at: Build performance metrics
Build-appliance
Status {{#testopia: |run_id={{#var:build-appliance}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:build-appliance}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:build-appliance}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
ADT
ADT on: Fedora i686
Status {{#testopia: |run_id={{#var:adt_fedora_i686}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:adt_fedora_i686}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:adt_fedora_i686}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
ADT on: Fedora i686
Status {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
ADT on: Ubuntu x86-64
Status {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
Eclipse Plugin
Status {{#testopia: |run_id={{#var:eclipseplugin}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:eclipseplugin}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:eclipseplugin}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}
Kernel
Status {{#testopia: |run_id={{#var:kernel}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}
Bugs {{#testopia: |run_id={{#var:kernel}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}
Agenda {{#testopia: |run_id={{#var:kernel}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}