WW44 - 2016-10-26 - Full Test Cycle 2.2 rc6

From Yocto Project
Revision as of 15:47, 27 October 2016 by Jose G Perez Carranza (talk | contribs) (Created page with "{| {{#vardefine:adt_fedora_i686| 6372, 6373}} {{#vardefine:adt_fedora_x86-64| }} {{#vardefine:adt_ubuntu_x86-64| 6370, 6371}} {{#vardefine:beaglebone_black| 6386, 6...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search
{{#vardefine:adt_fedora_i686| 6372, 6373}} {{#vardefine:adt_fedora_x86-64| }} {{#vardefine:adt_ubuntu_x86-64| 6370, 6371}} {{#vardefine:beaglebone_black| 6386, 6387}} {{#vardefine:edgerouter| 6388, 6389}} {{#vardefine:genericx86| 6358, 6365, 6392, 6394}} {{#vardefine:genericx86_64| 6367, 6397, 6399, 6400, 6401, 6402, 6403}} {{#vardefine:mpc8315e-rdb| 6390, 6391}} {{#vardefine:p1022ds-rdb| }} {{#vardefine:qemux86| 6368, 6396}} {{#vardefine:qemuarm| 6376, 6377, 6378, 6379}} {{#vardefine:qemumips| 6380, 6381, 6382, 6383}} {{#vardefine:qemuppc| 6384, 6385}} {{#vardefine:qemux86-64| 6369, 6393}} {{#vardefine:eclipseplugin| 6360, 6361}} {{#vardefine:meta-yocto| 6375}} {{#vardefine:eSDK| 6362, 6363}} {{#vardefine:toaster| 6357, 6364}} {{#vardefine:oe-core| 6348, 6349, 6350, 6351, 6352, 6353, 6355, 6354}} {{#vardefine:distro| }} {{#vardefine:bitbake| 6356}} {{#vardefine:runtime| 6395}} {{#vardefine:build-appliance| 6374}}

Testing data

 Test type:  Full Test Cycle
 Branch: master
 Build name: 2.2 rc4
 poky commit: 4b94b498e21aeba945fe7e72a6b7c4bb0314fb83
 Autobuilder images repository:  http://autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/
 Distributions tested: 
 -Ubuntu 16.04
 -Ubuntu 14 04 
 -OpenSuse 421
 -Opensuse 13.2  
 -Debian 8
 -Centos 7
 -Fedora 23
 -Fedora 24 

Test Run Summary Report

{{#testopia: run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:adt_ubuntu_1310_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}} report_id=concatstats}}


{{#testopia:

chart=bstatus report_id=bugs charttype=bar sortable=true nudechart=true chartpos=top

}}


{{#testopia:

run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}} charttype=bar report_id=status}}



Bugs Found during QA Test

New and notable issues Found during QA Test & Community

IDSummary (10 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18
10456X can not launch on qemumips64Medium+Armin KusterYi Zhao2.2.2VERIFIEDFIXEDRegression (Used to work)2016-10-19
10481[Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) WIC giving error: GPT: Use GNU Parted to correct GPT errors.MediumJair GonzalezJuan Ramos2.3 M1RESOLVEDFIXED---2016-10-21
10490Error on oeqa.runtime.rpm.RpmInstallRemoveTest when running testimageMedium+Jose Perez CJuan Ramos2.3 M2VERIFIEDNOTABUG---2016-10-24
10494[Test Case 1059] Parselogs is failing on genericx86-64-WIC on NUC giving error: Failed to load DMC firmwareMediumCalifornia Sullivan 2.2.2RESOLVEDFIXED---2016-10-24
eSDK
10447eSDK: runqemu fails unless unless DEPLOY_DIR_IMAGE is setMedium+Paul EggletonFrancisco Pedraza2.3 M4RESOLVEDFIXED---2016-10-17
Meta-yocto
10477yocto-bsp: tool should output a conf file to be consume by the qemu runnerMediumLeonardo Sandoval Gonzalez 2.5RESOLVEDOBSOLETE---2016-10-21
Runtime
10434runqemu can not launch qemu image in Build ApplianceMedium+Juro BystrickyYi Zhao2.3 M1VERIFIEDFIXEDRegression (Used to work)2016-10-14
Toaster
10463[Test Case 910] "Recipe file" column isn't sortableMediumMichael WoodLibertad2.3RESOLVEDNOTABUG---2016-10-19
10492[Test Case 1398] Build log cannot be downloadedMediumDavid ReynaDavid Lopez Barriba2.3 M1VERIFIEDINVALID---2016-10-24


High and M+ issues Found during QA Test (NOT NEW)

"None"


Other Bugs found during QA Test (NOT NEW)

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
10281Configuration variables: I can set a value for standard shell environment variable http_proxyLowToaster default assigneeyeoh ee pengFutureNEW---2016-09-14
Regression (Used to work)
10025Missing 'Connection Manager' icon for qemumips and qemumips64MediumJussi KukkonenYi Zhao2.4RESOLVEDWORKSFORMERegression (Used to work)2016-07-26
Toaster
8126Messages are missing when "IMAGE_FSTYPES" field is not properly editedMediumDavid ReynaDavid Lopez Barriba2.4 M4RESOLVEDFIXEDRegression (Used to work)2015-08-07

pTest open bugs

IDSummary (12 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
Runtime Testing
9373ptest for perl hangMediumAníbal Limón 2.3 M1RESOLVEDDUPLICATE---2016-04-01
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng4.99RESOLVEDFIXEDRegression (Used to work)2015-12-17
9372Ptest for acl hang because of lack of two user bin and daemonMediumChen Qioscar lopez arandas2.4RESOLVEDFIXED---2016-04-01
8656ptest - perl tests fail for genericx86-64 on NUC - 2.0.rc3MediumAníbal LimónFrancisco Pedraza2.3 M1RESOLVEDFIXEDRegression (Used to work)2015-11-06
9055pTest - acl (NUC) failed tested on NUC 5425WYK for YP 2.1_M2.rc1MediumRobert YangFrancisco Pedraza2.4RESOLVEDFIXEDRegression (Used to work)2016-02-04
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 4.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03
8146ptest - some test cases are failing for aclMediumAjay KumarRoss Burton2.2RESOLVEDDUPLICATE---2015-08-11
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
8831pTest - busybox tests fail for 2.1_M1.rc1Mediumdengke FutureRESOLVEDDUPLICATE---2015-12-17

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestone or release which are Medium+ or High and are not targeted for the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of this week's Yocto Bug Trend charts can be accessed on the following page

Build Performance Test

Detailed description on how performance test are run can be found on Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.2 M2. Performance execution has 2 stages, one executed form 1.6 to 2.1 in RO and another one started on 2.1 from GDC, there are difference between HW used in both location hence the numbers are different.

Below chars are showing the latest performance results taken from current milestone

Performance Charts

NOTE: Due bug 10285 performance results for "bitbake core-image-sato -c rootfs" measurements were failing on 2.2_M3.rc1, hence the numbers shown in the charts for that particular milestone are the ones reported on logs just before of the failure.

Performance Charts

For Bitbake core-image-sato with rm_work

Ww43-2k16 - rmwork.png

Building the Virtual kernel

Ww43-2k16 - kernel.png

Building the rootfs

Ww43-2k16 - rootfs.png

Building core-image-sato

Ww43-2k16 - sato.png

pTest results for genericx86-64 on NUC

< Back
Ran on a NUC and compared with commit 2679a347c576f5411fbe802d2f6201c94036ecb2

FULL LOG

Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 354 26 93.20% 96.00%
attr 143 143 0 100.00% 100.00%
bash 76 72 4 94.70% 95.00%
bluez5 25 25 0 100.00% 100.00%
busybox 487 480 7 98.60% 98.00%
bzip2 6 6 0 100.00% 100.00%
dbus-test 15 4 11 26.70% 27.00%
diffstat 440 440 0 100.00% 100.00%
diffutils 16 16 0 100.00% 100.00%
e2fsprogs 1 0 1 0.00% 0.00%
gawk 284 283 1 99.60% 100.00%
gdbm 22 22 0 100.00% 100.00%
gdk-pixbuf 17 17 0 100.00% 100.00%
glib-2.0 280 273 7 97.50% 98.00%
kbd 7 7 0 100.00%
libpcre 3 3 0 100.00% 100.00%
libxml2 85 82 3 96.50% 96.00%
lttng-tools 2467 2462 5 99.80% 99.00%
nettle 87 86 1 98.90% 99.00%
openssh 56 56 0 100.00% 100.00%
openssl 52 52 0 100.00% 100.00%
pango 12 12 0 100.00% 100.00%
perl 2263 2262 1 100.00% 100.00%
perl5 2263 2262 1 100.00% 100.00%
python 32056 31938 118 99.60% 100.00%
quilt 55 54 1 98.20% 100.00%
strace 264 263 1 99.60% 100.00%
tcl 148 148 0 100.00% 99.00%
util-linux 130 128 2 98.50% 100.00%
valgrind 538 2 536 0.40% 0.00%
zlib 2 2 0 100.00% 100.00%

Detailed Report

Distro Testing

Status {{#testopia: |run_id={{#var:distro}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:distro}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:distro}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Core Build System

Bitbake

Status {{#testopia: |run_id={{#var:bitbake}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:bitbake}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:bitbake}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Meta-Yocyo

Status {{#testopia: |run_id={{#var:meta-yocto}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:meta-yocto}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:meta-yocto}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

OE-Core

Status {{#testopia: |run_id={{#var:oe-core}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:oe-core}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:oe-core}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

eSDK

Status {{#testopia: |run_id={{#var:eSDK}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:eSDK}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:eSDK}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

BSP

genericx86: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:genericx86}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:genericx86}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:genericx86}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

genericx86_64: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:genericx86_64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:genericx86_64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:genericx86_64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Beaglebone Black: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:beaglebone_black}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:beaglebone_black}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:beaglebone_black}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

EdgeRouter: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:edgerouter}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:edgerouter}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:edgerouter}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

MPC8315e-rdb: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:mpc8315e-rdb}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:mpc8315e-rdb}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:mpc8315e-rdb}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

P1022ds-rdb: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:p1022ds-rdb}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:p1022ds-rdb}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:p1022ds-rdb}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}


Qemu

qemux86: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemux86}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemux86}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemux86}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemux86-64: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemux86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemux86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemux86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemuarm: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemuarm}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemuarm}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemuarm}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemumips: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemumips}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemumips}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemumips}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemuppc: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemuppc}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemuppc}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemuppc}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Toaster

Status {{#testopia: |run_id={{#var:toaster}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:toaster}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:toaster}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Runtime Testing

Status {{#testopia: |run_id={{#var:runtime}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:runtime}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:runtime}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Detailed result at: Build performance metrics

Build-appliance

Status {{#testopia: |run_id={{#var:build-appliance}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:build-appliance}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:build-appliance}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

ADT

ADT on: Fedora i686

Status {{#testopia: |run_id={{#var:adt_fedora_i686}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:adt_fedora_i686}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:adt_fedora_i686}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

ADT on: Fedora i686

Status {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

ADT on: Ubuntu x86-64

Status {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Eclipse Plugin

Status {{#testopia: |run_id={{#var:eclipseplugin}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:eclipseplugin}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:eclipseplugin}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}