WW44 - 2016-10-26 - Full Test Cycle 2.2 rc6

From Yocto Project
Jump to: navigation, search

This is a Partial Test Cycle detached form 2.2 rc4, so not all the components were executed and this report should be complemented with the rc4 for a full set of results

WW43 - 2016-10-18 - Full_Test_Cycle_2.2_rc4

Contents

Testing data

 Test type:  Partial Test Cycle for Dirty COW fix 
 Branch: morty
 Build name: 2.2 rc6
 poky commit: dc8508f609974cc99606b9042bfa7f870ce80228
 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc6/
 Distributions tested: 
 -Ubuntu 16.04
 -Ubuntu 14 04 
 -OpenSuse 421
 -Opensuse 13.2  
 -Debian 8
 -Centos 7
 -Fedora 23
 -Fedora 24 

Test Run Summary Report

Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
6423 Meta-yocto: master branchUbuntu 14.04 x86_6491.7%10477 8.3%10519 10521 0.0%none100.0%
6440 Build-Appliance: master branchVMWare Player on Ubuntu75.0%none25.0%10504 10434 0.0%none100.0%
6405 OE-Core: master branchCentOS 7.1 x86_64100.0%none0.0%none0.0%none100.0%
6404 OE-Core: master branchDebian 8 x86_64100.0%none0.0%none0.0%none100.0%
6406 OE-Core: master branchFedora 23 x86_64100.0%none0.0%none0.0%none100.0%
6407 OE-Core: master branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
6408 OE-Core: master branchOpenSuse 13.2 x86_64100.0%none0.0%none0.0%none100.0%
6409 OE-Core: master branchOpenSuse421 x86_64100.0%none0.0%none0.0%none100.0%
6410 OE-Core: master branchubuntu 14.04 GDC100.0%none0.0%none0.0%none100.0%
6411 OE-Core: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
6434 6435 BSP/QEMU: master branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
6436 6437 BSP/QEMU: master branchEdgeRouter100.0%none0.0%none0.0%none100.0%
6412 BSP/QEMU: master branchgenericx86 - on MMAX32bit98.1%none1.9%10455 0.0%none100.0%
6416 BSP/QEMU: master branchgenericx86-64 on MMAX6498.1%none1.9%10455 0.0%none100.0%
6414 BSP/QEMU: master branchgenericx86-64 on NUC98.1%none1.9%10494 0.0%none100.0%
6413 BSP/QEMU: master branchgenericx86-64-lsb on NUC100.0%none0.0%none0.0%none100.0%
6419 BSP/QEMU: master branchgenericx86-64-WIC on MMAX6497.7%none2.3%10481 0.0%none100.0%
6417 BSP/QEMU: master branchgenericx86-lsb on MMAX32bit98.1%none1.9%10514 0.0%none100.0%
6415 BSP/QEMU: master branchgenericx86-WIC on MMAX3297.7%none2.3%10481 0.0%none100.0%
6438 6439 BSP/QEMU: master branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
6421 BSP/QEMU: master branchqemu-x86100.0%none0.0%none0.0%none100.0%
6424 6425 BSP/QEMU: master branchqemuarm100.0%none0.0%none0.0%none100.0%
6426 6427 BSP/QEMU: master branchqemuarm64100.0%none0.0%none0.0%none100.0%
6428 6429 BSP/QEMU: master branchqemumips100.0%10025 0.0%none0.0%none100.0%
6430 6431 BSP/QEMU: master branchqemumips6498.4%none1.6%10456 0.0%none100.0%
6432 6433 BSP/QEMU: master branchqemuppc100.0%none0.0%none0.0%none100.0%
6422 BSP/QEMU: master branchqemux86-64100.0%none0.0%none0.0%none100.0%
Total: 27


Bug Status


Test Case Status Report
StatusCount
FAILED10
PASSED2459
Total: 2469



Bugs Found during QA Test

New and notable issues Found during QA Test & Community

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
10514test_parselogs on genericx86-lsb minnow board max 32 FAILED LOGIN (1) on '/dev/tty1' FOR 'UNKNOWN', Authentication failureMediumDavid Lopez BarribaFrancisco Pedraza2.3VERIFIEDINVALIDRegression (Found by QA test)2016-10-27
General Runtime
10519Can not run multiple QEMU machines under UNFSMedium+Robert YangYi Zhao2.3 M2VERIFIEDFIXEDRegression (Used to work)2016-10-28
Runtime
10504[Build Appliance] The default user builder can not loginMedium+Juro BystrickyYi Zhao2.3 M1VERIFIEDFIXEDRegression (Used to work)2016-10-27


High and M+ issues Found during QA Test (NOT NEW)

For this particular case were found on rc4 QA cycle WW43 - 2016-10-18 - Full_Test_Cycle_2.2_rc4

IDSummary (4 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
10456X can not launch on qemumips64Medium+Armin KusterYi Zhao2.2.2VERIFIEDFIXEDRegression (Used to work)2016-10-19
eSDK
10447eSDK: runqemu fails unless unless DEPLOY_DIR_IMAGE is setMedium+Paul EggletonFrancisco Pedraza2.3 M4RESOLVEDFIXED---2016-10-17
Meta-yocto
10477yocto-bsp: tool should output a conf file to be consume by the qemu runnerMediumLeonardo Sandoval Gonzalez 2.5RESOLVEDOBSOLETE---2016-10-21
Runtime
10434runqemu can not launch qemu image in Build ApplianceMedium+Juro BystrickyYi Zhao2.3 M1VERIFIEDFIXEDRegression (Used to work)2016-10-14


Other Bugs found during QA Test (NOT NEW)

IDSummary (4 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18
BSPs
10481[Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) WIC giving error: GPT: Use GNU Parted to correct GPT errors.MediumJair GonzalezJuan Ramos2.3 M1RESOLVEDFIXED---2016-10-21
10494[Test Case 1059] Parselogs is failing on genericx86-64-WIC on NUC giving error: Failed to load DMC firmwareMediumCalifornia Sullivan 2.2.2RESOLVEDFIXED---2016-10-24
Regression (Used to work)
10025Missing 'Connection Manager' icon for qemumips and qemumips64MediumJussi KukkonenYi Zhao2.4RESOLVEDWORKSFORMERegression (Used to work)2016-07-26

pTest open bugs

IDSummary (12 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
Runtime Testing
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng2.99ACCEPTEDRegression (Used to work)2015-12-17
9373ptest for perl hangMediumAníbal Limón 2.3 M1RESOLVEDDUPLICATE---2016-04-01
8146ptest - some test cases are failing for aclMediumAjay KumarRoss Burton2.2RESOLVEDDUPLICATE---2015-08-11
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 2.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09
8656ptest - perl tests fail for genericx86-64 on NUC - 2.0.rc3MediumAníbal LimónFrancisco Pedraza2.3 M1RESOLVEDFIXEDRegression (Used to work)2015-11-06
8831pTest - busybox tests fail for 2.1_M1.rc1Mediumdengke FutureRESOLVEDDUPLICATE---2015-12-17
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03
9055pTest - acl (NUC) failed tested on NUC 5425WYK for YP 2.1_M2.rc1MediumRobert YangFrancisco Pedraza2.4RESOLVEDFIXEDRegression (Used to work)2016-02-04
9372Ptest for acl hang because of lack of two user bin and daemonMediumChen Qioscar lopez arandas2.4RESOLVEDFIXED---2016-04-01

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestone or release which are Medium+ or High and are not targeted for the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of this week's Yocto Bug Trend charts can be accessed on the following page

Build Performance Test

Detailed description on how performance test are run can be found on Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.2 M2. Performance execution has 2 stages, one executed form 1.6 to 2.1 in RO and another one started on 2.1 from GDC, there are difference between HW used in both location hence the numbers are different.

Below chars are showing the latest performance results taken from current milestone

Performance Charts

NOTE: Due bug 10285 performance results for "bitbake core-image-sato -c rootfs" measurements were failing on 2.2_M3.rc1, hence the numbers shown in the charts for that particular milestone are the ones reported on logs just before of the failure.

Performance Charts

For Bitbake core-image-sato with rm_work

Ww44-2k16 - rmwork.png

Building the Virtual kernel

Ww44-2k16 - kernel.png

Building the rootfs

Ww44-2k16 - rootfs.png

Building core-image-sato

Ww44-2k16 - sato.png

pTest results for genericx86-64 on NUC

< Back
Ran on a NUC and compared with commit 4b94b498e21aeba945fe7e72a6b7c4bb0314fb83

FULL LOG

Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 354 26 93.20% 96.00%
attr 143 143 0 100.00% 100.00%
bash 76 72 4 94.70% 95.00%
bluez5 25 25 0 100.00% 100.00%
busybox 487 480 7 98.60% 98.00%
bzip2 6 6 0 100.00% 100.00%
dbus-test 15 4 11 26.70% 27.00%
diffstat 440 440 0 100.00% 100.00%
diffutils 16 16 0 100.00% 100.00%
e2fsprogs 1 0 1 0.00% 0.00%
gawk 284 283 1 99.60% 100.00%
gdbm 22 22 0 100.00% 100.00%
gdk-pixbuf 17 17 0 100.00% 100.00%
glib-2.0 280 273 7 97.50% 98.00%
kbd 7 7 0 100.00% 100.00%
libpcre 3 3 0 100.00% 100.00%
libxml2 85 82 3 96.50% 96.00%
lttng-tools 2467 2462 5 99.80% 99.00%
nettle 87 86 1 98.90% 99.00%
openssh 56 56 0 100.00% 100.00%
openssl 52 52 0 100.00% 100.00%
pango 12 12 0 100.00% 100.00%
perl 2263 2262 1 100.00% 100.00%
perl5 2263 2262 1 100.00% 100.00%
python 32056 31938 118 99.60% 100.00%
quilt 55 54 1 98.20% 100.00%
strace 264 263 1 99.60% 100.00%
tcl 148 148 0 100.00% 99.00%
util-linux 130 128 2 98.50% 100.00%
valgrind 538 2 536 0.40% 0.00%
zlib 2 2 0 100.00% 100.00%

Detailed Report

Distro Testing

Core Build System

Meta-Yocyo

Status

Bugs

Agenda

OE-Core

Status

Test Case Status Report

Bugs

**No results**

Agenda

Test Run Agenda

BSP

genericx86: core-image-sato-sdk

Status

Test Case Status Report

Bugs

Bug Status Report

Agenda

Test Run Agenda

genericx86_64: core-image-sato-sdk

Status

Test Case Status Report

Bugs

Bug Status Report

Agenda

Test Run Agenda