WW52 - 2016-12-19 - Full Test Cycle 2.3 M1 rc1

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 Test type:  Full Test Cycle
 Branch: master
 Build name: 2.3 M1 rc1
 poky commit: 573c646d4cc62dcd0c230381df4940bdf314d495
 Autobuilder images repository:  http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3_M1.rc1
 Distributions tested: 
 -Ubuntu 16.04
 -Ubuntu 14 04 
 -OpenSuse 421
 -Opensuse 13.2  
 -Debian 8
 -Centos 7
 -Fedora 23
 -Fedora 24 

Test Run Summary Report

Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
6626 6627 eSDK: master branchCentos 7.10%none0%none0%none0%
6658 6659 ADT: master branchFedora 25 i686100.0%none0.0%none0.0%none100.0%
6656 6657 ADT: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
6624 Eclipse-Plugin: master branchMars on Fedora 22 x86_6485.7%none14.3%10832 0.0%none100.0%
6625 Eclipse-Plugin: master branchNeon on Fedora 22 x86_6485.7%none14.3%10832 0.0%none100.0%
6654 Meta-yocto: master branchUbuntu 14.04 x86_6495.8%10477 4.2%10519 0.0%none100.0%
6655 Build-Appliance: master branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
6622 6628 Toaster: master branchDebian 4.3.0-0.bpo.1-amd6430.3%10947 4.0%10846 10281 9252 65.7%10846 9252 100.0%
6614 OE-Core: master branchCentOS 7.1 x86_64100.0%none0.0%none0.0%none100.0%
6615 OE-Core: master branchDebian 8 x86_64100.0%none0.0%none0.0%none100.0%
6616 OE-Core: master branchFedora 23 x86_64100.0%none0.0%none0.0%none100.0%
6617 OE-Core: master branchFedora 24 x86_64100.0%10826 0.0%none0.0%none100.0%
6618 OE-Core: master branchOpenSuse 13.2 x86_64100.0%10826 0.0%none0.0%none100.0%
6619 OE-Core: master branchOpenSuse421 x86_64100.0%10826 0.0%none0.0%none100.0%
6620 OE-Core: master branchubuntu 14.04 GDC100.0%10826 0.0%none0.0%none100.0%
6621 OE-Core: master branchUbuntu 16.04 x86_64100.0%10826 0.0%none0.0%none100.0%
6632 BitBake: master branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
6648 6649 BSP/QEMU: master branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
6650 6651 BSP/QEMU: master branchEdgeRouter100.0%none0.0%none0.0%none100.0%
6636 6665 BSP/QEMU: master branchgenericx86 - on MMAX32bit99.0%none1.0%10455 0.0%none100.0%
6630 BSP/QEMU: master branchgenericx86-64 on MMAX6498.2%none1.8%10455 0.0%none100.0%
6629 6668 BSP/QEMU: master branchgenericx86-64 on NUC99.0%none1.0%10455 0.0%none100.0%
6635 BSP/QEMU: master branchgenericx86-64-lsb on MMAX64100.0%none0.0%none0.0%none100.0%
6634 BSP/QEMU: master branchgenericx86-64-lsb on NUC100.0%none0.0%none0.0%none100.0%
6637 BSP/QEMU: master branchgenericx86-64-WIC on MMAX64100.0%none0.0%none0.0%none100.0%
6631 BSP/QEMU: master branchgenericx86-64-WIC on NUC100.0%none0.0%none0.0%none100.0%
6666 BSP/QEMU: master branchgenericx86-lsb on MMAX32bit100.0%none0.0%none0.0%none100.0%
6667 BSP/QEMU: master branchgenericx86-WIC on MMAX32100.0%9271 0.0%none0.0%none100.0%
6652 6653 BSP/QEMU: master branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
6662 6669 BSP/QEMU: master branchqemu-x86100.0%none0.0%none0.0%none100.0%
6638 6639 BSP/QEMU: master branchqemuarm100.0%none0.0%none0.0%none100.0%
6640 6641 BSP/QEMU: master branchqemuarm64100.0%none0.0%none0.0%none100.0%
6642 6643 BSP/QEMU: master branchqemumips100.0%none0.0%none0.0%none100.0%
6644 6645 BSP/QEMU: master branchqemumips6498.4%none1.6%10456 0.0%none100.0%
6646 6647 BSP/QEMU: master branchqemuppc100.0%none0.0%none0.0%none100.0%
6661 6670 BSP/QEMU: master branchqemux86-64100.0%none0.0%none0.0%none100.0%
6663 BSP/QEMU: master branchqemux86-lsb96.5%none3.5%10844 10845 0.0%none100.0%
6664 BSP/QEMU: master branchqemux86_64-lsb96.5%none3.5%10844 10845 0.0%none100.0%
6660 Runtime: master branchMultiple Environments100.0%none0.0%none0.0%none100.0%
Total: 39


Bug Status


Test Case Status Report
StatusCount
BLOCKED65
FAILED15
PASSED3128
Total: 3208



Bugs Found during QA Test

NOTE: There is workaround for [10826] and [10873], those are happening due special settings for QA purposes on Autobuilders, doing the below steps the execution of oe-selftest are running correctly

- uninstall "unittest-xml-reporting" form the worker on AB
- set max_user_watches=65535 (64K) [1]

New issues Found during QA Test & Community

IDSummary (10 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BitBake
10866tinfoil2 breaks the prserv functionalityHighRichard Purdie 2.3 M1RESOLVEDFIXED---2016-12-30
10873bitbake leak fd's and oe-selftest fails with max_user_watches errorHighAníbal LimónJose Perez C2.3 M2VERIFIEDFIXED---2017-01-03
10884build-perf:time for build "rootfs" is increased in around 20% on 2.3 M1HighLeonardo Sandoval Gonzalez 2.3 M3VERIFIEDFIXED---2017-01-05
BSPs
10844parselogs: qemux86 and x86-64 failing with message "qemux86-64 getty[728]"MediumChoong Yin Thongoscar lopez arandas2.4VERIFIEDFIXED---2016-12-22
10845test-image: compile and load a kernel module under the qemu-lsb image is failingMediumBruce Ashfield 2.3 M3RESOLVEDNOTABUG---2016-12-22
eSDK
10832eSDK install script failed with "tasks executed unexpectedly" messagesHighEd Bartoshoscar lopez arandas2.3 M1VERIFIEDFIXED---2016-12-20
OE-Core
10826selftest: tests that use runqemu fail on GDC AutobuildersHighAníbal LimónBenjamin Esquivel2.3 M2VERIFIEDFIXED---2016-12-19
10833runqemu / wic: IP of tap is not being set when launching .wic image on qemux86*Medium+Ed Bartosh 2.3 M4RESOLVEDFIXED---2016-12-20
System Startup
10843Wrong argument for rpc.mountd when qemu bootingMediumSaul WoldYi Zhao2.3VERIFIEDFIXEDRegression (Used to work)2016-12-22
Toaster
10846Toaster: [Test Case 1095] Filter doesn't show a dialogueMediumDavid ReynaJose Perez C2.4VERIFIEDDUPLICATE---2016-12-22


High and M+ issues Found during QA Test (NOT NEW)

IDSummary (4 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
10456X can not launch on qemumips64Medium+Armin KusterYi Zhao2.2.2VERIFIEDFIXEDRegression (Used to work)2016-10-19
General Runtime
10519Can not run multiple QEMU machines under UNFSMedium+Robert YangYi Zhao2.3 M2VERIFIEDFIXEDRegression (Used to work)2016-10-28
Meta-yocto
10477yocto-bsp: tool should output a conf file to be consume by the qemu runnerMediumLeonardo Sandoval Gonzalez 2.5RESOLVEDOBSOLETE---2016-10-21
Toaster
9252Build images doesn't work in ToasterMedium+brian averyDavid Lopez Barriba2.3 M2VERIFIEDFIXEDRegression (Used to work)2016-03-14


Other Bugs found during QA Test (NOT NEW)

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18
Toaster
10281Configuration variables: I can set a value for standard shell environment variable http_proxyLowToaster default assigneeyeoh ee pengFutureNEW---2016-09-14
Regression (Used to work)
10025Missing 'Connection Manager' icon for qemumips and qemumips64MediumJussi KukkonenYi Zhao2.4RESOLVEDWORKSFORMERegression (Used to work)2016-07-26

pTest open bugs

IDSummary (12 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
10841pTest : Various test cases are failing on "dbus-test" moduleMediumRoss Burton 2.5RESOLVEDDUPLICATE---2016-12-21
10842testrunner test timeout can lead to garbage outputMediumAníbal Limón 2.3 M4VERIFIEDFIXED---2016-12-21
10840pTest- openssl: test cases "test_bn" and "test_cms" are failingMedium+Alexander Kanavin 2.4 M1VERIFIEDFIXED---2016-12-21
Runtime Testing
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng2.99ACCEPTEDRegression (Used to work)2015-12-17
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 2.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09
8831pTest - busybox tests fail for 2.1_M1.rc1Mediumdengke FutureRESOLVEDDUPLICATE---2015-12-17
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03
9055pTest - acl (NUC) failed tested on NUC 5425WYK for YP 2.1_M2.rc1MediumRobert YangFrancisco Pedraza2.4RESOLVEDFIXEDRegression (Used to work)2016-02-04
9372Ptest for acl hang because of lack of two user bin and daemonMediumChen Qioscar lopez arandas2.4RESOLVEDFIXED---2016-04-01

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestone or release which are Medium+ or High and are not targeted for the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of this week's Yocto Bug Trend charts can be accessed on the following page

Build Performance Test

Detailed description on how performance test are run can be found on Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.3 M1. Performance execution has 2 stages, one executed form 1.6 to 2.1 in RO and another one started on 2.1 from GDC, there are difference between HW used in both location hence the numbers are different.

Below chars are showing the latest performance results taken from current milestone

Performance Charts

NOTE: Due bug 10285 performance results for "bitbake core-image-sato -c rootfs" measurements were failing on 2.2_M3.rc1, hence the numbers shown in the charts for that particular milestone are the ones reported on logs just before of the failure.

Performance Charts

For Bitbake core-image-sato with rm_work

Ww52-2k16 - rmwork.png

Building the Virtual kernel

Ww52-2k16 - kernel.png

Building the rootfs

Ww52-2k16 - rootfs.png

Building core-image-sato

Ww52-2k16 - sato.png

pTest results for genericx86-64 on NUC

< Back
Ran on a NUC and compared with commit 573c646d4cc62dcd0c230381df4940bdf314d495

FULL LOG

Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 354 26 93.20% 93.20%
attr 143 143 0 100.00% 100.00%
bash 76 72 4 94.70% 94.70%
bluez5 25 25 0 100.00% 100.00%
busybox 487 480 7 98.60% 98.60%
bzip2 6 6 0 100.00% 100.00%
dbus-test 15 4 11 26.70% 26.70%
diffstat 440 440 0 100.00% 100.00%
diffutils 17 17 0 100.00% 100.00%
e2fsprogs 1 0 1 0.00% 0.00%
gawk 298 297 1 99.70% 99.60%
gdbm 22 22 0 100.00% 100.00%
gdk-pixbuf 17 17 0 100.00% 100.00%
glib-2.0 280 274 6 97.90% 97.50%
kbd 7 7 0 100.00% 100.00%
libpcre 3 3 0 100.00% 100.00%
libxml2 85 82 3 96.50% 96.50%
lttng-tools 2467 2462 5 99.80% 99.80%
nettle 88 87 1 98.90% 98.90%
openssh 56 56 0 100.00% 100.00%
openssl 52 50 2 96.20% 100.00%
pango 12 12 0 100.00% 100.00%
perl 2286 2285 1 100.00% 100.00%
perl5 2286 2285 1 100.00% 100.00%
python 32056 31938 118 99.60% 99.60%
quilt 55 54 1 98.20% 98.20%
strace 278 277 1 99.60% 99.60%
tcl 148 148 0 100.00% 100.00%
util-linux 131 129 2 98.50% 98.50%
valgrind 0 0 0 0 0.40%
zlib 2 2 0 100.00% 100.00%

Detailed Report

Distro Testing

Core Build System

Bitbake

Status

Bugs

**No results**

Agenda

Meta-Yocyo

Status

Bugs

Agenda

OE-Core

Status

Test Case Status Report

Bugs

Bug Status Report

Agenda

Test Run Agenda

eSDK

Status

**No results**

Bugs

**No results**

Agenda

**No results**

BSP

genericx86: core-image-sato-sdk

Status

Test Case Status Report

Bugs

Bug Status Report

Agenda

Test Run Agenda