WW29 - 2017-07-18- Full Test Cycle - 2.3.1 rc2

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 
 Test type: Full Pass Test
 Branch: pyro
 Build name: yocto-2.3.1.rc2
 poky commit: 6bd890d9e011014cf323e61267f8b256949d44aa
 Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.3.1.rc2
 Distributions tested: 
     CentOS 7.1
     Debian 8
     Fedora 24
     Fedora 25
     Ubuntu 17.04
     Ubuntu 16.04
     OpenSuse 422
  
Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
7725 7726 eSDK: pyro branchCentos 7.1100.0%none0.0%none0.0%none100.0%
7722 CROPS: pyro branchCentos 7.1100.0%none0.0%none0.0%none100.0%
7682 7683 ADT: pyro branchFedora 25 i686100.0%none0.0%none0.0%none100.0%
7684 7685 ADT: pyro branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
7728 Kernel: pyro branchCentos 7100.0%none0.0%none0.0%none100.0%
7723 Eclipse-Plugin: pyro branchMars on Fedora 22 x86_64,85.7%none14.3%10832 0.0%none100.0%
7724 Eclipse-Plugin: pyro branchNeon on Fedora 22 x86_6485.7%none14.3%10832 0.0%none100.0%
7729 Meta-yocto: pyro branchUbuntu 14.04 x86_64100.0%10477 0.0%none0.0%none100.0%
7739 Build-Appliance: pyro branchuser-preference100.0%none0.0%none0.0%none100.0%
7740 7741 Toaster: pyro branchDebian 4.3.0-0.bpo.1-amd6498.6%none1.4%none0.0%none100.0%
7730 OE-Core: pyro branchCentOS 7.1 x86_64100.0%none0.0%none0.0%none100.0%
7731 OE-Core: pyro branchDebian 8 x86_64100.0%none0.0%none0.0%none100.0%
7733 OE-Core: pyro branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
7732 OE-Core: pyro branchFedora 25 x86_64100.0%none0.0%none0.0%none100.0%
7735 OE-Core: pyro branchOpenSuse 422 x86_64100.0%none0.0%none0.0%none100.0%
7736 OE-Core: pyro branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
7737 OE-Core: pyro branchUbuntu 17.04 x86-64100.0%none0.0%none0.0%none100.0%
7686 BitBake: pyro branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
7717 BSP/QEMU: pyro branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
7689 BSP/QEMU: pyro branchBeaglebone-Black100.0%none0.0%none0.0%none100.0%
7690 7718 BSP/QEMU: pyro branchEdgeRouter100.0%none0.0%none0.0%none100.0%
7710 BSP/QEMU: pyro branchgenericx86 - on MMAX32bit97.8%none2.2%10455 0.0%none100.0%
7711 BSP/QEMU: pyro branchgenericx86-64 on MMAX6497.8%none2.2%10455 0.0%none100.0%
7712 BSP/QEMU: pyro branchgenericx86-64 on NUC97.8%none2.2%10455 0.0%none100.0%
7720 BSP/QEMU: pyro branchgenericx86-64-lsb on MMAX64100.0%none0.0%none0.0%none100.0%
7719 BSP/QEMU: pyro branchgenericx86-64-lsb on NUC100.0%none0.0%none0.0%none100.0%
7713 BSP/QEMU: pyro branchgenericx86-64-WIC on MMAX64100.0%none0.0%none0.0%none100.0%
7714 BSP/QEMU: pyro branchgenericx86-64-WIC on NUC100.0%none0.0%none0.0%none100.0%
7688 BSP/QEMU: pyro branchGenericx86-64_NUC100.0%none0.0%none0.0%none100.0%
7721 BSP/QEMU: pyro branchgenericx86-lsb on MMAX32bit100.0%none0.0%none0.0%none100.0%
7715 BSP/QEMU: pyro branchgenericx86-WIC on MMAX32100.0%none0.0%none0.0%none100.0%
7687 BSP/QEMU: pyro branchGenericx86_MMAX32100.0%none0.0%none0.0%none100.0%
7691 7716 BSP/QEMU: pyro branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
7692 7699 BSP/QEMU: pyro branchqemu-x8698.0%none2.0%9656 0.0%none100.0%
7693 7700 BSP/QEMU: pyro branchqemuarm100.0%11795 0.0%none0.0%none100.0%
7694 7701 BSP/QEMU: pyro branchqemuarm64100.0%none0.0%none0.0%none100.0%
7695 7702 BSP/QEMU: pyro branchqemumips100.0%10025 0.0%none0.0%none100.0%
7696 7703 BSP/QEMU: pyro branchqemumips64100.0%10025 0.0%none0.0%none100.0%
7697 7704 BSP/QEMU: pyro branchqemuppc100.0%none0.0%none0.0%none100.0%
7698 BSP/QEMU: pyro branchqemux86-64100.0%none0.0%none0.0%none100.0%
7705 BSP/QEMU: pyro branchqemux86-64,100.0%none0.0%none0.0%none100.0%
7706 BSP/QEMU: pyro branchqemux86-64-WIC,100.0%none0.0%none0.0%none100.0%
7708 BSP/QEMU: pyro branchqemux86-lsb100.0%none0.0%none0.0%none100.0%
7707 BSP/QEMU: pyro branchqemux86-WIC100.0%none0.0%none0.0%none100.0%
7709 BSP/QEMU: pyro branchqemux86_64-lsb100.0%none0.0%none0.0%none100.0%
7727 Runtime: pyro branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
Total: 46


Bug Status


Test Case Status Report
StatusCount
FAILED7
PASSED2948
RUNNING14
Total: 2969

Bugs Found during QA Test

New issues Found during QA Test & Community

General Runtime
11795[Yocto 2.3.1] Can not stop tcf-agent daemonUndecidedArmin KusterYi Zhao---VERIFIEDFIXEDRegression (Used to work)2017-07-17


High and M+ issuesFound during QA Test (NOT NEW)

IDSummary (2 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
eSDK
10832eSDK install script failed with "tasks executed unexpectedly" messagesHighEd Bartoshoscar lopez arandas2.3 M1VERIFIEDFIXED---2016-12-20
Meta-yocto
10477yocto-bsp: tool should output a conf file to be consume by the qemu runnerMediumLeonardo Sandoval Gonzalez 2.5RESOLVEDOBSOLETE---2016-10-21

Other Bugs found during QA Test (NOT NEW)

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18
Regression (Found by QA test)
9656NFS boot failed due No working init found kernel panic on qemuMediumBruce Ashfield 2.99ACCEPTEDRegression (Found by QA test)2016-05-18
Regression (Used to work)
10025Missing 'Connection Manager' icon for qemumips and qemumips64MediumJussi KukkonenYi Zhao2.4RESOLVEDWORKSFORMERegression (Used to work)2016-07-26

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestoneor release which are Medium+ or High and are not targetedfor the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of thisweek's Yocto Bug Trend charts can be accessed on thefollowing [1]

Build Performance Test

Detailed description on how performance test is run can be found on the Performance Test wiki. The below charts are displaying the build performance trend from 2.1 M4 to 2.3.1 rc2. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and executed from 2.1 to 2.3.1 rc2 in GDC. There are differences between HW used in both locations, hence the numbers are different.

Below chars are showing the latest performance results taken from the current milestone.

Performance Charts

Ww29-2k17 - table.png

For Bitbake core-image-sato with rm_work

Ww29-2k17 - rmwork.png

Building the Virtual kernel

Ww29-2k17 - kernel.png

Building the rootfs

Ww29-2k17 - rootfs.png

Building core-image-sato

Ww29-2k17 - sato.png

eSDK Deploy time

Ww29-2k17 - esdk.png

pTest for genericx86-64 on NUC

open bugs

IDSummary (25 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
11674core-image-sato-sdk-ptest build error for cpio imageMediumRoss Burton 2.7IN PROGRESS REVIEW---2017-06-15
6707Add ptest for curlMediumArmin KusterApoorvFutureIN PROGRESS REVIEW---2014-09-11
4436Add ptest for glibcMediumArmin Kuster FutureNEW---2013-04-29
4437Add ptest for gccMediumArmin Kuster FutureNEW---2013-04-29
5605Add ptest for libavLowUnassigned FutureNEW---2013-11-27
10266Add ptest for gnutlsMediumArmin Kuster 2.99NEW---2016-09-13
11666attr-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-14
11667flex-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-14
11668libz-ptest contains build host referencesMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-15
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
10841pTest : Various test cases are failing on "dbus-test" moduleMediumRoss Burton 2.5RESOLVEDDUPLICATE---2016-12-21
5698bash: ptest cases failedMediumYi Zhao FutureRESOLVEDDUPLICATE---2013-12-31
10953util-linux installs libtool wrapper scripts instead of binaries for ptestMediumJuro Bystricky 2.5 M1RESOLVEDFIXED---2017-01-23
11024lttng-tools ships libtool wrappers as ptest binariesMediumdengke 2.7RESOLVEDFIXED---2017-02-07
11547Add regression check for ptestMedium+Robert Yang 2.4 M3RESOLVEDFIXED---2017-05-19
11596bzip2-ptest: improve reproducible buildMediumCinly Ooi 2.5 M1RESOLVEDFIXED---2017-05-31
11602acl-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-01
Program Management
8064ptest fails tracker bugMediumRoss Burton 2.5RESOLVEDOBSOLETE---2015-07-28
Recipe Reporting System (RRS)
8650RRS: track ptest-enabled and potentially ptest-enabled statusMediumPaul Eggletonyeoh ee peng2.99NEW---2015-11-05
Runtime Testing
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng2.99ACCEPTEDRegression (Used to work)2015-12-17
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03
6737Extend automated ptest testing to work with opkgUndecidedSonia Delgadoyeoh ee pengFutureRESOLVEDOBSOLETE---2014-09-18
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 2.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09

results

Results

< Back
Ran on a NUC and compared with commit Ptest 6964efddd31c479386d1643c1025bc102710392f

FULL LOG


Failures
 % increased
 % decreased


Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.00% 100.00%
attr 143 143 0 100.00% 100.00%
bash 76 72 4 94.70% 94.70%
bluez5 25 25 0 100.00% 100.00%
busybox 486 478 8 98.40% 98.60%
bzip2 6 6 0 100.00% 100.00%
dbus-test 15 4 11 26.70% 26.70%
diffstat 440 440 0 100.00% 100.00%
diffutils 17 17 0 100.00% 100.00%
e2fsprogs 1 1 0 100.00% 100.00%
flex 114 114 0 100.00% 100.00%
gawk 298 297 1 99.70% 99.70%
gdbm 22 22 0 100.00% 100.00%
gdk-pixbuf 20 20 0 100.00% 95.0%
glib-2.0 284 276 8 97.50% 97.50%
kbd 7 7 0 100.00% 100.00%
libpcre 3 3 0 100.00% 100.00%
libxml2 85 82 3 96.50% 97.70%
lttng-tools 2057 1988 69 96.60% 96.60%
nettle 88 87 1 98.90% 98.90%
openssh 34 33 1 97.10% 100.00%
openssl 52 52 0 100.00% 100.00%
pango 12 12 0 100.00% 100.00%
perl 2286 2285 1 100.00% 100.00%
perl5 2286 2285 1 100.00% 100.00%
python 32227 32108 119 99.60% 99.60%
quilt 57 57 0 100.00 100.00%
strace 335 334 1 99.70% 99.70%
tcl 148 146 2 98.60% 98.60%
util-linux 108 106 2 98.10% 98.10%
valgrind 553 296 257 53.50% 52.80%
zlib 2 2 0 100.00% 100.00%

Detailed Report

ADT

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

BitBake

Status

Bugs

**No results**

Agenda

BSPs

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

CROPS

Status

Bugs

**No results**

Agenda

Eclipse Plugin

Status

Bugs

Agenda

Status

Bugs

Agenda

eSDK

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

General Runtime

Status

Bugs

**No results**

Agenda

Kernel

Status

Bugs

**No results**

Agenda

Meta-yocto

Status

Bugs

Agenda

OE-Core

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda