WW38 - 2017-09-20- Full Test Cycle - 2.3.2 rc1

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 
 Test type: Full Pass Test 
 Branch: pyro
 Build name: yocto-2.3.2 rc1
 poky commit: ce26a57e04ad65c02087629701d96448a44e73d5
 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3.2.rc1
 Distributions tested: 
       CentOS 7.1
       Debian 8
       Fedora 24
       Fedora 25
       OpenSuse 422
       Ubuntu 16.04

  
Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
8192 8193 eSDK: pyro branchCentos 7.1100.0%none0.0%none0.0%none100.0%
8189 CROPS: pyro branchCentos 7.1100.0%none0.0%none0.0%none100.0%
8149 8150 ADT: pyro branchFedora 25 i686100.0%none0.0%none0.0%none100.0%
8151 8152 ADT: pyro branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
8195 Kernel: pyro branchCentos 7100.0%none0.0%none0.0%none100.0%
8190 Eclipse-Plugin: pyro branchMars on Fedora 22 x86_64,100.0%none0.0%none0.0%none100.0%
8191 Eclipse-Plugin: pyro branchNeon on Fedora 22 x86_64100.0%none0.0%none0.0%none100.0%
8196 Meta-yocto: pyro branchUbuntu 14.04 x86_64100.0%10477 0.0%none0.0%none100.0%
8206 Build-Appliance: pyro branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
8207 8208 Toaster: pyro branchDebian 4.3.0-0.bpo.1-amd64100.0%none0.0%none0.0%none100.0%
8197 OE-Core: pyro branchCentOS 7.1 x86_64100.0%none0.0%none0.0%none100.0%
8198 OE-Core: pyro branchDebian 8 x86_64100.0%none0.0%none0.0%none100.0%
8200 OE-Core: pyro branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
8199 OE-Core: pyro branchFedora 25 x86_64100.0%none0.0%none0.0%none100.0%
8202 OE-Core: pyro branchOpenSuse421 x86_64100.0%none0.0%none0.0%none100.0%
8203 OE-Core: pyro branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
8204 OE-Core: pyro branchUbuntu 17.04 x86-64100.0%12114 0.0%none0.0%none100.0%
8153 BitBake: pyro branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
8184 BSP/QEMU: pyro branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
8156 BSP/QEMU: pyro branchBeaglebone-Black100.0%none0.0%none0.0%none100.0%
8157 8185 BSP/QEMU: pyro branchEdgeRouter100.0%none0.0%none0.0%none100.0%
8177 BSP/QEMU: pyro branchgenericx86 - on MMAX32bit97.8%none2.2%10455 11879 0.0%none100.0%
8178 BSP/QEMU: pyro branchgenericx86-64 on MMAX6497.8%none2.2%10455 11879 0.0%none100.0%
8179 BSP/QEMU: pyro branchgenericx86-64 on NUC97.8%none2.2%10455 11879 0.0%none100.0%
8187 BSP/QEMU: pyro branchgenericx86-64-lsb on MMAX64100.0%none0.0%none0.0%none100.0%
8186 BSP/QEMU: pyro branchgenericx86-64-lsb on NUC100.0%none0.0%none0.0%none100.0%
8180 BSP/QEMU: pyro branchgenericx86-64-WIC on MMAX6497.8%none2.2%10455 11879 0.0%none100.0%
8181 BSP/QEMU: pyro branchgenericx86-64-WIC on NUC97.8%none2.2%10455 11879 0.0%none100.0%
8155 BSP/QEMU: pyro branchGenericx86-64_NUC100.0%none0.0%none0.0%none100.0%
8188 BSP/QEMU: pyro branchgenericx86-lsb on MMAX32bit100.0%none0.0%none0.0%none100.0%
8182 BSP/QEMU: pyro branchgenericx86-WIC on MMAX3297.8%none2.2%11879 0.0%none100.0%
8154 BSP/QEMU: pyro branchGenericx86_MMAX32100.0%none0.0%none0.0%none100.0%
8158 8183 BSP/QEMU: pyro branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
8159 8166 BSP/QEMU: pyro branchqemu-x86100.0%none0.0%none0.0%none100.0%
8160 8167 BSP/QEMU: pyro branchqemuarm100.0%none0.0%none0.0%none100.0%
8161 8168 BSP/QEMU: pyro branchqemuarm64100.0%none0.0%none0.0%none100.0%
8162 8169 BSP/QEMU: pyro branchqemumips100.0%none0.0%none0.0%none100.0%
8163 8170 BSP/QEMU: pyro branchqemumips64100.0%none0.0%none0.0%none100.0%
8164 8171 BSP/QEMU: pyro branchqemuppc100.0%none0.0%none0.0%none100.0%
8165 BSP/QEMU: pyro branchqemux86-64100.0%none0.0%none0.0%none100.0%
8172 BSP/QEMU: pyro branchqemux86-64,100.0%none0.0%none0.0%none100.0%
8173 BSP/QEMU: pyro branchqemux86-64-WIC,100.0%none0.0%none0.0%none100.0%
8175 BSP/QEMU: pyro branchqemux86-lsb100.0%none0.0%none0.0%none100.0%
8174 BSP/QEMU: pyro branchqemux86-WIC100.0%none0.0%none0.0%none100.0%
8176 BSP/QEMU: pyro branchqemux86_64-lsb100.0%none0.0%none0.0%none100.0%
8194 Runtime: pyro branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
Total: 46


Bug Status


Test Case Status Report
StatusCount
FAILED6
PASSED2982
RUNNING10
Total: 2998

Bugs Found during QA Test

New issues Found during QA Test & Community

"None"


High and M+ issuesFound during QA Test (NOT NEW)

BSPs
11879Testimage: multiple "uvesafb" errors present in log in minnowmax on genericx86/genericx86-64Medium+Alejandro Hernandez 2.4 M4RESOLVEDFIXED---2017-07-31

Other Bugs found during QA Test (NOT NEW)

IDSummary (2 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18
Meta-yocto
10477yocto-bsp: tool should output a conf file to be consume by the qemu runnerMediumLeonardo Sandoval Gonzalez 2.5RESOLVEDOBSOLETE---2016-10-21

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestoneor release which are Medium+ or High and are not targetedfor the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of thisweek's Yocto Bug Trend charts can be accessed on thefollowing [1]

Build Performance Test

Detailed description on how performance test is run canbe found on the Performance Test wiki. The below charts are displaying the build performance trend from 2.1 M4 to 2.3.2 rc1. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and executed from 2.1 to 2.4 in GDC. There are differences between HW used in both locations, hence the numbers are different

Below chars are showing the latestperformance results taken from the current milestone.

Performance Charts

Table

Ww38-2k17 - table.png

For Bitbake core-image-sato withrm_work

Ww38-2k17 - rmwork.png

Building the Virtual kernel

Ww38-2k17 - kernel.png

Building the rootfs

Ww38-2k17 - rootfs.png

Building core-image-sato

Ww38-2k17 - sato.png

eSDK Deploy time

Ww38-2k17 - esdk.png

pTest for genericx86-64 on NUC

open bugs

IDSummary (26 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
11674core-image-sato-sdk-ptest build error for cpio imageMediumRoss Burton 2.7IN PROGRESS REVIEW---2017-06-15
6707Add ptest for curlMediumArmin KusterApoorvFutureIN PROGRESS REVIEW---2014-09-11
4436Add ptest for glibcMediumArmin Kuster FutureNEW---2013-04-29
4437Add ptest for gccMediumArmin Kuster FutureNEW---2013-04-29
5605Add ptest for libavLowUnassigned FutureNEW---2013-11-27
10266Add ptest for gnutlsMediumArmin Kuster 2.99NEW---2016-09-13
11666attr-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-14
11667flex-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-14
11668libz-ptest contains build host referencesMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-15
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
10841pTest : Various test cases are failing on "dbus-test" moduleMediumRoss Burton 2.5RESOLVEDDUPLICATE---2016-12-21
5698bash: ptest cases failedMediumYi Zhao FutureRESOLVEDDUPLICATE---2013-12-31
10953util-linux installs libtool wrapper scripts instead of binaries for ptestMediumJuro Bystricky 2.5 M1RESOLVEDFIXED---2017-01-23
11024lttng-tools ships libtool wrappers as ptest binariesMediumdengke 2.7RESOLVEDFIXED---2017-02-07
11547Add regression check for ptestMedium+Robert Yang 2.4 M3RESOLVEDFIXED---2017-05-19
11596bzip2-ptest: improve reproducible buildMediumCinly Ooi 2.5 M1RESOLVEDFIXED---2017-05-31
11602acl-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-01
Program Management
8064ptest fails tracker bugMediumRoss Burton 2.5RESOLVEDOBSOLETE---2015-07-28
Recipe Reporting System (RRS)
8650RRS: track ptest-enabled and potentially ptest-enabled statusMediumPaul Eggletonyeoh ee peng2.99NEW---2015-11-05
Runtime Testing
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng2.99ACCEPTEDRegression (Used to work)2015-12-17
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03
6737Extend automated ptest testing to work with opkgUndecidedSonia Delgadoyeoh ee pengFutureRESOLVEDOBSOLETE---2014-09-18
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 2.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09
Test Plans/Suite
12094ptest: modify the result chart so that the current release commit is compared with the previous commit releaseHighJose Perez C 2.5 M1RESOLVEDFIXED---2017-09-15

results

< Back
Ran on a NUC and compared with commit Ptest 6bd890d9e011014cf323e61267f8b256949d44aa

FULL LOG


Failures
 % increased
 % decreased


Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.0% 100.0%
attr 143 143 0 100.0% 100.0%
bash 76 72 4 94.7% 94.7%
bluez5 25 25 0 100.0% 100.0%
busybox 518 509 9 98.3% 98.4%
bzip2 6 6 0 100.00% 100.0%
dbus-test 15 4 11 26.7% 26.7%
diffstat 440 440 0 100.0% 100.0%
diffutils 17 17 0 100.0% 100.0%
e2fsprogs 1 1 0 100.0% 100.0%
flex 114 114 0 100.0% 100.0%
gawk 298 297 1 99.7% 99.7%
gdbm 22 22 0 100.0% 100.0%
gdk-pixbuf 20 20 0 100.0% 100.0%
glib-2.0 252 251 1 99.6% 97.5%
kbd 7 7 0 100.0% 100.0%
libpcre 3 3 0 100.0% 100.0%
libxml2 39 36 3 92.3% 96.5%
lttng-tools 2057 1988 69 96.6% 96.6%
nettle 87 86 1 98.9% 98.9%
openssh 34 33 1 97.1% 97.1%
openssl 52 52 0 100.00% 100.00%
pango 12 12 0 100.0% 100.0%
perl 2286 2285 1 100.0% 100.0%
perl5 2286 2285 1 100.0% 100.0%
python 32234 32108 126 99.6% 99.6%
quilt 57 57 0 100.0% 100.0%
strace 335 334 1 99.7% 99.7%
tcl 148 146 2 98.6% 98.6%
util-linux 108 106 2 98.1% 98.1%
valgrind 553 296 257 53.5% 53.5%
zlib 2 2 0 100.0% 100.0%

Detailed Report

ADT

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

BitBake

Status

Bugs

**No results**

Agenda

BSPs

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

CROPS

Status

Bugs

**No results**

Agenda

Eclipse Plugin

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

eSDK

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

General Runtime

Status

Bugs

**No results**

Agenda

Kernel

Status

Bugs

**No results**

Agenda

Meta-yocto

Status

Bugs

Agenda

OE-Core

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda