WW25 - 2017-06-22 - Full Test Cycle 2.4 M1 rc1

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 
 Test type: Full Test Cycle
 Branch: master
 Build name: yocto-2.4_M1.rc1
 poky commit: 6964efddd31c479386d1643c1025bc102710392f 
 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.4_M1.rc1
 Distributions tested: 
     CentOS 7.1
     Debian 8
     Fedora 24
     Fedora 25
     OpenSuse 13.2
     OpenSuse 421
     Ubuntu 16.04
     Ubuntu 16.10

  
Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
7534 7535 eSDK: master branchCentos 7.1100.0%11401 0.0%none0.0%none100.0%
7531 CROPS: master branchCentos 7.1100.0%none0.0%none0.0%none100.0%
7491 7492 ADT: master branchFedora 25 i68694.2%none5.8%11687 0.0%none100.0%
7493 7494 ADT: master branchUbuntu 16.04 x86_6494.2%none5.8%11687 0.0%none100.0%
7537 Kernel: master branchCentos 7100.0%none0.0%none0.0%none100.0%
7532 Eclipse-Plugin: master branchMars on Fedora 22 x86_64,28.6%none0.0%none71.4%11687 100.0%
7533 Eclipse-Plugin: master branchNeon on Fedora 22 x86_6428.6%none0.0%none71.4%11687 100.0%
7538 Meta-yocto: master branchUbuntu 14.04 x86_6495.8%10477 4.2%11687 0.0%none100.0%
7547 Build-Appliance: master branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
7550 Toaster: master branchDebian 4.3.0-0.bpo.1-amd64100.0%none0.0%none0.0%none100.0%
7539 OE-Core: master branchCentOS 7.1 x86_64100.0%none0.0%none0.0%none100.0%
7540 OE-Core: master branchDebian 8 x86_64100.0%none0.0%none0.0%none100.0%
7542 OE-Core: master branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
7541 OE-Core: master branchFedora 25 x86_64100.0%none0.0%none0.0%none100.0%
7543 OE-Core: master branchOpenSuse 13.2 x86_64100.0%none0.0%none0.0%none100.0%
7544 OE-Core: master branchOpenSuse421 x86_64100.0%none0.0%none0.0%none100.0%
7545 OE-Core: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
7546 OE-Core: master branchUbuntu 16.10 x86-64100.0%none0.0%none0.0%none100.0%
7495 BitBake: master branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
7526 BSP/QEMU: master branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
7498 BSP/QEMU: master branchBeaglebone-Black100.0%none0.0%none0.0%none100.0%
7499 7527 BSP/QEMU: master branchEdgeRouter100.0%none0.0%none0.0%none100.0%
7519 BSP/QEMU: master branchgenericx86 - on MMAX32bit97.8%none2.2%10455 0.0%none100.0%
7520 BSP/QEMU: master branchgenericx86-64 on MMAX6497.8%none2.2%10455 0.0%none100.0%
7551 BSP/QEMU: master branchgenericx86-64 on MMAX64 - GDCHW0.0%none100.0%11681 0.0%none100.0%
7521 BSP/QEMU: master branchgenericx86-64 on NUC100.0%none0.0%none0.0%none100.0%
7529 BSP/QEMU: master branchgenericx86-64-lsb on MMAX64100.0%none0.0%none0.0%none100.0%
7528 BSP/QEMU: master branchgenericx86-64-lsb on NUC100.0%none0.0%none0.0%none100.0%
7522 BSP/QEMU: master branchgenericx86-64-WIC on MMAX64100.0%none0.0%none0.0%none100.0%
7523 BSP/QEMU: master branchgenericx86-64-WIC on NUC100.0%none0.0%none0.0%none100.0%
7497 BSP/QEMU: master branchGenericx86-64_NUC100.0%none0.0%none0.0%none100.0%
7530 BSP/QEMU: master branchgenericx86-lsb on MMAX32bit100.0%none0.0%none0.0%none100.0%
7524 BSP/QEMU: master branchgenericx86-WIC on MMAX32100.0%none0.0%none0.0%none100.0%
7496 BSP/QEMU: master branchGenericx86_MMAX32100.0%none0.0%none0.0%none100.0%
7500 7525 BSP/QEMU: master branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
7501 7508 BSP/QEMU: master branchqemu-x8695.9%none4.1%11687 0.0%none100.0%
7502 7509 BSP/QEMU: master branchqemuarm98.1%none0.0%none1.9%11687 100.0%
7503 7510 BSP/QEMU: master branchqemuarm6498.1%none0.0%none1.9%11687 100.0%
7504 7511 BSP/QEMU: master branchqemumips98.1%none0.0%none1.9%11687 100.0%
7505 7512 BSP/QEMU: master branchqemumips6498.1%none0.0%none1.9%11687 100.0%
7506 7513 BSP/QEMU: master branchqemuppc98.1%none0.0%none1.9%11687 100.0%
7507 BSP/QEMU: master branchqemux86-6483.3%none16.7%11687 0.0%none100.0%
7514 BSP/QEMU: master branchqemux86-64,100.0%none0.0%none0.0%none100.0%
7515 BSP/QEMU: master branchqemux86-64-WIC,100.0%none0.0%none0.0%none100.0%
7517 BSP/QEMU: master branchqemux86-lsb100.0%none0.0%none0.0%none100.0%
7516 BSP/QEMU: master branchqemux86-WIC100.0%none0.0%none0.0%none100.0%
7518 BSP/QEMU: master branchqemux86_64-lsb100.0%none0.0%none0.0%none100.0%
7536 Runtime: master branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
7548 Package-Management-Issues: master branchAny Supported Distro100.0%none0.0%none0.0%none100.0%
Total: 49


Bug Status


Test Case Status Report
StatusCount
BLOCKED15
FAILED21
PASSED3077
RUNNING10
Total: 3123

Bugs Found during QA Test

New issues Found during QA Test & Community

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
11681poky-tiny fails to build because of resource unavailabilityMediumJuan Ramos 2.4VERIFIEDFIXED---2017-06-16
Development Manual
11699[megamanual] dnf using rpm: 5.18.4.3.1. Using RPM section changes requiredMediumKristi 2.4 M2VERIFIEDFIXED---2017-06-21
General Runtime
11687Can not run QEMU machines under UNFSMedium+Robert YangYi Zhao2.4 M2VERIFIEDFIXEDRegression (Used to work)2017-06-19


High and M+ issuesFound during QA Test (NOT NEW)

Meta-yocto
10477yocto-bsp: tool should output a conf file to be consume by the qemu runnerMediumLeonardo Sandoval Gonzalez 2.5RESOLVEDOBSOLETE---2016-10-21

Other Bugs found during QA Test (NOT NEW)

IDSummary (2 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18
Runtime Testing
11401oe-selftest failed SANITY_TESTED_DISTROS asked to be unsetMediumJose Perez Coscar lopez arandas2.4RESOLVEDDUPLICATE---2017-04-21

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestoneor release which are Medium+ or High and are not targetedfor the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of thisweek's Yocto Bug Trend charts can be accessed on thefollowing [1]

Build Performance Test

Detailed description on how performance test is run can be found on the Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.4 M1. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and another one started on 2.1 from GDC. There are differences between HW used in both locations, hence the numbers are different

Below chars are showing the latestperformance results taken from the current milestone.

Performance Charts

For Bitbake core-image-sato withrm_work

Ww25-2k17 - rmwork.png

Building the Virtual kernel

Ww25-2k17 - kernel.png

Building the rootfs

Ww25-2k17 - rootfs.png

Building core-image-sato

Ww25-2k17 - sato.png

eSDK Deploy time

Ww25-2k17 - esdk.png

pTest for genericx86-64 on NUC

open bugs

IDSummary (23 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
10266Add ptest for gnutlsMediumUnassigned 4.99NEW---2016-09-13
5605Add ptest for ffmpegLowTony Tascioglu FutureNEW---2013-11-27
6707Add ptest for curlMediumUnassignedApoorvFutureNEW---2014-09-11
11674core-image-sato-sdk-ptest build error for cpio imageMediumRoss Burton 4.99NEW---2017-06-15
10841pTest : Various test cases are failing on "dbus-test" moduleMediumRoss Burton 2.5RESOLVEDDUPLICATE---2016-12-21
10953util-linux installs libtool wrapper scripts instead of binaries for ptestMediumJuro Bystricky 2.5 M1RESOLVEDFIXED---2017-01-23
11024lttng-tools ships libtool wrappers as ptest binariesMediumdengke 2.7RESOLVEDFIXED---2017-02-07
11547Add regression check for ptestMedium+Robert Yang 2.4 M3RESOLVEDFIXED---2017-05-19
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
11596bzip2-ptest: improve reproducible buildMediumCinly Ooi 2.5 M1RESOLVEDFIXED---2017-05-31
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
11602acl-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-01
11666attr-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-14
5698bash: ptest cases failedMediumYi Zhao FutureRESOLVEDDUPLICATE---2013-12-31
11667flex-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-14
11668libz-ptest contains build host referencesMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-15
Program Management
8064ptest fails tracker bugMediumRoss Burton 2.5RESOLVEDOBSOLETE---2015-07-28
Recipe Reporting System (RRS)
8650RRS: track ptest-enabled and potentially ptest-enabled statusMediumUnassignedyeoh ee peng4.99NEW---2015-11-05
Runtime Testing
6737Extend automated ptest testing to work with opkgUndecidedSonia Delgadoyeoh ee pengFutureRESOLVEDOBSOLETE---2014-09-18
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 4.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng4.99RESOLVEDFIXEDRegression (Used to work)2015-12-17
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03

results

< Back
Ran on a NUC and compared with commit Ptest 2568e18701fb20d7105eb6e4929f3aff4b9f9c06

FULL LOG


Failures
 % increased
 % decreased


Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.00% 100.00%
attr 143 143 0 100.00% 100.00%
bash 76 72 4 94.70% 94.70%
bluez5 7 7 0 100.00% 100.00%
busybox 487 480 7 98.60% 98.40%
bzip2 6 6 0 100.00% 100.00%
dbus-test 15 4 11 26.70% 26.70%
diffstat 440 440 0 100.00% 100.00%
diffutils 17 17 0 100.00% 100.00%
e2fsprogs 1 1 0 100.00% 100.00%
flex 114 114 0 100.00% 100.00%
gawk 298 297 1 99.70% 99.70%
gdbm 25 25 0 100.00% 100.00%
gdk-pixbuf 20 19 1 95.0% 95.0%
glib-2.0 284 277 7 97.50% 97.20%
gzip 18 18 0 100.00% 0%
kbd 7 7 0 100.00% 100.00%
libpcre 3 3 0 100.00% 100.00%
libxml2 132 129 3 97.70% 96.50%
lttng-tools 2057 1988 69 96.60% 96.60%
nettle 88 87 1 98.90% 98.90%
openssh 58 58 0 100.00% 100.00%
openssl 52 52 0 100.00% 100.00%
pango 12 12 0 100.00% 100.00%
perl 2286 2285 1 100.00% 100.00%
perl5 2286 2285 1 100.00% 100.00%
python 32230 32212 118 99.60% 99.60%
quilt 57 57 0 100.00 100.00%
strace 335 334 1 99.70% 99.70%
tcl 148 146 2 98.60% 100.00%
util-linux 108 106 2 98.10% 98.20%
valgrind 555 293 262 52.80% 53.50%
zlib 2 2 0 100.00% 100.00%

Detailed Report

ADT

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

BitBake

Status

Bugs

**No results**

Agenda

BSPs

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

CROPS

Status

Bugs

**No results**

Agenda

Eclipse Plugin

Status

Bugs

Agenda

Status

Bugs

Agenda

eSDK

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

General Runtime

Status

Bugs

**No results**

Agenda

Kernel

Status

Bugs

**No results**

Agenda

Meta-yocto

Status

Bugs

Agenda

OE-Core

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda