WW37 - 2017-09-13 - Full Point Release Test Cycle 2.2.2 rc2

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 
 Test type: Full Pass Test
 Branch: morty
 Build name: 2.2.2 RC2
 poky commit: 60402978fe3648bf560b3386c6e9dd661cdf2083 
 Autobuilder images repository: 
            https://autobuilder.yocto.io/pub/releases/yocto-2.2.2.rc2
 Distributions tested: 
          Fedora 25
          Ubuntu 16
          CentOS 7.1
          Debian 8
          Fedora 24
          OpenSuse 42.3
          Ubuntu 17

  
Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
8134 eSDK: morty branchCentos 7.10.0%none100.0%12080 0.0%none100.0%
8133 eSDK: morty branchDebian 80.0%none100.0%12082 0.0%none100.0%
8091 8092 ADT: morty branchFedora 25 i686100.0%none0.0%none0.0%none100.0%
8093 8094 ADT: morty branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
8131 Eclipse-Plugin: morty branchMars on Fedora 22 x86_64,100.0%none0.0%none0.0%none100.0%
8132 Eclipse-Plugin: morty branchNeon on Fedora 22 x86_64100.0%none0.0%none0.0%none100.0%
8136 Meta-yocto: morty branchUbuntu 14.04 x86_64100.0%none0.0%none0.0%none100.0%
8146 Build-Appliance: morty branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
8147 8148 Toaster: morty branchDebian 4.3.0-0.bpo.1-amd64100.0%none0.0%none0.0%none100.0%
8137 OE-Core:morty branchCentOS 7.1 x86_64100.0%none0.0%none0.0%none100.0%
8138 OE-Core:morty branchDebian 8 x86_64100.0%none0.0%none0.0%none100.0%
8140 OE-Core:morty branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
8139 OE-Core:morty branchFedora 25 x86_64100.0%none0.0%none0.0%none100.0%
8141 OE-Core:morty branchOpenSuse 42.3 x86_64100.0%none0.0%none0.0%none100.0%
8143 OE-Core:morty branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
8144 OE-Core:morty branchUbuntu 17.04 x86-6499.5%none0.5%12073 0.0%none100.0%
8095 BitBake: morty branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
8126 BSP/QEMU: morty branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
8098 BSP/QEMU: morty branchBeaglebone-Black100.0%none0.0%none0.0%none100.0%
8099 8127 BSP/QEMU: morty branchEdgeRouter100.0%none0.0%none0.0%none100.0%
8119 BSP/QEMU: morty branchgenericx86 - on MMAX32bit98.1%none1.9%10455 9271 0.0%none100.0%
8120 BSP/QEMU: morty branchgenericx86-64 on MMAX6498.1%none1.9%10455 0.0%none100.0%
8121 BSP/QEMU: morty branchgenericx86-64 on NUC98.1%none1.9%10455 0.0%none100.0%
8129 BSP/QEMU: morty branchgenericx86-64-lsb on MMAX64100.0%none0.0%none0.0%none100.0%
8128 BSP/QEMU: morty branchgenericx86-64-lsb on NUC100.0%none0.0%none0.0%none100.0%
8122 BSP/QEMU: morty branchgenericx86-64-WIC on MMAX64100.0%none0.0%none0.0%none100.0%
8123 BSP/QEMU: morty branchgenericx86-64-WIC on NUC100.0%none0.0%none0.0%none100.0%
8097 BSP/QEMU: morty branchGenericx86-64_NUC100.0%none0.0%none0.0%none100.0%
8130 BSP/QEMU: morty branchgenericx86-lsb on MMAX32bit100.0%none0.0%none0.0%none100.0%
8124 BSP/QEMU: morty branchgenericx86-WIC on MMAX32100.0%9271 0.0%none0.0%none100.0%
8096 BSP/QEMU: morty branchGenericx86_MMAX32100.0%none0.0%none0.0%none100.0%
8100 8125 BSP/QEMU: morty branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
8101 8108 BSP/QEMU: morty branchqemu-x86100.0%none0.0%none0.0%none100.0%
8102 8109 BSP/QEMU: morty branchqemuarm100.0%none0.0%none0.0%none100.0%
8103 8110 BSP/QEMU: morty branchqemuarm64100.0%none0.0%none0.0%none100.0%
8104 8111 BSP/QEMU: morty branchqemumips100.0%none0.0%none0.0%none100.0%
8105 8112 BSP/QEMU: morty branchqemumips64100.0%none0.0%none0.0%none100.0%
8106 8113 BSP/QEMU: morty branchqemuppc100.0%none0.0%none0.0%none100.0%
8107 BSP/QEMU: morty branchqemux86-64100.0%none0.0%none0.0%none100.0%
8114 BSP/QEMU: morty branchqemux86-64,100.0%none0.0%none0.0%none100.0%
8115 BSP/QEMU: morty branchqemux86-64-WIC,100.0%none0.0%none0.0%none100.0%
8117 BSP/QEMU: morty branchqemux86-lsb98.1%none1.9%12075 0.0%none100.0%
8116 BSP/QEMU: morty branchqemux86-WIC100.0%none0.0%none0.0%none100.0%
8118 BSP/QEMU: morty branchqemux86_64-lsb98.1%none1.9%12075 0.0%none100.0%
8135 Runtime: morty branchuser-preference (any supported environment)100.0%none0.0%none0.0%none100.0%
Total: 45


Bug Status


Test Case Status Report
StatusCount
FAILED14
PASSED3101
Total: 3115

Bugs Found during QA Test

New issues Found during QA Test & Community

IDSummary (4 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
12075[morty] testimage quemux86/qemux86-64 lsb tcgetattr: Input/output error^MMedium+Armin Kuster 2.2.3RESOLVEDWORKSFORME---2017-09-13
Build Testing
12081[morty] PTEST: Package Name flex, gzip, slang, do not appear in 2.2.2 rc2Medium+Juro Bystricky 2.1.4RESOLVEDWONTFIX---2017-09-13
eSDK
12080[Morty][Test Case 1605] TCTEMP_2.3_AUTO_sdkext_eSDK_devtool_build_makeMedium+Paul EggletonFrancisco Pedraza2.2.2RESOLVEDNOTABUGRegression (Used to work)2017-09-13
OE-Core
12073selftest: setUpClass failed failing on ubuntu 17.04 (oeqa.selftest.signing.Signing)Medium+oscar lopez arandas 2.5 M1RESOLVEDDUPLICATE---2017-09-12


High and M+ issuesFound during QA Test (NOT NEW)

"None"

Other Bugs found during QA Test (NOT NEW)

---
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestoneor release which are Medium+ or High and are not targetedfor the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of thisweek's Yocto Bug Trend charts can be accessed on thefollowing [1]

Build Performance Test

Detailed description on how performance test are run can be found on Performance Test wiki. Performance execution can be divided in 2 stages, the first was executed form 1.6 to 2.1 by the Rumania (RO) team and the latest started on 2.1 executed by the Guadalajara (GDC) team. There are differences between HW used in both location hence the numbers are different. For more details click on the following links:

Below chars are showing the latestperformance results taken from the current milestone.

Performance Charts

Comparison table

Ww37-2k17 - table.png

For Bitbake core-image-sato withrm_work

Ww37-2k17 - rmwork.png

Building the Virtual kernel

Ww37-2k17 - kernel.png

Building the rootfs

Ww37-2k17 - rootfs.png

Building core-image-sato

Ww37-2k17 - sato.png

eSDK Deploy time

Ww37-2k17 - esdk.png

pTest for genericx86-64 on NUC

open bugs

IDSummary (27 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
Build Testing
12081[morty] PTEST: Package Name flex, gzip, slang, do not appear in 2.2.2 rc2Medium+Juro Bystricky 2.1.4RESOLVEDWONTFIX---2017-09-13
Manual Testing
11942Tidy up ptest compliance scriptsMedium+yeoh ee peng 2.99RESOLVEDFIXED---2017-08-16
OE-Core
6707Add ptest for curlMediumArmin KusterApoorvFutureIN PROGRESS REVIEW---2014-09-11
4436Add ptest for glibcMediumArmin Kuster FutureNEW---2013-04-29
5605Add ptest for libavLowUnassigned FutureNEW---2013-11-27
10266Add ptest for gnutlsMediumArmin Kuster 2.99NEW---2016-09-13
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
11997libxml2-ptest : build host referencesMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-08-29
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
10841pTest : Various test cases are failing on "dbus-test" moduleMediumRoss Burton 2.5RESOLVEDDUPLICATE---2016-12-21
11998sed-ptest missingMediumRoss Burton 2.5RESOLVEDFIXED---2017-08-29
5698bash: ptest cases failedMediumYi Zhao FutureRESOLVEDDUPLICATE---2013-12-31
10953util-linux installs libtool wrapper scripts instead of binaries for ptestMediumJuro Bystricky 2.5 M1RESOLVEDFIXED---2017-01-23
11024lttng-tools ships libtool wrappers as ptest binariesMediumdengke 2.7RESOLVEDFIXED---2017-02-07
11596bzip2-ptest: improve reproducible buildMediumCinly Ooi 2.5 M1RESOLVEDFIXED---2017-05-31
11667flex-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-14
11668libz-ptest contains build host referencesMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-15
Program Management
8064ptest fails tracker bugMediumRoss Burton 2.5RESOLVEDOBSOLETE---2015-07-28
Recipe Reporting System (RRS)
8650RRS: track ptest-enabled and potentially ptest-enabled statusMediumPaul Eggletonyeoh ee peng2.99NEW---2015-11-05
Runtime Testing
11813ptest- openssh cases failedMediumArmin Kuster 2.99ACCEPTED---2017-07-19
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng2.99ACCEPTEDRegression (Used to work)2015-12-17
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03
6737Extend automated ptest testing to work with opkgUndecidedSonia Delgadoyeoh ee pengFutureRESOLVEDOBSOLETE---2014-09-18
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 2.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09
11811ptest-busybox cases failedMediumJuro Bystricky 2.5 M4RESOLVEDNOTABUG---2017-07-19
11812ptest- libxml2 cases failedMediumJuro Bystricky 2.4VERIFIEDFIXED---2017-07-19

results

< Back
Ran on a NUC and compared with commit Ptest 5f6945f5031e1a4ca116cc1eccf4c2f9dc228547

FULL LOG


Failures
 % increased
 % decreased


Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 354 26 93.20% 100.00%
attr 143 143 0 100.00% 100.00%
bash 76 72 4 94.70% 94.70%
bluez5 25 25 0 100.00% 100.00%
busybox 518 509 9 98.30% 98.30%
bzip2 6 6 0 100.00% 100.00%
dbus-test 15 4 11 26.70% 26.70%
diffstat 440 440 0 100.00% 100.00%
diffutils 16 16 0 100.00% 100.00%
e2fsprogs 1 0 1 0.0% 100.00%
flex
gawk 284 283 1 99.60% 99.70%
gdbm 22 22 0 100.00% 100.00%
gdk-pixbuf 17 17 0 100.0% 95.0%
glib-2.0 249 247 2 99.20% 99.60%
gzip
kbd 7 7 0 100.00% 100.00%
libpcre 3 3 0 100.00% 100.00%
libxml2 85 82 3 96.50% 92.30%
lttng-tools 2467 2462 5 99.80% 96.60%
nettle 87 86 1 98.90% 98.90%
openssh 56 56 0 100.00% 100.00%
openssl 52 52 0 100.00% 100.00%
pango 12 12 0 100.00% 100.00%
perl 2263 2262 1 100.00% 100.00%
perl5 2263 2262 1 100.00% 100.00%
python 32060 31934 126 99.60% 99.60%
quilt 55 54 1 98.2% 100.0%
slang
strace 264 263 1 99.70% 99.70%
tcl 148 148 0 100.00% 98.60%
util-linux 130 128 2 98.50% 82.90%
valgrind 553 2 551 0.40% 53.50%
zlib 2 2 0 100.00% 100.00%

Detailed Report

ADT

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

BitBake

Status

Bugs

**No results**

Agenda

BSPs

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Eclipse Plugin

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

eSDK

Status

Bugs

Agenda

Status

Bugs

Agenda

General Runtime

Status

Bugs

**No results**

Agenda

Meta-yocto

Status

Bugs

**No results**

Agenda

OE-Core

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda