WW32 - 2017-08-09 - Full Test Cycle 2.4 M2 rc3

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 
 Test type: Full Pass Test
 Branch: master
 Build name: 2.4 M2 RC3
 poky commit: 9ed748a542b520c1cb763d981969233c0f5efd4e
 Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.4_M2.rc3
 Distributions tested: 
       CentOS 7.1
       Debian 8
       Fedora 24
       Fedora 25
       OpenSuse 422
       Ubuntu 16.04
       Ubuntu 17.04

  
Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
7846 7847 eSDK: master branchCentos 7.1100.0%none0.0%none0.0%none100.0%
7843 CROPS: master branchCentos 7.1100.0%none0.0%none0.0%none100.0%
7803 7804 ADT: master branchFedora 26 i686100.0%none0.0%none0.0%none100.0%
7805 7806 ADT: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
7849 Kernel: master branchCentos 7100.0%none0.0%none0.0%none100.0%
7844 Eclipse-Plugin: master branchMars on Fedora 22 x86_64,71.4%none28.6%11833 11923 0.0%none100.0%
7845 Eclipse-Plugin: master branchNeon on Fedora 22 x86_64100.0%11923 0.0%none0.0%none100.0%
7850 Meta-yocto: master branchUbuntu 14.04 x86_64100.0%10477 0.0%none0.0%none100.0%
7860 Build-Appliance: master branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
7861 7862 Toaster: master branchDebian 4.3.0-0.bpo.1-amd64100.0%11833 0.0%none0.0%none100.0%
7851 OE-Core: master branchCentOS 7.1 x86_64100.0%none0.0%none0.0%none100.0%
7852 OE-Core: master branchDebian 8 x86_64100.0%none0.0%none0.0%none100.0%
7854 OE-Core: master branchFedora 24 x86_6499.2%none0.8%11917 0.0%none100.0%
7853 OE-Core: master branchFedora 25 x86_64100.0%none0.0%none0.0%none100.0%
7856 OE-Core: master branchOpenSuse 422 x86_64100.0%none0.0%none0.0%none100.0%
7857 OE-Core: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
7858 OE-Core: master branchUbuntu 17.04 x86-64100.0%none0.0%none0.0%none100.0%
7807 BitBake: master branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
7867 7868 BSP/QEMU: master branchAny QEMU100.0%none0.0%none0.0%none100.0%
7838 BSP/QEMU: master branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
7810 BSP/QEMU: master branchBeaglebone-Black100.0%none0.0%none0.0%none100.0%
7811 7839 BSP/QEMU: master branchEdgeRouter100.0%none0.0%none0.0%none100.0%
7831 BSP/QEMU: master branchgenericx86 - on MMAX32bit97.8%none2.2%10455 11879 0.0%none100.0%
7832 BSP/QEMU: master branchgenericx86-64 on MMAX6497.8%none2.2%10455 11879 0.0%none100.0%
7833 BSP/QEMU: master branchgenericx86-64 on NUC97.8%none2.2%9267 0.0%none100.0%
7841 BSP/QEMU: master branchgenericx86-64-lsb on MMAX64100.0%none0.0%none0.0%none100.0%
7840 BSP/QEMU: master branchgenericx86-64-lsb on NUC100.0%none0.0%none0.0%none100.0%
7834 BSP/QEMU: master branchgenericx86-64-WIC on MMAX6497.8%none2.2%11879 0.0%none100.0%
7835 BSP/QEMU: master branchgenericx86-64-WIC on NUC97.8%none2.2%9267 0.0%none100.0%
7809 BSP/QEMU: master branchGenericx86-64_NUC100.0%none0.0%none0.0%none100.0%
7842 BSP/QEMU: master branchgenericx86-lsb on MMAX32bit100.0%none0.0%none0.0%none100.0%
7836 BSP/QEMU: master branchgenericx86-WIC on MMAX3297.8%none2.2%11879 0.0%none100.0%
7808 BSP/QEMU: master branchGenericx86_MMAX32100.0%none0.0%none0.0%none100.0%
7812 7837 BSP/QEMU: master branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
7813 7820 BSP/QEMU: master branchqemu-x86100.0%none0.0%none0.0%none100.0%
7814 7821 BSP/QEMU: master branchqemuarm100.0%none0.0%none0.0%none100.0%
7815 7822 BSP/QEMU: master branchqemuarm64100.0%none0.0%none0.0%none100.0%
7816 7823 BSP/QEMU: master branchqemumips100.0%none0.0%none0.0%none100.0%
7817 7824 BSP/QEMU: master branchqemumips64100.0%none0.0%none0.0%none100.0%
7818 7825 BSP/QEMU: master branchqemuppc100.0%none0.0%none0.0%none100.0%
7819 BSP/QEMU: master branchqemux86-64100.0%none0.0%none0.0%none100.0%
7826 BSP/QEMU: master branchqemux86-64,100.0%none0.0%none0.0%none100.0%
7827 BSP/QEMU: master branchqemux86-64-WIC,100.0%none0.0%none0.0%none100.0%
7829 BSP/QEMU: master branchqemux86-lsb100.0%none0.0%none0.0%none100.0%
7828 BSP/QEMU: master branchqemux86-WIC100.0%none0.0%none0.0%none100.0%
7830 BSP/QEMU: master branchqemux86_64-lsb100.0%none0.0%none0.0%none100.0%
7848 Runtime: master branchuser-preference (any supported environment)100.0%none0.0%none0.0%none100.0%
7863 Package-Management-Issues: master branchAny Supported Distro100.0%none0.0%none0.0%none100.0%
Total: 48


Bug Status


Test Case Status Report
StatusCount
FAILED10
PASSED3493
RUNNING9
Total: 3512

Bugs Found during QA Test

New issues Found during QA Test & Community

IDSummary (2 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
Eclipse Plugin
11923[Eclipse] Failed to debug a Project in Eclipse and displays error: java.lang.NullPointerExceptionMedium+LibertadLibertad2.4 M4RESOLVEDNOTABUG---2017-08-10
OE-Core
11917(buildoptions) Failure on test_arch_work_dir and layer_with_out_git_dirMedium+oscar lopez arandas 2.4 M4RESOLVEDFIXED---2017-08-09


High and M+ issuesFound during QA Test (NOT NEW)

IDSummary (2 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
11879Testimage: multiple "uvesafb" errors present in log in minnowmax on genericx86/genericx86-64Medium+Alejandro Hernandez 2.4 M4RESOLVEDFIXED---2017-07-31
Toaster
11833Toaster timeout for bitbake serverMedium+David ReynaLibertad2.4 M3RESOLVEDDUPLICATE---2017-07-25

Other Bugs found during QA Test (NOT NEW)

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18
Meta-yocto
10477yocto-bsp: tool should output a conf file to be consume by the qemu runnerMediumLeonardo Sandoval Gonzalez 2.5RESOLVEDOBSOLETE---2016-10-21
Regression (Found by QA test)
9267test_parselogs (oeqa.runtime.parselogs.ParseLogsTest) failed on NUC 6i5SYH, firmware load for i915/skl_guc_ver4.bin failedMediumCalifornia SullivanApoorv2.5RESOLVEDFIXEDRegression (Found by QA test)2016-03-15

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestoneor release which are Medium+ or High and are not targetedfor the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of thisweek's Yocto Bug Trend charts can be accessed on thefollowing [1]

Build Performance Test

Detailed description on how performance test are run can be found on the Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.4 M1. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and another one started on 2.1 from GDC. There are differences between HW used in both locations, hence the numbers are different.

Below charts are showing the latest performance results taken from the current milestone.

Performance Charts

Table

Ww32-2k17 - table.png


For Bitbake core-image-sato withrm_work

Ww32-2k17 - rmwork.png

Building the Virtual kernel

Ww32-2k17 - kernel.png

Building the rootfs

Ww32-2k17 - rootfs.png

Building core-image-sato

Ww32-2k17 - sato.png

eSDK Deploy time

Ww32-2k17 - esdk.png

pTest for genericx86-64 on NUC

open bugs

IDSummary (25 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
10266Add ptest for gnutlsMediumUnassigned 4.99NEW---2016-09-13
5605Add ptest for ffmpegLowTony Tascioglu FutureNEW---2013-11-27
6707Add ptest for curlMediumUnassignedApoorvFutureNEW---2014-09-11
11674core-image-sato-sdk-ptest build error for cpio imageMediumRoss Burton 4.99NEW---2017-06-15
10841pTest : Various test cases are failing on "dbus-test" moduleMediumRoss Burton 2.5RESOLVEDDUPLICATE---2016-12-21
10953util-linux installs libtool wrapper scripts instead of binaries for ptestMediumJuro Bystricky 2.5 M1RESOLVEDFIXED---2017-01-23
11024lttng-tools ships libtool wrappers as ptest binariesMediumdengke 2.7RESOLVEDFIXED---2017-02-07
11547Add regression check for ptestMedium+Robert Yang 2.4 M3RESOLVEDFIXED---2017-05-19
11596bzip2-ptest: improve reproducible buildMediumCinly Ooi 2.5 M1RESOLVEDFIXED---2017-05-31
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
11602acl-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-01
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
11666attr-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-14
5698bash: ptest cases failedMediumYi Zhao FutureRESOLVEDDUPLICATE---2013-12-31
11667flex-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-14
11668libz-ptest contains build host referencesMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-15
Program Management
8064ptest fails tracker bugMediumRoss Burton 2.5RESOLVEDOBSOLETE---2015-07-28
Recipe Reporting System (RRS)
8650RRS: track ptest-enabled and potentially ptest-enabled statusMediumUnassignedyeoh ee peng4.99NEW---2015-11-05
Runtime Testing
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03
6737Extend automated ptest testing to work with opkgUndecidedSonia Delgadoyeoh ee pengFutureRESOLVEDOBSOLETE---2014-09-18
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 4.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09
11813ptest- openssh cases failedMediumArmin Kuster 4.99RESOLVEDFIXED---2017-07-19
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng4.99RESOLVEDFIXEDRegression (Used to work)2015-12-17
11812ptest- libxml2 cases failedMediumJuro Bystricky 2.4VERIFIEDFIXED---2017-07-19

results

< Back
Ran on a NUC and compared with commit Ptest 8e15e9b6e478f6368034519b2a8fd3c7ea71d23b

FULL LOG


Failures
 % increased
 % decreased


Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.00% 100.00%
attr 143 143 0 100.00% 100.00%
bash 76 72 4 94.70% 94.70%
bluez5 25 25 0 100.00% 100.00%
busybox 518 509 9 98.30% 98.30%
bzip2 6 6 0 100.00% 100.00%
dbus-test 15 4 11 26.70% 26.70%
diffstat 440 440 0 100.00% 100.00%
diffutils 17 17 0 100.00% 100.00%
e2fsprogs 1 1 0 100.00% 100.00%
flex 114 114 0 100.00% 100.00%
gawk 298 297 1 99.70% 99.70%
gdbm 25 25 0 100.00% 100.00%
gdk-pixbuf 20 19 1 95.0% 95.0%
glib-2.0 253 252 1 99.60% 98.80%
gzip 18 18 0 100.0% 100.0%
kbd 7 7 0 100.00% 100.00%
libpcre 3 3 0 100.00% 100.00%
libxml2 39 36 3 92.30% 92.30%
lttng-tools 2057 1988 69 96.60% 96.60%
nettle 88 87 1 98.90% 98.90%
openssh 58 58 0 100.00% 100.00%
openssl 52 52 0 100.00% 100.00%
pango 12 12 0 100.00% 100.00%
perl 2286 2285 1 100.00% 100.00%
perl5 2286 2285 1 100.00% 100.00%
quilt 57 57 0 100.00 100.00%
slang 97 97 0 100.00 100.00%
strace 353 352 1 99.70% 99.70%
tcl 148 146 2 98.60% 98.60%
util-linux 123 102 21 82.90% 82.90%
valgrind 559 299 260 53.50% 53.80%
zlib 2 2 0 100.00% 100.00%


Compliance Report

POSIX results

General results for POSIX execution can be found POSIX_result and greater details on each architecture can be found POSIX_details

Bellow you will see the last two execution results for comparison:


Yocto 2.4 M2 RC3 build

Status genericx86-64 beaglebone edgerouter mpc8315e-rdb
failures 46 45 46 45

Yocto 2.4 M2 RC2 build

Status genericx86-64 beaglebone edgerouter mpc8315e-rdb
failures 46 45 46 45

Yocto 2.4 M1 RC1 build

Status genericx86-64 beaglebone edgerouter mpc8315e-rdb
failures 45 45 46 45


LTP results

General results for LTP can be found here LTP_results and for greater information click on each architecture to be redirected to their details. There is currently an infrastructure issue that is blocking the beblebone and edgerouter testing for LTP, results are not shown in the tables below.


Yocto 2.4 M2 RC3 build

Status genericx86-64 beaglebone edgerouter mpc8315e-rdb
total 1384 1346 1346
failures 23 15 27

Yocto 2.4 M2 RC2 build

Status genericx86-64 beaglebone edgerouter mpc8315e-rdb
total 1384 1348
failures 24 30

Yocto 2.4 M1 RC1 build

Status genericx86-64 beaglebone edgerouter mpc8315e-rdb
total 1384 1348
failures 22 30

Detailed Report

ADT

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

BitBake

Status

Bugs

**No results**

Agenda

BSPs

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

CROPS

Status

Bugs

**No results**

Agenda

Eclipse Plugin

Status

Bugs

Agenda

Status

Bugs

Agenda

eSDK

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

General Runtime

Status

Bugs

**No results**

Agenda

Kernel

Status

Bugs

**No results**

Agenda

Meta-yocto

Status

Bugs

Agenda

OE-Core

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda