WW35 - 2017-08-30 - Full Test Cycle 2.4 M3

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 
 Test type: Full Pass Test 
 Branch: master
 Build name: 2.4 M3 RC1
 poky commit: 5f6945f5031e1a4ca116cc1eccf4c2f9dc228547
 Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.4_M3.rc1
 Distributions tested: 
          CentOS 7.1 x86_64
          Debian 8 x86_64
          Fedora 24 x86_64
          Fedora 25 x86_64
          OpenSuse 422
          ubuntu 16.04 x86_64
          Ubuntu 17.04 x86-64
          Fedora 24 x86_64


  
Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
7991 7992 eSDK: master branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
7988 CROPS: master branchCentos 7.1100.0%none0.0%none0.0%none100.0%
7948 7949 ADT: master branchFedora 26 i68693.3%none6.7%11994 0.0%none100.0%
7950 7951 ADT: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
7994 Kernel: master branchCentos 7100.0%none0.0%none0.0%none100.0%
7989 Eclipse-Plugin: master branchMars on Fedora 22 x86_64,100.0%none0.0%none0.0%none100.0%
7990 Eclipse-Plugin: master branchNeon on Fedora 22 x86_64100.0%none0.0%none0.0%none100.0%
7995 Meta-yocto: master branchUbuntu 14.04 x86_6495.8%10477 11995 4.2%11984 0.0%none100.0%
8005 Build-Appliance: master branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
8006 8007 Toaster: master branchDebian 4.3.0-0.bpo.1-amd6493.3%none6.7%12005 12006 9977 12004 0.0%none100.0%
7996 OE-Core: master branchCentOS 7.1 x86_6499.6%none0.4%11999 0.0%none100.0%
7997 OE-Core: master branchDebian 8 x86_64100.0%none0.0%none0.0%none100.0%
7999 OE-Core: master branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
7998 OE-Core: master branchFedora 25 x86_64100.0%none0.0%none0.0%none100.0%
8000 OE-Core: master branchOpenSuse 422 x86_64100.0%none0.0%none0.0%none100.0%
8002 OE-Core: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
8003 OE-Core: master branchUbuntu 17.04 x86-64100.0%none0.0%none0.0%none100.0%
7952 BitBake: master branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
7955 7983 BSP/QEMU: master branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
7956 7984 BSP/QEMU: master branchEdgeRouter100.0%none0.0%none0.0%none100.0%
7976 BSP/QEMU: master branchgenericx86 - on MMAX32bit97.8%none2.2%10455 11879 0.0%none100.0%
7977 BSP/QEMU: master branchgenericx86-64 on MMAX6497.8%none2.2%10455 11879 0.0%none100.0%
7978 BSP/QEMU: master branchgenericx86-64 on NUC97.8%none2.2%11879 0.0%none100.0%
7986 BSP/QEMU: master branchgenericx86-64-lsb on MMAX64100.0%none0.0%none0.0%none100.0%
7985 BSP/QEMU: master branchgenericx86-64-lsb on NUC100.0%none0.0%none0.0%none100.0%
7979 BSP/QEMU: master branchgenericx86-64-WIC on MMAX6497.8%none2.2%11879 0.0%none100.0%
7980 BSP/QEMU: master branchgenericx86-64-WIC on NUC97.8%none2.2%11879 0.0%none100.0%
7954 BSP/QEMU: master branchGenericx86-64_NUC100.0%none0.0%none0.0%none100.0%
7987 BSP/QEMU: master branchgenericx86-lsb on MMAX32bit100.0%none0.0%none0.0%none100.0%
7981 BSP/QEMU: master branchgenericx86-WIC on MMAX3297.8%none2.2%11879 0.0%none100.0%
7953 BSP/QEMU: master branchGenericx86_MMAX32100.0%none0.0%none0.0%none100.0%
7957 7982 BSP/QEMU: master branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
7958 7965 BSP/QEMU: master branchqemu-x86100.0%none0.0%none0.0%none100.0%
7959 7966 BSP/QEMU: master branchqemuarm100.0%none0.0%none0.0%none100.0%
7960 7967 BSP/QEMU: master branchqemuarm64100.0%none0.0%none0.0%none100.0%
7961 7968 BSP/QEMU: master branchqemumips100.0%none0.0%none0.0%none100.0%
7962 7969 BSP/QEMU: master branchqemumips64100.0%none0.0%none0.0%none100.0%
7963 7970 BSP/QEMU: master branchqemuppc100.0%none0.0%none0.0%none100.0%
7964 BSP/QEMU: master branchqemux86-64100.0%none0.0%none0.0%none100.0%
7971 BSP/QEMU: master branchqemux86-64,100.0%none0.0%none0.0%none100.0%
7972 BSP/QEMU: master branchqemux86-64-WIC,100.0%none0.0%none0.0%none100.0%
7974 BSP/QEMU: master branchqemux86-lsb100.0%none0.0%none0.0%none100.0%
7973 BSP/QEMU: master branchqemux86-WIC100.0%none0.0%none0.0%none100.0%
7975 BSP/QEMU: master branchqemux86_64-lsb100.0%none0.0%none0.0%none100.0%
7993 Runtime: master branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
8008 Package-Management-Issues: master branchAny Supported Distro100.0%none0.0%none0.0%none100.0%
Total: 46


Bug Status


Test Case Status Report
StatusCount
FAILED21
PASSED3457
RUNNING11
Total: 3489

Bugs Found during QA Test

New issues Found during QA Test & Community

IDSummary (8 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
12013Poky tiny image ask for passwordMedium+Alejandro Hernandez 2.4 M4VERIFIEDNOTABUG---2017-08-30
General Runtime
11994Graphical qemu doesn't work over remote X on Fedora 26Medium+Mingli YuYi Zhao4.99RESOLVEDOBSOLETERegression (Used to work)2017-08-29
Meta-yocto
11984crosstap doesn't work on 2.4 M3 RC1Medium+Saul WoldYi Zhao2.4 M4VERIFIEDFIXEDRegression (Used to work)2017-08-28
11995Update yocto-bsp script for kernel 4.12Medium+Leonardo Sandoval GonzalezYi Zhao2.4 M4VERIFIEDFIXEDRegression (Found by QA test)2017-08-29
OE-Core
11999OE-Core Selftest test_recipetool_create_github (recipetool.RecipetoolTests) FailUndecidedJussi Kukkonen ---RESOLVEDNOTABUG---2017-08-29
Toaster
12004Views: Edit Columns button is not alphabetically orderedMediumDavid ReynaLibertad2.4 M4RESOLVEDFIXED---2017-08-30
12005Cant download other artifacts when modifying built recipeMediumDavid Reyna 4.99ACCEPTED---2017-08-30
12006Image recipes view: Recipe file link not redirecting correctlyMediumDavid ReynaLibertad2.4 M4RESOLVEDFIXED---2017-08-30


High and M+ issuesFound during QA Test (NOT NEW)

BSPs
11879Testimage: multiple "uvesafb" errors present in log in minnowmax on genericx86/genericx86-64Medium+Alejandro Hernandez 2.4 M4RESOLVEDFIXED---2017-07-31

Other Bugs found during QA Test (NOT NEW)

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18
Meta-yocto
10477yocto-bsp: tool should output a conf file to be consume by the qemu runnerMediumLeonardo Sandoval Gonzalez 2.5RESOLVEDOBSOLETE---2016-10-21
Toaster
9977Time, CPU and Disk I/O tables have the wrong core columnsLowToaster default assignee FutureNEW---2016-07-19

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestoneor release which are Medium+ or High and are not targetedfor the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of thisweek's Yocto Bug Trend charts can be accessed on thefollowing [1]

Build Performance Test

Detailed description on how performance test are run can be found on the Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.4 M1. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and another one started on 2.1 from GDC. There are differences between HW used in both locations, hence the numbers are different.

Below charts are showing the latest performance results taken from the current milestone.

Performance Charts

Table

Ww35-2k17 - table.png

For Bitbake core-image-sato withrm_work

Ww35-2k17 - rmwork.png

Building the Virtual kernel

Ww35-2k17 - kernel.png

Building the rootfs

Ww35-2k17 - rootfs.png

Building core-image-sato

Ww35-2k17 - sato.png

eSDK Deploy time

Ww35-2k17 - esdk.png

pTest for genericx86-64 on NUC

open bugs

IDSummary (25 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
10266Add ptest for gnutlsMediumUnassigned 4.99NEW---2016-09-13
5605Add ptest for ffmpegLowTony Tascioglu FutureNEW---2013-11-27
6707Add ptest for curlMediumUnassignedApoorvFutureNEW---2014-09-11
11674core-image-sato-sdk-ptest build error for cpio imageMediumRoss Burton 4.99NEW---2017-06-15
10841pTest : Various test cases are failing on "dbus-test" moduleMediumRoss Burton 2.5RESOLVEDDUPLICATE---2016-12-21
10953util-linux installs libtool wrapper scripts instead of binaries for ptestMediumJuro Bystricky 2.5 M1RESOLVEDFIXED---2017-01-23
11024lttng-tools ships libtool wrappers as ptest binariesMediumdengke 2.7RESOLVEDFIXED---2017-02-07
11547Add regression check for ptestMedium+Robert Yang 2.4 M3RESOLVEDFIXED---2017-05-19
11596bzip2-ptest: improve reproducible buildMediumCinly Ooi 2.5 M1RESOLVEDFIXED---2017-05-31
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
11602acl-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-01
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
11666attr-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-14
5698bash: ptest cases failedMediumYi Zhao FutureRESOLVEDDUPLICATE---2013-12-31
11667flex-ptest references host build pathMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-14
11668libz-ptest contains build host referencesMediumJuro Bystricky 2.4RESOLVEDFIXED---2017-06-15
Program Management
8064ptest fails tracker bugMediumRoss Burton 2.5RESOLVEDOBSOLETE---2015-07-28
Recipe Reporting System (RRS)
8650RRS: track ptest-enabled and potentially ptest-enabled statusMediumUnassignedyeoh ee peng4.99NEW---2015-11-05
Runtime Testing
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng4.99RESOLVEDFIXEDRegression (Used to work)2015-12-17
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03
6737Extend automated ptest testing to work with opkgUndecidedSonia Delgadoyeoh ee pengFutureRESOLVEDOBSOLETE---2014-09-18
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 4.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09
11813ptest- openssh cases failedMediumArmin Kuster 4.99RESOLVEDFIXED---2017-07-19
11812ptest- libxml2 cases failedMediumJuro Bystricky 2.4VERIFIEDFIXED---2017-07-19

results

< Back
Ran on a NUC and compared with commit Ptest 9ed748a542b520c1cb763d981969233c0f5efd4e

FULL LOG


Failures
 % increased
 % decreased


Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.00% 100.00%
attr 143 143 0 100.00% 100.00%
bash 79 75 4 94.90% 94.70%
bluez5 25 25 0 100.00% 100.00%
busybox 518 509 9 98.30% 98.30%
bzip2 6 6 0 100.00% 100.00%
dbus-test 15 4 11 26.70% 26.70%
diffstat 440 440 0 100.00% 100.00%
diffutils 17 17 0 100.00% 100.00%
e2fsprogs 1 1 0 100.00% 100.00%
flex 114 114 0 100.00% 100.00%
gawk 298 297 1 99.70% 99.70%
gdbm 25 25 0 100.00% 100.00%
gdk-pixbuf 20 20 0 100.0% 95.0%
glib-2.0 253 252 1 99.60% 99.60%
gzip 18 18 0 100.0% 100.0%
kbd 7 7 0 100.00% 100.00%
libpcre 3 3 0 100.00% 100.00%
libxml2 39 36 3 92.30% 92.30%
lttng-tools 2057 1988 69 96.60% 96.60%
nettle 88 87 1 98.90% 98.90%
openssh 58 58 0 100.00% 100.00%
openssl 52 52 0 100.00% 100.00%
pango 12 12 0 100.00% 100.00%
perl 2286 2285 1 100.00% 100.00%
perl5 2286 2285 1 100.00% 100.00%
quilt 57 57 0 100.00% 100.00%
slang 97 97 0 100.00% 100.00%
strace 355 354 1 99.70% 99.70%
tcl 148 147 1 99.30% 98.60%
util-linux 123 102 21 82.90% 82.90%
valgrind 559 299 260 53.50% 53.50%
zlib 2 2 0 100.00% 100.00%

Compliance Report

POSIX results

General results for POSIX execution can be found POSIX_result and greater details on each architecture can be found POSIX_details

Bellow you will see the last two execution results for comparison:


Yocto 2.4 M3 RC1 build

Status genericx86-64 beaglebone edgerouter mpc8315e-rdb
failures 46 45 46 45

Yocto 2.4 M2 RC3 build

Status genericx86-64 beaglebone edgerouter mpc8315e-rdb
failures 46 45 46 45

Yocto 2.4 M1 RC1 build

Status genericx86-64 beaglebone edgerouter mpc8315e-rdb
failures 45 45 46 45


LTP results

General results for LTP can be found here LTP_results and for greater information click on each architecture to be redirected to their details. There is currently an infrastructure issue that is blocking the beblebone and edgerouter testing for LTP, results are not shown in the tables below.

LTP Result are being updated soon.

Yocto 2.4 M3 RC1 build

Status genericx86-64 beaglebone edgerouter mpc8315e-rdb
total
failures

Yocto 2.4 M2 RC3 build

Status genericx86-64 beaglebone edgerouter mpc8315e-rdb
total 1384 1346 1346
failures 23 15 27

Yocto 2.4 M1 RC1 build

Status genericx86-64 beaglebone edgerouter mpc8315e-rdb
total 1384 1348
failures 22 30

Detailed Report

ADT

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

BitBake

Status

Bugs

**No results**

Agenda

BSPs

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

CROPS

Status

Bugs

**No results**

Agenda

Eclipse Plugin

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

eSDK

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

General Runtime

Status

Bugs

**No results**

Agenda

Kernel

Status

Bugs

**No results**

Agenda

Meta-yocto

Status

Bugs

Agenda

OE-Core

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda