WW11 - 2017-03-15 - Full Test Cycle 2.3 M3 rc1

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 Test type: Full Pass Test Cycle
 Branch: master
 Build name: 2.3 M3 RC1
 poky commit: 6e20b31d5d17133e0fca086e12a0ad06ab5c4cc8 
 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3_M3.rc1
 Distributions tested: 
 -Ubuntu 16.04
 -Ubuntu 16 10 
 -OpenSuse 421
 -Opensuse 13.2  
 -Debian 8
 -Centos 7
 -Fedora 24
 -Fedora 25 

Test Run Summary Report

Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
6882 6883 eSDK: master branchCentos 7.1100.0%none0.0%none0.0%none100.0%
6880 6881 CROPS: master branchCentos 7.1100.0%none0.0%none0.0%none100.0%
6842 6843 ADT: master branchFedora 25 i68699.2%11193 0.8%11223 0.0%none100.0%
6844 6845 ADT: master branchUbuntu 16.04 x86_6499.2%11193 0.8%11223 0.0%none100.0%
6885 Kernel: master branchCentos 7100.0%none0.0%none0.0%none100.0%
6898 Eclipse-Plugin: master branchMars on Fedora 22 x86_64,100.0%none0.0%none0.0%none100.0%
6899 Eclipse-Plugin: master branchNeon on Debian 4.3 x86_6457.1%none42.9%11222 0.0%none100.0%
6886 Meta-yocto: master branchUbuntu 14.04 x86_6487.5%none12.5%11064 11174 0.0%none100.0%
6895 Build-Appliance: master branchVMWare Player on Ubuntu33.3%none66.7%11144 0.0%none100.0%
6896 6897 Toaster: master branchDebian 4.3.0-0.bpo.1-amd6460.6%none4.0%10281 10846 11225 35.4%10846 11225 100.0%
6887 OE-Core: master branchCentOS 7.1 x86_6499.5%none0.5%11025 0.0%none100.0%
6888 OE-Core: master branchDebian 8 x86_64100.0%none0.0%none0.0%none100.0%
6890 OE-Core: master branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
6889 OE-Core: master branchFedora 25 x86_64100.0%none0.0%none0.0%none100.0%
6891 OE-Core: master branchOpenSuse 13.2 x86_64100.0%none0.0%none0.0%none100.0%
6892 OE-Core: master branchOpenSuse421 x86_64100.0%none0.0%none0.0%none100.0%
6893 OE-Core: master branchUbuntu 16.04 x86_6499.5%none0.5%11185 0.0%none100.0%
6894 OE-Core: master branchUbuntu 16.10 x86-64100.0%none0.0%none0.0%none100.0%
6846 BitBake: master branchFedora 24 x86_6487.5%none12.5%none0.0%none100.0%
6875 BSP/QEMU: master branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
6849 BSP/QEMU: master branchBeaglebone-Black100.0%none0.0%none0.0%none100.0%
6850 6876 BSP/QEMU: master branchEdgeRouter100.0%none0.0%none0.0%none100.0%
6868 BSP/QEMU: master branchgenericx86 - on MMAX32bit97.1%none2.9%10455 0.0%none100.0%
6869 BSP/QEMU: master branchgenericx86-64 on MMAX6494.3%none5.7%11173 10455 0.0%none100.0%
6870 BSP/QEMU: master branchgenericx86-64 on NUC94.3%none5.7%11173 10455 0.0%none100.0%
6878 BSP/QEMU: master branchgenericx86-64-lsb on MMAX6497.3%none2.7%11173 0.0%none100.0%
6877 BSP/QEMU: master branchgenericx86-64-lsb on NUC97.3%none2.7%11173 0.0%none100.0%
6871 BSP/QEMU: master branchgenericx86-64-WIC on MMAX6497.1%none2.9%11173 0.0%none100.0%
6872 BSP/QEMU: master branchgenericx86-64-WIC on NUC97.1%none2.9%11173 0.0%none100.0%
6848 BSP/QEMU: master branchGenericx86-64_NUC100.0%none0.0%none0.0%none100.0%
6879 BSP/QEMU: master branchgenericx86-lsb on MMAX32bit97.3%none2.7%11173 0.0%none100.0%
6873 BSP/QEMU: master branchgenericx86-WIC on MMAX3297.1%none2.9%11173 0.0%none100.0%
6847 BSP/QEMU: master branchGenericx86_MMAX32100.0%11224 0.0%none0.0%none100.0%
6851 6874 BSP/QEMU: master branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
6852 6900 BSP/QEMU: master branchqemu-x8695.1%11220 4.9%11119 11173 0.0%none100.0%
6853 6901 BSP/QEMU: master branchqemuarm100.0%10858 0.0%none0.0%none100.0%
6854 6902 BSP/QEMU: master branchqemuarm64100.0%none0.0%none0.0%none100.0%
6855 6903 BSP/QEMU: master branchqemumips100.0%10025 0.0%none0.0%none100.0%
6856 6904 BSP/QEMU: master branchqemumips64100.0%10025 0.0%none0.0%none100.0%
6857 6905 BSP/QEMU: master branchqemuppc100.0%none0.0%none0.0%none100.0%
6858 6906 BSP/QEMU: master branchqemux86-6495.1%none4.9%11119 11173 0.0%none100.0%
6866 BSP/QEMU: master branchqemux86-lsb100.0%none0.0%none0.0%none100.0%
6867 BSP/QEMU: master branchqemux86_64-lsb97.3%none2.7%11173 0.0%none100.0%
6918 Automated Build Testing: master branchgeneric-environment100.0%11154 11172 0.0%none0.0%none100.0%
6884 Runtime: master branchMultiple Environments0.0%none11.1%11224 88.9%none100.0%
Total: 45


Bug Status


Test Case Status Report
StatusCount
BLOCKED43
FAILED34
PASSED2765
RUNNING269
Total: 3111


Bugs Found during QA Test

New issues Found during QA Test & Community

IDSummary (12 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BitBake
11172bitbake: Fetcher failure when building core-image-minimal with "ERROR: socat: not found"HighRichard Purdie 2.3 M3VERIFIEDFIXED---2017-03-16
BSPs
11173testimage: "test_dnf_install_from_http" is failingUndecidedAlexander KanavinJuan Ramos---RESOLVEDFIXED---2017-03-16
11220runqemu: Qemux86 not boot with UbuntuMediumbrian averyoscar lopez arandas2.3 M4VERIFIEDFIXED---2017-03-22
Build Testing
11154buildperf: Fedora machine failing on buildperf execution due could not find 'ip' toolHighLeonardo Sandoval Gonzalez 2.3 M4VERIFIEDFIXED---2017-03-14
Eclipse Plugin
11222[eclipse] C compiler cannot create executables when reconfiguring projectMedium+brian averyoscar lopez arandas2.4 M3VERIFIEDINVALID---2017-03-23
Meta-yocto
11174Update yocto-bsp script for kernel 4.10Medium+Leonardo Sandoval GonzalezYi Zhao2.3 M4VERIFIEDFIXEDRegression (Used to work)2017-03-17
OE-Core
11144Build Appliance : Toaster requires pod2manMedium+Juro Bystricky 2.3 M4VERIFIEDFIXEDRegression (Used to work)2017-03-13
11185selftest/devtool: failing when parsing “mraa” recipeMedium+Paul Eggleton 2.3 M4VERIFIEDFIXED---2017-03-17
11193runqemu can not launch image if enable "rm_work"HighChen Qi 2.3 M4RESOLVEDDUPLICATERegression (Used to work)2017-03-21
11223devtool runqemu doesn't work in eSDKHighbrian averyYi Zhao2.3 M4VERIFIEDFIXEDRegression (Used to work)2017-03-23
Runtime Testing
11224Compliance: LTP, POSIX, LSB tests are not creating results file (.targz)Medium+dengke 2.4 M1RESOLVEDFIXED---2017-03-23
Toaster
11225Build from Command line fail when finish all the tasksMediumDavid ReynaDavid Lopez Barriba2.3 M4VERIFIEDFIXED---2017-03-23


High and M+ issues Found during QA Test (NOT NEW)

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
ADT
11119runqemu-extract-sdk is failing when extracting .tar.bz fileHighbrian avery 2.3 M4VERIFIEDFIXED---2017-03-02
Meta-yocto
11064[Yocto 2.2.1] kernel version mismatched when using yocto-bsp to create custom qemu bspMedium+Armin KusterYi Zhao2.2.2RESOLVEDFIXEDRegression (Used to work)2017-02-17
OE-Core
11025selftest/devtool: Build mdadm failing on Centos7 and Ubuntu 1404Medium+Paul EggletonJuan Ramos2.3 M3VERIFIEDFIXED---2017-02-07


Other Bugs found during QA Test (NOT NEW)

IDSummary (5 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18
Toaster
10281Configuration variables: I can set a value for standard shell environment variable http_proxyLowToaster default assigneeyeoh ee pengFutureNEW---2016-09-14
10846Toaster: [Test Case 1095] Filter doesn't show a dialogueMediumDavid ReynaJose Perez C2.4VERIFIEDDUPLICATE---2016-12-22
Regression (Used to work)
10025Missing 'Connection Manager' icon for qemumips and qemumips64MediumJussi KukkonenYi Zhao2.4RESOLVEDWORKSFORMERegression (Used to work)2016-07-26
General Runtime
10858Can not stop tcf-agent daemonMediumbrian averyYi Zhao2.4 M1VERIFIEDFIXEDRegression (Used to work)2016-12-26

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestone or release which are Medium+ or High and are not targeted for the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      
BitBake
10984Medium+[Test Case 1058] bitbake can use alternate download methods than gitRichard Purdieoscar lopez arandas2.3 M3CLOSED---2017-01-27
  1      

Bug Trend

An overview of this week's Yocto Bug Trend charts can be accessed on the following page

Build Performance Test

Detailed description on how performance test are run can be found on Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.3 M1. Performance execution has 2 stages, one executed form 1.6 to 2.1 in RO and another one started on 2.1 from GDC, there are difference between HW used in both location hence the numbers are different.

Below chars are showing the latest performance results taken from current milestone

Performance Charts

  • NOTE: During the 2.3 M2 QA cycle period, Fedora machine was under debugging activities to investigate more on bug 10884 hence only results for Ubuntu machine are displayed on the charts.
  • NOTE: Due bug 10285 performance results for "bitbake core-image-sato -c rootfs" measurements were failing on 2.2_M3.rc1, hence the numbers shown in the charts for that particular milestone are the ones reported on logs just before the failure.

Performance Charts

For Bitbake core-image-sato with rm_work

Ww11-2k17 - rmwork.png

Building the Virtual kernel

Ww11-2k17 - kernel.png

Building the rootfs

Ww11-2k17 - rootfs.png

Building core-image-sato

Ww11-2k17 - sato.png

eSDK Deploy time

Ww11-2k17 - esdk.png

pTest for genericx86-64 on NUC

open bugs

IDSummary (13 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
11221pTest: various quilt test cases are failingMedium+Jussi Kukkonen 2.3 M4RESOLVEDFIXED---2017-03-22
10841pTest : Various test cases are failing on "dbus-test" moduleMediumRoss Burton 2.5RESOLVEDDUPLICATE---2016-12-21
10842testrunner test timeout can lead to garbage outputMediumAníbal Limón 2.3 M4VERIFIEDFIXED---2016-12-21
10840pTest- openssl: test cases "test_bn" and "test_cms" are failingMedium+Alexander Kanavin 2.4 M1VERIFIEDFIXED---2016-12-21
Runtime Testing
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng2.99ACCEPTEDRegression (Used to work)2015-12-17
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 2.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09
8831pTest - busybox tests fail for 2.1_M1.rc1Mediumdengke FutureRESOLVEDDUPLICATE---2015-12-17
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03
9055pTest - acl (NUC) failed tested on NUC 5425WYK for YP 2.1_M2.rc1MediumRobert YangFrancisco Pedraza2.4RESOLVEDFIXEDRegression (Used to work)2016-02-04
9372Ptest for acl hang because of lack of two user bin and daemonMediumChen Qioscar lopez arandas2.4RESOLVEDFIXED---2016-04-01

results

Results

< Back
Ran on a NUC and compared with commit Ptest 62d7d4130202d8ede16abf9e7d779361ca70847e

FULL LOG


Failures
 % increased
 % decreased


Results
Package Name
Tests ran
Passed
Failed
Passrate
Last Passrate
acl 380 380 0 100.00% 93.20%
attr 143 143 0 100.00% 100.00%
bash 76 72 4 94.70% 94.70%
bluez5 25 25 0 100.00% 100.00%
busybox 486 478 9 98.40% 98.30%
bzip2 6 6 0 100.00% 100.00%
dbus-test 15 4 11 26.70% 26.70%
diffstat 440 440 0 100.00% 100.00%
diffutils 17 17 0 100.00% 100.00%
e2fsprogs 1 1 0 100.00% 0.00%
flex 114 114 0 100% --
gawk 298 297 1 99.70% 99.70%
gdbm 22 22 0 100.00% 100.00%
gdk-pixbuf 20 20 0 100.00% 100.00%
glib-2.0 284 277 7 97.5% 100.00%
kbd 7 7 0 100.00% 100.00%
libpcre 3 3 0 100.00% 100.00%
libxml2 85 82 3 96.50% 96.50%
lttng-tools 2057 1988 69 96.60% 99.80%
nettle 88 87 1 98.90% 98.90%
openssh 34 33 1 97.10% 100.00%
openssl 52 50 2 96.20% 100.00%
pango 12 12 0 100.00% 100.00%
perl 2286 2285 1 100.00% 100.00%
perl5 2286 2285 1 100.00% 100.00%
python 32230 32112 118 99.60% 99.60%
quilt 57 19 38 33.30% 100.00%
strace 335 334 5 98.20% 99.60%
tcl 148 147 1 99.30% 100.00%
util-linux 109 107 2 98.20% 98.50%
valgrind -- -- -- -- 0.00%
zlib 2 2 0 100.00% 100.00%

Detailed Report

Distro Testing

Core Build System

Bitbake

Status

Bugs

**No results**

Agenda

Meta-Yocyo

Status

Bugs

Agenda

OE-Core

Status

Test Case Status Report

Bugs

Bug Status Report

Agenda

Test Run Agenda

eSDK

Status

Test Case Status Report

Bugs

**No results**

Agenda

Test Run Agenda

BSP

genericx86: core-image-sato-sdk

Status

Test Case Status Report

Bugs

Bug Status Report

Agenda

Test Run Agenda