WW14 - 2017-03-28 - Full Test Cycle 2.3 M3 rc2

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 Test type: Partial Test Cycle
 Branch: master
 Build name: 2.3 M3 rc2
 poky commit: 415b72ffcbd26e5f3664370d8b2a9b8105fb6342
 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3_M3.rc2/
 Distributions tested: 
 -Ubuntu 16.04
 -Ubuntu 14 04 
 -OpenSuse 421
 -Opensuse 13.2  
 -Debian 8
 -Centos 7
 -Fedora 23
 -Fedora 24  

Test Run Summary Report

AutBuild-Testing

Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
6961 6962 eSDK: master branchCentos 7.1100.0%none0.0%none0.0%none100.0%
6958 CROPS: master branchCentos 7.1100.0%none0.0%none0.0%none100.0%
6919 6920 ADT: master branchFedora 25 i68699.1%11193 0.9%11223 0.0%none100.0%
6921 6922 ADT: master branchUbuntu 16.04 x86_6499.1%11193 0.9%11223 0.0%none100.0%
6964 Kernel: master branchCentos 7100.0%none0.0%none0.0%none100.0%
6959 Eclipse-Plugin: master branchMars on Fedora 22 x86_64,42.9%none0.0%none57.1%11222 100.0%
6960 Eclipse-Plugin: master branchNeon on Fedora 22 x86_6442.9%none0.0%none57.1%11222 100.0%
6965 Meta-yocto: master branchUbuntu 14.04 x86_64100.0%10477 0.0%none0.0%none100.0%
6974 Build-Appliance: master branchVMWare Player on Ubuntu66.7%none33.3%11266 0.0%none100.0%
6975 6976 Toaster: master branchDebian 4.3.0-0.bpo.1-amd6495.2%none1.6%10846 3.2%10846 100.0%
6966 OE-Core: master branchCentOS 7.1 x86_6499.5%none0.0%none0.5%11284 100.0%
6967 OE-Core: master branchDebian 8 x86_6499.5%none0.0%none0.5%11284 100.0%
6969 OE-Core: master branchFedora 24 x86_6499.5%none0.0%none0.5%11284 100.0%
6968 OE-Core: master branchFedora 25 x86_6499.5%none0.0%none0.5%11284 100.0%
6970 OE-Core: master branchOpenSuse 13.2 x86_6499.5%none0.0%none0.5%11284 100.0%
6971 OE-Core: master branchOpenSuse421 x86_6499.5%none0.0%none0.5%11284 100.0%
6972 OE-Core: master branchUbuntu 16.04 x86_6499.5%none0.0%none0.5%11284 100.0%
6973 OE-Core: master branchUbuntu 16.10 x86-6499.5%none0.0%none0.5%11284 100.0%
6923 BitBake: master branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
6952 BSP/QEMU: master branchBeaglebone Black97.1%none2.9%11267 0.0%none100.0%
6926 BSP/QEMU: master branchBeaglebone-Black77.8%none5.6%11267 16.7%11267 100.0%
6927 6953 BSP/QEMU: master branchEdgeRouter100.0%none0.0%none0.0%none100.0%
6945 BSP/QEMU: master branchgenericx86 - on MMAX32bit97.1%none2.9%10455 0.0%none100.0%
6946 BSP/QEMU: master branchgenericx86-64 on MMAX6497.1%none2.9%10455 0.0%none100.0%
6947 BSP/QEMU: master branchgenericx86-64 on NUC97.1%none2.9%10455 0.0%none100.0%
6955 BSP/QEMU: master branchgenericx86-64-lsb on MMAX64100.0%none0.0%none0.0%none100.0%
6954 BSP/QEMU: master branchgenericx86-64-lsb on NUC100.0%none0.0%none0.0%none100.0%
6948 BSP/QEMU: master branchgenericx86-64-WIC on MMAX64100.0%none0.0%none0.0%none100.0%
6949 BSP/QEMU: master branchgenericx86-64-WIC on NUC100.0%none0.0%none0.0%none100.0%
6925 BSP/QEMU: master branchGenericx86-64_NUC100.0%none0.0%none0.0%none100.0%
6956 BSP/QEMU: master branchgenericx86-lsb on MMAX32bit100.0%none0.0%none0.0%none100.0%
6950 BSP/QEMU: master branchgenericx86-WIC on MMAX32100.0%none0.0%none0.0%none100.0%
6924 BSP/QEMU: master branchGenericx86_MMAX32100.0%none0.0%none0.0%none100.0%
6928 6951 BSP/QEMU: master branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
6929 6936 BSP/QEMU: master branchqemu-x8697.5%none2.5%11119 0.0%none100.0%
6930 6937 BSP/QEMU: master branchqemuarm100.0%10858 0.0%none0.0%none100.0%
6931 6938 BSP/QEMU: master branchqemuarm64100.0%none0.0%none0.0%none100.0%
6932 6939 BSP/QEMU: master branchqemumips100.0%10025 0.0%none0.0%none100.0%
6933 6940 BSP/QEMU: master branchqemumips64100.0%10025 0.0%none0.0%none100.0%
6934 6941 BSP/QEMU: master branchqemuppc100.0%none0.0%none0.0%none100.0%
6963 Runtime: master branchMultiple Environments100.0%none0.0%none0.0%none100.0%
Total: 41


Bug Status


Test Case Status Report
StatusCount
BLOCKED21
FAILED10
PASSED2627
RUNNING215
Total: 2873


Bugs Found during QA Test

New issues Found during QA Test & Community

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
AutoBuilder
11284selftest: tests that capture output fail on GDC AutobuildersHighJoshua LockJose Perez C2.3 M4VERIFIEDFIXEDRegression (Found by QA test)2017-04-01
BSPs
11267X can not launch on beagleboneHighKevin HaoYi Zhao2.3 M4VERIFIEDFIXEDRegression (Used to work)2017-03-30
OE-Core
11266runqemu: fails when rm_work is activeMedium+Chen Qi 2.3 M4VERIFIEDFIXED---2017-03-29


High and M+ issues Found during QA Test (NOT NEW)

IDSummary (5 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
ADT
11119runqemu-extract-sdk is failing when extracting .tar.bz fileHighbrian avery 2.3 M4VERIFIEDFIXED---2017-03-02
Eclipse Plugin
11222[eclipse] C compiler cannot create executables when reconfiguring projectMedium+brian averyoscar lopez arandas2.4 M3VERIFIEDINVALID---2017-03-23
Meta-yocto
10477yocto-bsp: tool should output a conf file to be consume by the qemu runnerMediumLeonardo Sandoval Gonzalez 2.5RESOLVEDOBSOLETE---2016-10-21
OE-Core
11193runqemu can not launch image if enable "rm_work"HighChen Qi 2.3 M4RESOLVEDDUPLICATERegression (Used to work)2017-03-21
11223devtool runqemu doesn't work in eSDKHighbrian averyYi Zhao2.3 M4VERIFIEDFIXEDRegression (Used to work)2017-03-23


Other Bugs found during QA Test (NOT NEW)

IDSummary (5 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18
BSPs
11173testimage: "test_dnf_install_from_http" is failingUndecidedAlexander KanavinJuan Ramos---RESOLVEDFIXED---2017-03-16
Toaster
10846Toaster: [Test Case 1095] Filter doesn't show a dialogueMediumDavid ReynaJose Perez C2.4VERIFIEDDUPLICATE---2016-12-22
Regression (Used to work)
10025Missing 'Connection Manager' icon for qemumips and qemumips64MediumJussi KukkonenYi Zhao2.4RESOLVEDWORKSFORMERegression (Used to work)2016-07-26
General Runtime
10858Can not stop tcf-agent daemonMediumbrian averyYi Zhao2.4 M1VERIFIEDFIXEDRegression (Used to work)2016-12-26

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestone or release which are Medium+ or High and are not targeted for the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      
BitBake
10984Medium+[Test Case 1058] bitbake can use alternate download methods than gitRichard Purdieoscar lopez arandas2.3 M3CLOSED---2017-01-27
  1      

Bug Trend

An overview of this week's Yocto Bug Trend charts can be accessed on the following page

Build Performance Test

Detailed description on how performance test is run can be found on the Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.3 M3. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and another one started on 2.1 from GDC. There are differences between HW used in both locations, hence the numbers are different.

Below chars are showing the latest performance results taken from the current milestone.

Performance Charts

NOTE: During the 2.3 M2 QA cycle period, the Fedora machine was under debugging activities to investigate more on bug 10884, hence only results for the Ubuntu machine are displayed on the charts.

Performance Charts

For Bitbake core-image-sato with rm_work

Ww14-2k17 - rmwork.png

Building the Virtual kernel

Ww14-2k17 - kernel.png

Building the rootfs

Ww14-2k17 - rootfs.png

Building core-image-sato

Ww14-2k17 - sato.png

eSDK Deploy time

Ww14-2k17 - esdk.png

pTest for genericx86-64 on NUC

open bugs

IDSummary (12 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
10841pTest : Various test cases are failing on "dbus-test" moduleMediumRoss Burton 2.5RESOLVEDDUPLICATE---2016-12-21
10842testrunner test timeout can lead to garbage outputMediumAníbal Limón 2.3 M4VERIFIEDFIXED---2016-12-21
10840pTest- openssl: test cases "test_bn" and "test_cms" are failingMedium+Alexander Kanavin 2.4 M1VERIFIEDFIXED---2016-12-21
Runtime Testing
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng2.99ACCEPTEDRegression (Used to work)2015-12-17
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 2.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09
8831pTest - busybox tests fail for 2.1_M1.rc1Mediumdengke FutureRESOLVEDDUPLICATE---2015-12-17
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03
9055pTest - acl (NUC) failed tested on NUC 5425WYK for YP 2.1_M2.rc1MediumRobert YangFrancisco Pedraza2.4RESOLVEDFIXEDRegression (Used to work)2016-02-04
9372Ptest for acl hang because of lack of two user bin and daemonMediumChen Qioscar lopez arandas2.4RESOLVEDFIXED---2016-04-01

results

< Back
Ran on a NUC and compared with commit Ptest 7a0e795373653886452a7a2992ced10080711c26

FULL LOG


Failures
 % increased
 % decreased


Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.00% 100.00%
attr 143 143 0 100.00% 100.00%
bash 76 72 4 94.70% 94.70%
bluez5 25 25 0 100.00% 100.00%
busybox 486 478 8 98.40% 98.40%
bzip2 6 6 0 100.00% 100.00%
dbus-test 15 4 11 26.70% 26.70%
diffstat 440 440 0 100.00% 100.00%
diffutils 17 17 0 100.00% 100.00%
e2fsprogs 1 1 0 100.00% 100.00%
flex 114 114 0 100.00% 100.00%
gawk 298 297 1 99.70% 99.70%
gdbm 22 22 0 100.00% 100.00%
gdk-pixbuf 20 20 0 100.00% 100.00%
glib-2.0 284 277 7 97.50% 97.50%
kbd 7 7 0 100.00% 100.00%
libpcre 3 3 0 100.00% 100.00%
libxml2 85 82 3 96.50% 96.50%
lttng-tools 2057 1988 69 96.60% 96.60%
nettle 88 87 1 98.90% 98.90%
openssh 34 33 1 97.10% 97.10%
openssl 52 52 0 100.00% 100.00%
pango 12 12 0 100.00% 100.00%
perl 2286 2285 1 100.00% 100.00%
perl5 2286 2285 1 100.00% 100.00%
python 32230 32212 118 99.60% 99.60%
quilt 57 57 0 100.00 100.00%
strace 335 334 1 99.70% 99.70%
tcl 148 146 2 100.00% 98.60%
util-linux 108 106 2 98.10% 98.10%
valgrind 553 296 257 53.50% 53.50%
zlib 2 2 0 100.00% 100.00%

Detailed Report

Distro Testing

Core Build System

Bitbake

Status

Bugs

**No results**

Agenda

Meta-Yocyo

Status

Bugs

Agenda

OE-Core

Status

Test Case Status Report

Bugs

Bug Status Report

Agenda

Test Run Agenda

eSDK

Status

Test Case Status Report

Bugs

**No results**

Agenda

Test Run Agenda

BSP

genericx86: core-image-sato-sdk

Status

Test Case Status Report

Bugs

Bug Status Report

Agenda

Test Run Agenda