WW17 - 2017-04-21 - Full Test Cycle 2.3 rc2

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 Test type: Full Test Cycle
 Branch: master
 Build name: 2.3 rc2
 poky commit: 7a0e795373653886452a7a2992ced10080711c26
 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3.rc2/
 Distributions tested: 
 -Ubuntu 16.10
 -Ubuntu 16.04 
 -OpenSuse 42.2
 -Opensuse 13.2  
 -Debian 8
 -Centos 7
 -Fedora 24
 -Fedora 25  

Test Run Summary Report

AutBuild-Testing

Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
7097 7098 eSDK: master branchCentos 7.1100.0%none0.0%none0.0%none100.0%
7094 CROPS: master branchCentos 7.1100.0%none0.0%none0.0%none100.0%
7113 CROPS: master branchuser-preference (any supported environment)66.7%none33.3%11421 11431 0.0%none100.0%
7055 7056 ADT: master branchFedora 25 i68694.2%none5.8%11409 0.0%none100.0%
7057 7058 ADT: master branchUbuntu 16.04 x86_6494.2%none5.8%11409 0.0%none100.0%
7100 Kernel: master branchCentos 7100.0%none0.0%none0.0%none100.0%
7095 Eclipse-Plugin: master branchMars on Fedora 22 x86_64,28.6%none0.0%none71.4%11409 100.0%
7096 Eclipse-Plugin: master branchNeon on Fedora 22 x86_6428.6%none0.0%none71.4%11409 100.0%
7101 Meta-yocto: master branchUbuntu 14.04 x86_6495.8%none0.0%none4.2%11409 100.0%
7110 Build-Appliance: master branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
7111 7112 Toaster: master branchDebian 4.3.0-0.bpo.1-amd64100.0%none0.0%none0.0%none100.0%
7102 OE-Core: master branchCentOS 7.1 x86_6499.5%none0.5%11300 0.0%none100.0%
7103 OE-Core: master branchDebian 8 x86_6499.5%none0.5%11300 0.0%none100.0%
7105 OE-Core: master branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
7104 OE-Core: master branchFedora 25 x86_64100.0%none0.0%none0.0%none100.0%
7106 OE-Core: master branchOpenSuse 13.2 x86_64100.0%none0.0%none0.0%none100.0%
7107 OE-Core: master branchOpenSuse421 x86_64100.0%none0.0%none0.0%none100.0%
7108 OE-Core: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
7109 OE-Core: master branchUbuntu 16.10 x86-64100.0%none0.0%none0.0%none100.0%
7059 BitBake: master branchFedora 24 x86_6487.5%none12.5%11419 0.0%none100.0%
7088 BSP/QEMU: master branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
7062 BSP/QEMU: master branchBeaglebone-Black100.0%none0.0%none0.0%none100.0%
7063 7089 BSP/QEMU: master branchEdgeRouter100.0%none0.0%none0.0%none100.0%
7081 BSP/QEMU: master branchgenericx86 - on MMAX32bit97.8%none2.2%10455 0.0%none100.0%
7082 BSP/QEMU: master branchgenericx86-64 on MMAX6497.8%none2.2%10455 0.0%none100.0%
7083 BSP/QEMU: master branchgenericx86-64 on NUC97.8%none2.2%10455 0.0%none100.0%
7091 BSP/QEMU: master branchgenericx86-64-lsb on MMAX64100.0%none0.0%none0.0%none100.0%
7090 BSP/QEMU: master branchgenericx86-64-lsb on NUC100.0%none0.0%none0.0%none100.0%
7084 BSP/QEMU: master branchgenericx86-64-WIC on MMAX64100.0%none0.0%none0.0%none100.0%
7085 BSP/QEMU: master branchgenericx86-64-WIC on NUC100.0%none0.0%none0.0%none100.0%
7061 BSP/QEMU: master branchGenericx86-64_NUC100.0%none0.0%none0.0%none100.0%
7092 BSP/QEMU: master branchgenericx86-lsb on MMAX32bit100.0%none0.0%none0.0%none100.0%
7086 BSP/QEMU: master branchgenericx86-WIC on MMAX32100.0%none0.0%none0.0%none100.0%
7060 BSP/QEMU: master branchGenericx86_MMAX32100.0%none0.0%none0.0%none100.0%
7064 7087 BSP/QEMU: master branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
7065 7072 BSP/QEMU: master branchqemu-x86100.0%none0.0%none0.0%none100.0%
7066 7073 BSP/QEMU: master branchqemuarm98.1%10858 0.0%none1.9%11409 100.0%
7067 7074 BSP/QEMU: master branchqemuarm6498.1%none0.0%none1.9%11409 100.0%
7068 7075 BSP/QEMU: master branchqemumips98.1%10025 0.0%none1.9%11409 100.0%
7069 7076 BSP/QEMU: master branchqemumips6498.1%10025 0.0%none1.9%11409 100.0%
7070 7077 BSP/QEMU: master branchqemuppc98.1%none0.0%none1.9%11409 100.0%
7071 7078 BSP/QEMU: master branchqemux86-64100.0%none0.0%none0.0%none100.0%
7079 BSP/QEMU: master branchqemux86-lsb100.0%none0.0%none0.0%none100.0%
7080 BSP/QEMU: master branchqemux86_64-lsb100.0%none0.0%none0.0%none100.0%
7099 Runtime: master branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
Total: 45


Bug Status


Test Case Status Report
StatusCount
BLOCKED16
FAILED22
PASSED3037
RUNNING7
Total: 3082


Bugs Found during QA Test

New issues Found during QA Test & Community

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
CROPS
11421Crops-ESDK: devtool build v4l2loopback-driver, fails when trying to access /lib/modules/<laptop_kernel_version>MediumTodor MinchevLibertad2.3 M4RESOLVEDFIXED---2017-04-26
11431Crops-ESDK: devtool is unable to add recipe for nodejs-native recipeMediumAnuj Mittalnagarajagopal2.7IN PROGRESS DESIGN COMPLETE---2017-04-27
OE-Core
11409runqemu-extract-sdk doesn't work in toolchain for 2.3 rc2 buildMedium+Robert YangYi Zhao2.3 M4VERIFIEDFIXEDRegression (Used to work)2017-04-25


High and M+ issues Found during QA Test (NOT NEW)

eSDK
11300selftest: do_populate_sdk_ext task failing on eSDK test on GDC ABsMedium+Jose Perez Coscar lopez arandas2.4 M1VERIFIEDFIXED---2017-04-05


Other Bugs found during QA Test (NOT NEW)

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18
Regression (Used to work)
10025Missing 'Connection Manager' icon for qemumips and qemumips64MediumJussi KukkonenYi Zhao2.4RESOLVEDWORKSFORMERegression (Used to work)2016-07-26
General Runtime
10858Can not stop tcf-agent daemonMediumbrian averyYi Zhao2.4 M1VERIFIEDFIXEDRegression (Used to work)2016-12-26

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestone or release which are Medium+ or High and are not targeted for the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      
BitBake
10984Medium+[Test Case 1058] bitbake can use alternate download methods than gitRichard Purdieoscar lopez arandas2.3 M3CLOSED---2017-01-27
  1      

Bug Trend

An overview of this week's Yocto Bug Trend charts can be accessed on the following page

Build Performance Test

Detailed description on how performance test is run can be found on the Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.3 M3. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and another one started on 2.1 from GDC. There are differences between HW used in both locations, hence the numbers are different.

Below chars are showing the latest performance results taken from the current milestone.

Performance Charts

NOTE: During the 2.3 M2 QA cycle period, the Fedora machine was under debugging activities to investigate more on bug 10884, hence only results for the Ubuntu machine are displayed on the charts.

Performance Charts

For Bitbake core-image-sato with rm_work

Ww17-2k17 - rmwork.png

Building the Virtual kernel

Ww17-2k17 - kernel.png

Building the rootfs

Ww17-2k17 - rootfs.png

Building core-image-sato

Ww17-2k17 - sato.png

eSDK Deploy time

Ww17-2k17 - esdk.png

pTest for genericx86-64 on NUC

open bugs

IDSummary (12 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
10841pTest : Various test cases are failing on "dbus-test" moduleMediumRoss Burton 2.5RESOLVEDDUPLICATE---2016-12-21
10842testrunner test timeout can lead to garbage outputMediumAníbal Limón 2.3 M4VERIFIEDFIXED---2016-12-21
10840pTest- openssl: test cases "test_bn" and "test_cms" are failingMedium+Alexander Kanavin 2.4 M1VERIFIEDFIXED---2016-12-21
Runtime Testing
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng2.99ACCEPTEDRegression (Used to work)2015-12-17
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 2.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09
8831pTest - busybox tests fail for 2.1_M1.rc1Mediumdengke FutureRESOLVEDDUPLICATE---2015-12-17
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03
9055pTest - acl (NUC) failed tested on NUC 5425WYK for YP 2.1_M2.rc1MediumRobert YangFrancisco Pedraza2.4RESOLVEDFIXEDRegression (Used to work)2016-02-04
9372Ptest for acl hang because of lack of two user bin and daemonMediumChen Qioscar lopez arandas2.4RESOLVEDFIXED---2016-04-01

results

< Back
Ran on a NUC and compared with commit Ptest 415b72ffcbd26e5f3664370d8b2a9b8105fb6342

FULL LOG


Failures
 % increased
 % decreased


Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.00% 100.00%
attr 143 143 0 100.00% 100.00%
bash 76 72 4 94.70% 94.70%
bluez5 25 25 0 100.00% 100.00%
busybox 486 478 8 98.40% 98.40%
bzip2 6 6 0 100.00% 100.00%
dbus-test 15 4 11 26.70% 26.70%
diffstat 440 440 0 100.00% 100.00%
diffutils 17 17 0 100.00% 100.00%
e2fsprogs 1 1 0 100.00% 100.00%
flex 114 114 0 100.00% 100.00%
gawk 298 297 1 99.70% 99.70%
gdbm 22 22 0 100.00% 100.00%
gdk-pixbuf 20 20 0 100.00% 100.00%
glib-2.0 284 277 7 97.50% 97.50%
kbd 7 7 0 100.00% 100.00%
libpcre 3 3 0 100.00% 100.00%
libxml2 85 82 3 96.50% 96.50%
lttng-tools 2057 1988 69 96.60% 96.60%
nettle 88 87 1 98.90% 98.90%
openssh 34 33 1 97.10% 97.10%
openssl 52 52 0 100.00% 96.20%
pango 12 12 0 100.00% 100.00%
perl 2286 2285 1 100.00% 100.00%
perl5 2286 2285 1 100.00% 100.00%
python 32230 32212 118 99.60% 99.60%
quilt 57 57 0 100.00 33.30%
strace 335 334 1 99.70% 99.70%
tcl 148 146 2 98.60% 98.60%
util-linux 108 106 2 98.10% 98.10%
valgrind 553 296 257 53.50% 53.50%
zlib 2 2 0 100.00% 100.00%

Detailed Report

Distro Testing

Core Build System

Bitbake

Status

Bugs

Agenda

Meta-Yocyo

Status

Bugs

Agenda

OE-Core

Status

Test Case Status Report

Bugs

Bug Status Report

Agenda

Test Run Agenda

eSDK

Status

Test Case Status Report

Bugs

**No results**

Agenda

Test Run Agenda

BSP

genericx86: core-image-sato-sdk

Status

Test Case Status Report

Bugs

Bug Status Report

Agenda

Test Run Agenda