WW19 - 2017-05-02 - Full Test Cycle 2.3 rc4

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 Test type: Full Test Cycle
 Branch: master
 Build name: 2.3 rc4
 poky commit: 381897c64069ea43d595380a3ae913bcc79cf7e1
 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3.rc4/
 Distributions tested: 
 -Ubuntu 16.10
 -Ubuntu 16.04 
 -OpenSuse 42.2
 -Opensuse 13.2  
 -Debian 8
 -Centos 7
 -Fedora 24
 -Fedora 25  

Test Run Summary Report

AutBuild-Testing

Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
7156 7157 eSDK: master branchCentos 7.1100.0%none0.0%none0.0%none100.0%
7153 CROPS: master branchCentos 7.1100.0%none0.0%none0.0%none100.0%
7172 CROPS: master branchuser-preference (any supported environment)50.0%11471 50.0%11471 11421 11431 0.0%none100.0%
7114 7115 ADT: master branchFedora 25 i686100.0%none0.0%none0.0%none100.0%
7116 7117 ADT: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
7159 Kernel: master branchCentos 7100.0%none0.0%none0.0%none100.0%
7154 Eclipse-Plugin: master branchMars on Debian 4.3 x86_6485.7%none0.0%none14.3%11471 100.0%
7155 Eclipse-Plugin: master branchNeon on Fedora 22 x86_6485.7%none0.0%none14.3%11471 100.0%
7160 Meta-yocto: master branchUbuntu 14.04 x86_64100.0%10477 0.0%none0.0%none100.0%
7169 Build-Appliance: master branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
7170 7171 Toaster: master branchDebian 4.3.0-0.bpo.1-amd6499.0%none0.0%none1.0%10281 100.0%
7161 OE-Core: master branchCentOS 7.1 x86_64100.0%none0.0%none0.0%none100.0%
7162 OE-Core: master branchDebian 8 x86_64100.0%none0.0%none0.0%none100.0%
7164 OE-Core: master branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
7163 OE-Core: master branchFedora 25 x86_64100.0%none0.0%none0.0%none100.0%
7165 OE-Core: master branchOpenSuse 13.2 x86_64100.0%none0.0%none0.0%none100.0%
7166 OE-Core: master branchOpenSuse421 x86_64100.0%none0.0%none0.0%none100.0%
7167 OE-Core: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
7168 OE-Core: master branchUbuntu 16.10 x86-64100.0%none0.0%none0.0%none100.0%
7118 BitBake: master branchFedora 24 x86_64100.0%none0.0%none0.0%none100.0%
7147 BSP/QEMU: master branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
7121 BSP/QEMU: master branchBeaglebone-Black100.0%none0.0%none0.0%none100.0%
7122 7148 BSP/QEMU: master branchEdgeRouter100.0%none0.0%none0.0%none100.0%
7140 BSP/QEMU: master branchgenericx86 - on MMAX32bit97.8%none2.2%10455 0.0%none100.0%
7141 BSP/QEMU: master branchgenericx86-64 on MMAX6497.8%none2.2%10455 0.0%none100.0%
7142 BSP/QEMU: master branchgenericx86-64 on NUC97.8%none2.2%10455 0.0%none100.0%
7150 BSP/QEMU: master branchgenericx86-64-lsb on MMAX64100.0%none0.0%none0.0%none100.0%
7149 BSP/QEMU: master branchgenericx86-64-lsb on NUC100.0%none0.0%none0.0%none100.0%
7143 BSP/QEMU: master branchgenericx86-64-WIC on MMAX64100.0%none0.0%none0.0%none100.0%
7144 BSP/QEMU: master branchgenericx86-64-WIC on NUC100.0%none0.0%none0.0%none100.0%
7120 BSP/QEMU: master branchGenericx86-64_NUC100.0%none0.0%none0.0%none100.0%
7151 BSP/QEMU: master branchgenericx86-lsb on MMAX32bit100.0%none0.0%none0.0%none100.0%
7145 BSP/QEMU: master branchgenericx86-WIC on MMAX32100.0%none0.0%none0.0%none100.0%
7119 BSP/QEMU: master branchGenericx86_MMAX32100.0%none0.0%none0.0%none100.0%
7123 7146 BSP/QEMU: master branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
7124 7131 BSP/QEMU: master branchqemu-x86100.0%none0.0%none0.0%none100.0%
7125 7132 BSP/QEMU: master branchqemuarm100.0%10858 0.0%none0.0%none100.0%
7126 7133 BSP/QEMU: master branchqemuarm64100.0%10858 0.0%none0.0%none100.0%
7127 7134 BSP/QEMU: master branchqemumips100.0%10025 0.0%none0.0%none100.0%
7128 7135 BSP/QEMU: master branchqemumips64100.0%10025 0.0%none0.0%none100.0%
7129 7136 BSP/QEMU: master branchqemuppc100.0%none0.0%none0.0%none100.0%
7130 7137 BSP/QEMU: master branchqemux86-64100.0%none0.0%none0.0%none100.0%
7138 BSP/QEMU: master branchqemux86-lsb100.0%none0.0%none0.0%none100.0%
7139 BSP/QEMU: master branchqemux86_64-lsb100.0%none0.0%none0.0%none100.0%
7158 Runtime: master branchMultiple Environments100.0%none0.0%none0.0%none100.0%
Total: 45


Bug Status


Test Case Status Report
StatusCount
BLOCKED3
FAILED6
PASSED3072
RUNNING5
Total: 3086


Bugs Found during QA Test

New issues Found during QA Test & Community

eSDK
11471eSDK toolchain cannot install using i586 extensible-toolchainUndecidedPaul EggletonFrancisco Pedraza2.3 M4RESOLVEDNOTABUG---2017-05-05


High and M+ issues Found during QA Test (NOT NEW)

IDSummary (2 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
CROPS
11431Crops-ESDK: devtool is unable to add recipe for nodejs-native recipeMediumAnuj Mittalnagarajagopal2.7IN PROGRESS DESIGN COMPLETE---2017-04-27
Meta-yocto
10477yocto-bsp: tool should output a conf file to be consume by the qemu runnerMediumLeonardo Sandoval Gonzalez 2.5RESOLVEDOBSOLETE---2016-10-21


Other Bugs found during QA Test (NOT NEW)

IDSummary (5 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18
CROPS
11421Crops-ESDK: devtool build v4l2loopback-driver, fails when trying to access /lib/modules/<laptop_kernel_version>MediumTodor MinchevLibertad2.3 M4RESOLVEDFIXED---2017-04-26
Toaster
10281Configuration variables: I can set a value for standard shell environment variable http_proxyLowToaster default assigneeyeoh ee pengFutureNEW---2016-09-14
Regression (Used to work)
10025Missing 'Connection Manager' icon for qemumips and qemumips64MediumJussi KukkonenYi Zhao2.4RESOLVEDWORKSFORMERegression (Used to work)2016-07-26
General Runtime
10858Can not stop tcf-agent daemonMediumbrian averyYi Zhao2.4 M1VERIFIEDFIXEDRegression (Used to work)2016-12-26

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestone or release which are Medium+ or High and are not targeted for the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      
BitBake
10984Medium+[Test Case 1058] bitbake can use alternate download methods than gitRichard Purdieoscar lopez arandas2.3 M3CLOSED---2017-01-27
  1      

Bug Trend

An overview of this week's Yocto Bug Trend charts can be accessed on the following page

Build Performance Test

Detailed description on how performance test is run can be found on the Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.3 M3. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and another one started on 2.1 from GDC. There are differences between HW used in both locations, hence the numbers are different.

Below chars are showing the latest performance results taken from the current milestone.

Performance Charts

NOTE: During the 2.3 M2 QA cycle period, the Fedora machine was under debugging activities to investigate more on bug 10884, hence only results for the Ubuntu machine are displayed on the charts.

Performance Charts

For Bitbake core-image-sato with rm_work

Ww19-2k17 - rmwork.png

Building the Virtual kernel

Ww19-2k17 - kernel.png

Building the rootfs

Ww19-2k17 - rootfs.png

Building core-image-sato

Ww19-2k17 - sato.png

eSDK Deploy time

Ww19-2k17 - esdk.png

pTest for genericx86-64 on NUC

open bugs

IDSummary (12 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
10841pTest : Various test cases are failing on "dbus-test" moduleMediumRoss Burton 2.5RESOLVEDDUPLICATE---2016-12-21
10842testrunner test timeout can lead to garbage outputMediumAníbal Limón 2.3 M4VERIFIEDFIXED---2016-12-21
10840pTest- openssl: test cases "test_bn" and "test_cms" are failingMedium+Alexander Kanavin 2.4 M1VERIFIEDFIXED---2016-12-21
Runtime Testing
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng2.99ACCEPTEDRegression (Used to work)2015-12-17
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 2.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09
8831pTest - busybox tests fail for 2.1_M1.rc1Mediumdengke FutureRESOLVEDDUPLICATE---2015-12-17
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03
9055pTest - acl (NUC) failed tested on NUC 5425WYK for YP 2.1_M2.rc1MediumRobert YangFrancisco Pedraza2.4RESOLVEDFIXEDRegression (Used to work)2016-02-04
9372Ptest for acl hang because of lack of two user bin and daemonMediumChen Qioscar lopez arandas2.4RESOLVEDFIXED---2016-04-01

results

< Back
Ran on a NUC and compared with commit Ptest 7a0e795373653886452a7a2992ced10080711c26

FULL LOG


Failures
 % increased
 % decreased


Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.00% 100.00%
attr 143 143 0 100.00% 100.00%
bash 76 72 4 94.70% 94.70%
bluez5 25 25 0 100.00% 100.00%
busybox 486 478 8 98.40% 98.40%
bzip2 6 6 0 100.00% 100.00%
dbus-test 15 4 11 26.70% 26.70%
diffstat 440 440 0 100.00% 100.00%
diffutils 17 17 0 100.00% 100.00%
e2fsprogs 1 1 0 100.00% 100.00%
flex 114 114 0 100.00% 100.00%
gawk 298 297 1 99.70% 99.70%
gdbm 22 22 0 100.00% 100.00%
gdk-pixbuf 20 20 0 100.00% 100.00%
glib-2.0 284 277 7 97.50% 97.50%
kbd 7 7 0 100.00% 100.00%
libpcre 3 3 0 100.00% 100.00%
libxml2 85 82 3 96.50% 96.50%
lttng-tools 2057 1988 69 96.60% 96.60%
nettle 88 87 1 98.90% 98.90%
openssh 34 33 1 97.10% 97.10%
openssl 52 52 0 100.00% 100.00%
pango 12 12 0 100.00% 100.00%
perl 2286 2285 1 100.00% 100.00%
perl5 2286 2285 1 100.00% 100.00%
python 32230 32212 118 99.60% 99.60%
quilt 57 57 0 100.00 100.00%
strace 335 334 1 99.70% 99.70%
tcl 148 147 1 99.30% 98.60%
util-linux 108 106 2 98.10% 98.10%
valgrind 553 296 257 53.50% 53.50%
zlib 2 2 0 100.00% 100.00%

Detailed Report

Distro Testing

Core Build System

Bitbake

Status

Bugs

**No results**

Agenda

Meta-Yocyo

Status

Bugs

Agenda

OE-Core

Status

Test Case Status Report

Bugs

**No results**

Agenda

Test Run Agenda

eSDK

Status

Test Case Status Report

Bugs

**No results**

Agenda

Test Run Agenda

BSP

genericx86: core-image-sato-sdk

Status

Test Case Status Report

Bugs

Bug Status Report

Agenda

Test Run Agenda