WW05 - 2017-01-24 - Full Test Cycle 2.3 M2 rc1

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 Test type:  Full Test Cycle
 Branch: master
 Build name: 2.3 M2 rc1
 poky commit: 62d7d4130202d8ede16abf9e7d779361ca70847e
 Autobuilder images repository:  http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3_M2.rc1/
 Distributions tested: 
 -Ubuntu 16.04
 -Ubuntu 14 04 
 -OpenSuse 421
 -Opensuse 13.2  
 -Debian 8
 -Centos 7
 -Fedora 23
 -Fedora 24 

Test Run Summary Report

NOTE: There were missing Test Runs due the lack of resources for the execution.

Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
6711 6712 eSDK: master branchCentos 7.1100.0%none0.0%none0.0%none100.0%
6697 CROPS: master branchuser-preference (any supported environment)91.7%11000 8.3%11040 0.0%none100.0%
6713 Kernel: master branchCentOS 7.350.0%none0.0%none50.0%none100.0%
6715 Eclipse-Plugin: master branchNeon on Debian 4.3 x86_64100.0%none0.0%none0.0%none100.0%
6682 Meta-yocto: master branchuser-preference (any supported environment)88.9%10970 11.1%10990 11007 0.0%none100.0%
6677 6678 Toaster: master branchDebian 4.3.0-0.bpo.1-amd6488.9%none0.0%none11.1%10846 100.0%
6679 OE-Core: master branchCentOS 7.1 x86_64100.0%none0.0%none0.0%none100.0%
6693 OE-Core: master branchDebian 8 x86_64100.0%none0.0%none0.0%none100.0%
6691 OE-Core: master branchFedora 23 x86_64100.0%none0.0%none0.0%none100.0%
6692 OE-Core: master branchFedora 24 x86_64100.0%10968 0.0%none0.0%none100.0%
6680 OE-Core: master branchOpenSuse 13.2 x86_64100.0%none0.0%none0.0%none100.0%
6690 OE-Core: master branchOpenSuse421 x86_64100.0%none0.0%none0.0%none100.0%
6694 OE-Core: master branchubuntu 14.04 GDC100.0%10968 0.0%none0.0%none100.0%
6681 OE-Core: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
6695 OE-Core: master branchUbuntu 16.10 x86-64100.0%10968 0.0%none0.0%none100.0%
6687 BitBake: master branchFedora 24 x86_6475.0%none25.0%10983 10984 0.0%none100.0%
6696 6704 BSP/QEMU: master branchgenericx86 - on MMAX32bit43.9%none0.0%none56.1%10964 100.0%
6703 BSP/QEMU: master branchgenericx86-64 on MMAX640.0%none0.0%none100.0%10964 100.0%
6684 6702 BSP/QEMU: master branchgenericx86-64 on NUC43.3%none0.0%none56.7%10964 100.0%
6709 BSP/QEMU: master branchgenericx86-64-lsb on MMAX640.0%none0.0%none100.0%10964 100.0%
6708 BSP/QEMU: master branchgenericx86-64-lsb on NUC0.0%none0.0%none100.0%10964 100.0%
6706 BSP/QEMU: master branchgenericx86-64-WIC on MMAX640.0%none0.0%none100.0%10964 100.0%
6705 BSP/QEMU: master branchgenericx86-64-WIC on NUC0.0%none0.0%none100.0%10964 100.0%
6710 BSP/QEMU: master branchgenericx86-lsb on MMAX32bit0.0%none0.0%none100.0%10964 100.0%
6707 BSP/QEMU: master branchgenericx86-WIC on MMAX320.0%none0.0%none100.0%10964 100.0%
6685 6689 BSP/QEMU: master branchqemu-x860.0%none0.9%10964 99.1%none100.0%
6686 6688 BSP/QEMU: master branchqemux86-640.0%none0.0%none100.0%10964 100.0%
Total: 27


Bug Status


Test Case Status Report
StatusCount
BLOCKED737
FAILED6
PASSED1906
RUNNING6
Total: 2655


Bugs Found during QA Test

New issues Found during QA Test & Community

IDSummary (5 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BitBake
10983[Test Case 810] Avoid cache reload for incremental development using a backgrounded bitbake serverMedium+Richard Purdie 2.3 M4RESOLVEDFIXED---2017-01-27
BSPs
10964testimage: Test suite is not running when images are downloaded from AutobuilderHighMariano LopezJose Perez C2.3 M2VERIFIEDFIXED---2017-01-25
Meta-yocto
10970pybootchartgui.py has not been migrated to python3MediumLeonardo Sandoval Gonzalez 2.3 M3RESOLVEDWONTFIX---2017-01-25
OE-Core
10968selftest: devtool and recipetool fails on load plugin test.Medium+Jose Perez C 2.3 M3VERIFIEDNOTABUG---2017-01-25
10990[crosstap] systemtap error on core-image-minimal with tools-profileMedium+Saul Wold 2.3 M3VERIFIEDFIXED---2017-01-30


High and M+ issues Found during QA Test (NOT NEW)

"None"


Other Bugs found during QA Test (NOT NEW)

---
10846Toaster: [Test Case 1095] Filter doesn't show a dialogueMediumDavid ReynaJose Perez C2.4VERIFIEDDUPLICATE---2016-12-22

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestone or release which are Medium+ or High and are not targeted for the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of this week's Yocto Bug Trend charts can be accessed on the following page

Build Performance Test

Detailed description on how performance test are run can be found on Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.3 M1. Performance execution has 2 stages, one executed form 1.6 to 2.1 in RO and another one started on 2.1 from GDC, there are difference between HW used in both location hence the numbers are different.

Below chars are showing the latest performance results taken from current milestone

Performance Charts

NOTE: During the 2.3 M2 QA cycle period, Fedora machine was under debugging activities to investigate more on bug 10884 hence only results for Ubuntu machine are displayed on the charts.

Performance Charts

For Bitbake core-image-sato with rm_work

Ww05-2k17 - rmwork.png

Building the Virtual kernel

Ww05-2k17 - kernel.png

Building the rootfs

Ww05-2k17 - rootfs.png

Building core-image-sato

Ww05-2k17 - sato.png

eSDK Deploy time

Ww05-2k17 - esdk.png


pTest for genericx86-64 on NUC

open bugs

IDSummary (12 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
5695python: ptest cases failedMediumRoss Burton 2.6RESOLVEDFIXED---2013-12-31
5697busybox: ptest cases failedMediumChangqing Li 2.7 M2RESOLVEDFIXED---2013-12-31
10841pTest : Various test cases are failing on "dbus-test" moduleMediumRoss Burton 2.5RESOLVEDDUPLICATE---2016-12-21
10842testrunner test timeout can lead to garbage outputMediumAníbal Limón 2.3 M4VERIFIEDFIXED---2016-12-21
10840pTest- openssl: test cases "test_bn" and "test_cms" are failingMedium+Alexander Kanavin 2.4 M1VERIFIEDFIXED---2016-12-21
Runtime Testing
8828pTest - lttng-tools tests failMediumRichard Purdieyeoh ee peng2.99ACCEPTEDRegression (Used to work)2015-12-17
8471ptest - some valgrind ptests fail on 2.4 M1.rc1MediumMaxin B. John 2.99RESOLVEDDUPLICATERegression (Used to work)2015-10-09
8831pTest - busybox tests fail for 2.1_M1.rc1Mediumdengke FutureRESOLVEDDUPLICATE---2015-12-17
8832pTest - bash tests fail for 2.1_M1.rc1MediumChangqing Li 2.7 M2RESOLVEDWORKSFORME---2015-12-17
9042pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1MediumChangqing Li 2.7 M2RESOLVEDFIXEDRegression (Used to work)2016-02-03
9055pTest - acl (NUC) failed tested on NUC 5425WYK for YP 2.1_M2.rc1MediumRobert YangFrancisco Pedraza2.4RESOLVEDFIXEDRegression (Used to work)2016-02-04
9372Ptest for acl hang because of lack of two user bin and daemonMediumChen Qioscar lopez arandas2.4RESOLVEDFIXED---2016-04-01

results

Results

< Back
Ran on a NUC and compared with commit 573c646d4cc62dcd0c230381df4940bdf314d495

FULL LOG


Failures
 % increased
 % decreased


Results
Package Name
Tests ran
Passed
Failed
Passrate
Last Passrate
acl 380 354 26 93.20% 93.20%
attr 143 143 0 100.00% 100.00%
bash 76 72 4 94.70% 94.70%
bluez5 25 25 0 100.00% 100.00%
busybox 518 509 9 98.30% 98.60%
bzip2 6 6 0 100.00% 100.00%
dbus-test 15 4 11 26.70% 26.70%
diffstat 440 440 0 100.00% 100.00%
diffutils 17 17 0 100.00% 100.00%
e2fsprogs 1 1 0 100.00% 0.00%
gawk 298 297 1 99.70% 99.70%
gdbm 22 22 0 100.00% 100.00%
gdk-pixbuf 18 18 0 100.00% 100.00%
glib-2.0 250 250 0 100.00% 97.90%
kbd 7 7 0 100.00% 100.00%
libpcre 3 3 0 100.00% 100.00%
libxml2 85 82 3 96.50% 96.50%
lttng-tools 2057 1988 69 96.60% 99.80%
nettle 88 87 1 98.90% 98.90%
openssh 56 56 0 100.00% 100.00%
openssl 52 52 0 100.00% 96.20%
pango 12 12 0 100.00% 100.00%
perl 2286 2285 1 100.00% 100.00%
perl5 2286 2285 1 100.00% 100.00%
python 32058 31932 126 99.60% 99.60%
quilt 57 57 0 100.00% 98.20%
strace 278 273 5 98.20% 99.60%
tcl 148 147 1 99.30% 100.00%
util-linux 109 107 2 98.20% 98.50%
valgrind 553 2 551 0.40% 0.00%
zlib 2 2 0 100.00% 100.00%

Detailed Report

Distro Testing

Status

Bugs

Agenda

Core Build System

Bitbake

Status

Bugs

Agenda

Meta-Yocyo

Status

Bugs

Agenda

OE-Core

Status

Test Case Status Report

Bugs

Bug Status Report

Agenda

Test Run Agenda

eSDK

Status

Test Case Status Report

Bugs

**No results**

Agenda

Test Run Agenda