{{#vardefine:adt_fedora_i686| }} {{#vardefine:adt_fedora_x86-64| }} {{#vardefine:adt_ubuntu_x86-64| }} {{#vardefine:beaglebone_black| }} {{#vardefine:edgerouter| }} {{#vardefine:genericx86| 6696, 6704, 6710, 6707}} {{#vardefine:genericx86_64| 6703, 6684, 6702, 6709, 6708, 6706, 6705}} {{#vardefine:mpc8315e-rdb| }} {{#vardefine:p1022ds-rdb| }} {{#vardefine:qemux86| 6685, 6689}} {{#vardefine:qemuarm| }} {{#vardefine:qemumips| }} {{#vardefine:qemuppc| }} {{#vardefine:qemux86-64| 6686, 6688}} {{#vardefine:eclipseplugin| 6715}} {{#vardefine:meta-yocto| 6682}} {{#vardefine:eSDK| 6711, 6712}} {{#vardefine:toaster| 6677, 6678}} {{#vardefine:oe-core| 6679, 6693, 6691, 6692, 6680, 6690, 6694, 6681, 6695}} {{#vardefine:distro| 6697}} {{#vardefine:bitbake| 6687}} {{#vardefine:runtime| }} {{#vardefine:build-appliance| }} {{#vardefine:kernel| 6713}}

Testing data

 Test type:  Full Test Cycle
 Branch: master
 Build name: 2.3 M2 rc1
 poky commit: 62d7d4130202d8ede16abf9e7d779361ca70847e
 Autobuilder images repository:  http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3_M2.rc1/
 Distributions tested: 
 -Ubuntu 16.04
 -Ubuntu 14 04 
 -OpenSuse 421
 -Opensuse 13.2  
 -Debian 8
 -Centos 7
 -Fedora 23
 -Fedora 24

Test Run Summary Report

NOTE: There were missing Test Runs due the lack of resources for the execution.

{{#testopia:

run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:kernel}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:adt_ubuntu_1310_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}}

report_id=concatstats}}

{{#testopia:

chart=bstatus

report_id=bugs

charttype=bar

sortable=true

nudechart=true

chartpos=top

}}

{{#testopia:

run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:kernel}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}}

charttype=bar

report_id=status}}

Bugs Found during QA Test

New issues Found during QA Test & Community

ID	Summary (5 tasks) ⇒	P	Assignee	QA	Milestone	Status	Resolution	cf_regression_type	Created
BitBake
10983	[Test Case 810] Avoid cache reload for incremental development using a backgrounded bitbake server	Medium+	Richard Purdie		2.3 M4	RESOLVED	FIXED	---	2017-01-27
BSPs
10964	testimage: Test suite is not running when images are downloaded from Autobuilder	High	Mariano Lopez	Jose Perez C	2.3 M2	VERIFIED	FIXED	---	2017-01-25
Meta-yocto
10970	pybootchartgui.py has not been migrated to python3	Medium	Leonardo Sandoval Gonzalez		2.3 M3	RESOLVED	WONTFIX	---	2017-01-25
OE-Core
10968	selftest: devtool and recipetool fails on load plugin test.	Medium+	Jose Perez C		2.3 M3	VERIFIED	NOTABUG	---	2017-01-25
10990	[crosstap] systemtap error on core-image-minimal with tools-profile	Medium+	Saul Wold		2.3 M3	VERIFIED	FIXED	---	2017-01-30

High and M+ issues Found during QA Test (NOT NEW)

"None"

Other Bugs found during QA Test (NOT NEW)

---
10846	Toaster: [Test Case 1095] Filter doesn't show a dialogue	Medium	David Reyna	Jose Perez C	2.4	VERIFIED	DUPLICATE	---	2016-12-22

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestone or release which are Medium+ or High and are not targeted for the next milestones

ID	P	Summary	Assignee	QA	Milestone	Status	cf_regression_type	Created
		0

Bug Trend

An overview of this week's Yocto Bug Trend charts can be accessed on the following page

Build Performance Test

Detailed description on how performance test are run can be found on Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.3 M1. Performance execution has 2 stages, one executed form 1.6 to 2.1 in RO and another one started on 2.1 from GDC, there are difference between HW used in both location hence the numbers are different.

RO Charts

GDC Charts

Below chars are showing the latest performance results taken from current milestone

Performance Charts

NOTE: During the 2.3 M2 QA cycle period, Fedora machine was under debugging activities to investigate more on bug 10884 hence only results for Ubuntu machine are displayed on the charts.

Performance Charts

For Bitbake core-image-sato with rm_work

Building the Virtual kernel

Building the rootfs

Building core-image-sato

eSDK Deploy time

pTest for genericx86-64 on NUC

open bugs

ID	Summary (12 tasks) ⇒	P	Assignee	QA	Milestone	Status	Resolution	cf_regression_type	Created
OE-Core
5697	busybox: ptest cases failed	Medium	Changqing Li		2.7 M2	RESOLVED	FIXED	---	2013-12-31
5695	python: ptest cases failed	Medium	Ross Burton		2.6	RESOLVED	FIXED	---	2013-12-31
10841	pTest : Various test cases are failing on "dbus-test" module	Medium	Ross Burton		2.5	RESOLVED	DUPLICATE	---	2016-12-21
10840	pTest- openssl: test cases "test_bn" and "test_cms" are failing	Medium+	Alexander Kanavin		2.4 M1	VERIFIED	FIXED	---	2016-12-21
10842	testrunner test timeout can lead to garbage output	Medium	Aníbal Limón		2.3 M4	VERIFIED	FIXED	---	2016-12-21
Runtime Testing
8831	pTest - busybox tests fail for 2.1_M1.rc1	Medium	dengke		Future	RESOLVED	DUPLICATE	---	2015-12-17
8828	pTest - lttng-tools tests fail	Medium	Richard Purdie	yeoh ee peng	4.99	RESOLVED	FIXED	Regression (Used to work)	2015-12-17
9372	Ptest for acl hang because of lack of two user bin and daemon	Medium	Chen Qi	oscar lopez arandas	2.4	RESOLVED	FIXED	---	2016-04-01
8471	ptest - some valgrind ptests fail on 2.4 M1.rc1	Medium	Maxin B. John		4.99	RESOLVED	DUPLICATE	Regression (Used to work)	2015-10-09
9055	pTest - acl (NUC) failed tested on NUC 5425WYK for YP 2.1_M2.rc1	Medium	Robert Yang	Francisco Pedraza	2.4	RESOLVED	FIXED	Regression (Used to work)	2016-02-04
9042	pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1	Medium	Changqing Li		2.7 M2	RESOLVED	FIXED	Regression (Used to work)	2016-02-03
8832	pTest - bash tests fail for 2.1_M1.rc1	Medium	Changqing Li		2.7 M2	RESOLVED	WORKSFORME	---	2015-12-17

results

Results

< Back
Ran on a NUC and compared with commit 573c646d4cc62dcd0c230381df4940bdf314d495

FULL LOG

Failures

% increased

% decreased

Results
Package Name	Tests ran	Passed	Failed	Passrate	Last Passrate
acl	380	354	26	93.20%	93.20%
attr	143	143	0	100.00%	100.00%
bash	76	72	4	94.70%	94.70%
bluez5	25	25	0	100.00%	100.00%
busybox	518	509	9	98.30%	98.60%
bzip2	6	6	0	100.00%	100.00%
dbus-test	15	4	11	26.70%	26.70%
diffstat	440	440	0	100.00%	100.00%
diffutils	17	17	0	100.00%	100.00%
e2fsprogs	1	1	0	100.00%	0.00%
gawk	298	297	1	99.70%	99.70%
gdbm	22	22	0	100.00%	100.00%
gdk-pixbuf	18	18	0	100.00%	100.00%
glib-2.0	250	250	0	100.00%	97.90%
kbd	7	7	0	100.00%	100.00%
libpcre	3	3	0	100.00%	100.00%
libxml2	85	82	3	96.50%	96.50%
lttng-tools	2057	1988	69	96.60%	99.80%
nettle	88	87	1	98.90%	98.90%
openssh	56	56	0	100.00%	100.00%
openssl	52	52	0	100.00%	96.20%
pango	12	12	0	100.00%	100.00%
perl	2286	2285	1	100.00%	100.00%
perl5	2286	2285	1	100.00%	100.00%
python	32058	31932	126	99.60%	99.60%
quilt	57	57	0	100.00%	98.20%
strace	278	273	5	98.20%	99.60%
tcl	148	147	1	99.30%	100.00%
util-linux	109	107	2	98.20%	98.50%
valgrind	553	2	551	0.40%	0.00%
zlib	2	2	0	100.00%	100.00%

Detailed Report

Distro Testing

Status {{#testopia: |run_id={{#var:distro}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:distro}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:distro}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Core Build System

Bitbake

Status {{#testopia: |run_id={{#var:bitbake}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:bitbake}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:bitbake}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Meta-Yocyo

Status {{#testopia: |run_id={{#var:meta-yocto}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:meta-yocto}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:meta-yocto}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

OE-Core

Status {{#testopia: |run_id={{#var:oe-core}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:oe-core}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:oe-core}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

eSDK

Status {{#testopia: |run_id={{#var:eSDK}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:eSDK}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:eSDK}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

BSP

genericx86: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:genericx86}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:genericx86}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:genericx86}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

genericx86_64: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:genericx86_64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:genericx86_64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:genericx86_64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Beaglebone Black: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:beaglebone_black}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:beaglebone_black}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:beaglebone_black}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

EdgeRouter: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:edgerouter}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:edgerouter}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:edgerouter}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

MPC8315e-rdb: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:mpc8315e-rdb}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:mpc8315e-rdb}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:mpc8315e-rdb}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

P1022ds-rdb: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:p1022ds-rdb}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:p1022ds-rdb}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:p1022ds-rdb}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Qemu

qemux86: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemux86}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemux86}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemux86}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemux86-64: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemux86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemux86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemux86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemuarm: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemuarm}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemuarm}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemuarm}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemumips: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemumips}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemumips}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemumips}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemuppc: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemuppc}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemuppc}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemuppc}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Toaster

Status {{#testopia: |run_id={{#var:toaster}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:toaster}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:toaster}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Runtime Testing

Status {{#testopia: |run_id={{#var:runtime}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:runtime}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:runtime}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Detailed result at: Build performance metrics

Build-appliance

Status {{#testopia: |run_id={{#var:build-appliance}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:build-appliance}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:build-appliance}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

ADT

ADT on: Fedora i686

Status {{#testopia: |run_id={{#var:adt_fedora_i686}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:adt_fedora_i686}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:adt_fedora_i686}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

ADT on: Fedora i686

Status {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

ADT on: Ubuntu x86-64

Status {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Eclipse Plugin

Status {{#testopia: |run_id={{#var:eclipseplugin}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:eclipseplugin}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:eclipseplugin}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Kernel

Status {{#testopia: |run_id={{#var:kernel}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:kernel}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:kernel}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

WW05 - 2017-01-24 - Full Test Cycle 2.3 M2 rc1

Testing data

Test Run Summary Report

Bugs Found during QA Test

New issues Found during QA Test & Community

High and M+ issues Found during QA Test (NOT NEW)

Other Bugs found during QA Test (NOT NEW)

Current community Open Bugs - Medium+/High

Bug Trend

Build Performance Test

Performance Charts

Performance Charts

pTest for genericx86-64 on NUC

open bugs

results

Detailed Report

Distro Testing

Core Build System

Bitbake

Meta-Yocyo

OE-Core

eSDK

BSP

genericx86: core-image-sato-sdk

genericx86_64: core-image-sato-sdk

Beaglebone Black: core-image-sato-sdk

EdgeRouter: core-image-sato-sdk

MPC8315e-rdb: core-image-sato-sdk

P1022ds-rdb: core-image-sato-sdk

Qemu

qemux86: core-image-sato-sdk

qemux86-64: core-image-sato-sdk

qemuarm: core-image-sato-sdk

qemumips: core-image-sato-sdk

qemuppc: core-image-sato-sdk

Toaster

Runtime Testing

Build-appliance

ADT

ADT on: Fedora i686

ADT on: Fedora i686

ADT on: Ubuntu x86-64

Eclipse Plugin

Kernel

Navigation menu

Search