Latest revision as of 21:08, 5 December 2017

TITLE : WW## - YYYY-MM-DD - Full Test Cycle 2.x Mx rcx

{{#vardefine:adt_fedora_i686| }} {{#vardefine:adt_fedora_x86-64| }} {{#vardefine:adt_ubuntu_x86-64| }} {{#vardefine:beaglebone_black| }} {{#vardefine:edgerouter| }} {{#vardefine:genericx86| }} {{#vardefine:genericx86_64| }} {{#vardefine:mpc8315e-rdb| }} {{#vardefine:p1022ds-rdb| }} {{#vardefine:qemux86| }} {{#vardefine:qemuarm| }} {{#vardefine:qemumips| }} {{#vardefine:qemuppc| }} {{#vardefine:qemux86-64| }} {{#vardefine:eclipseplugin| }} {{#vardefine:meta-yocto| }} {{#vardefine:eSDK| }} {{#vardefine:toaster| }} {{#vardefine:oe-core| }} {{#vardefine:distro| }} {{#vardefine:bitbake| }} {{#vardefine:runtime| }} {{#vardefine:build-appliance| }} {{#vardefine:kernel| }} {{#vardefine:AutBuild-Testing| }} {{#vardefine:CROPS| }}

Testing data

 Test type:
 Branch:
 Build name:
 poky commit:
 Autobuilder images repository:
 Distributions tested: 
 -
 -

Test Run Summary Report

AutBuild-Testing

{{#testopia:

run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:kernel}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:adt_ubuntu_1310_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}}, {{#var:AutBuild-Testing}}, {{#var:CROPS}}

report_id=concatstats}}

{{#testopia:

chart=bstatus

report_id=bugs

charttype=bar

sortable=true

nudechart=true

chartpos=top

}}

{{#testopia:

run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:kernel}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}}, {{#var:AutBuild-Testing}}, {{#var:CROPS}}

charttype=bar

report_id=status}}

Bugs Found during QA Test

New issues Found during QA Test & Community

Meta-yocto
1	Should install stripped-down gnome-icon-theme	High	Ross Burton - Disabled		---	RESOLVED	FIXED	---	2005-10-12

High and M+ issues Found during QA Test (NOT NEW)

Meta-yocto
1	Should install stripped-down gnome-icon-theme	High	Ross Burton - Disabled		---	RESOLVED	FIXED	---	2005-10-12

Other Bugs found during QA Test (NOT NEW)

---
1	Should install stripped-down gnome-icon-theme	High	Ross Burton - Disabled		---	RESOLVED	FIXED	---	2005-10-12

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestone or release which are Medium+ or High and are not targeted for the next milestones

ID	P	Summary	Assignee	QA	Milestone	Status	cf_regression_type	Created
		0
BitBake
10984	Medium+	[Test Case 1058] bitbake can use alternate download methods than git	Richard Purdie	oscar lopez arandas	2.3 M3	CLOSED	---	2017-01-27
		1

Bug Trend

An overview of this week's Yocto Bug Trend charts can be accessed on the following page

Build Performance Test

Detailed description on how performance test are run can be found on Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.3 M1. Performance execution has 2 stages, one executed form 1.6 to 2.1 in RO and another one started on 2.1 from GDC, there are difference between HW used in both location hence the numbers are different.

RO Charts

GDC Charts

Below chars are showing the latest performance results taken from current milestone

Performance Charts

NOTE: During the 2.3 M2 QA cycle period, Fedora machine was under debugging activities to investigate more on bug 10884 hence only results for Ubuntu machine are displayed on the charts.

Performance Charts

For Bitbake core-image-sato with rm_work

Building the Virtual kernel

Building the rootfs

Building core-image-sato

eSDK Deploy time

pTest for genericx86-64 on NUC

open bugs

ID	Summary (12 tasks) ⇒	P	Assignee	QA	Milestone	Status	Resolution	cf_regression_type	Created
OE-Core
5697	busybox: ptest cases failed	Medium	Changqing Li		2.7 M2	RESOLVED	FIXED	---	2013-12-31
5695	python: ptest cases failed	Medium	Ross Burton		2.6	RESOLVED	FIXED	---	2013-12-31
10841	pTest : Various test cases are failing on "dbus-test" module	Medium	Ross Burton		2.5	RESOLVED	DUPLICATE	---	2016-12-21
10842	testrunner test timeout can lead to garbage output	Medium	Aníbal Limón		2.3 M4	VERIFIED	FIXED	---	2016-12-21
10840	pTest- openssl: test cases "test_bn" and "test_cms" are failing	Medium+	Alexander Kanavin		2.4 M1	VERIFIED	FIXED	---	2016-12-21
Runtime Testing
8828	pTest - lttng-tools tests fail	Medium	Richard Purdie	yeoh ee peng	4.99	RESOLVED	FIXED	Regression (Used to work)	2015-12-17
9372	Ptest for acl hang because of lack of two user bin and daemon	Medium	Chen Qi	oscar lopez arandas	2.4	RESOLVED	FIXED	---	2016-04-01
8471	ptest - some valgrind ptests fail on 2.4 M1.rc1	Medium	Maxin B. John		4.99	RESOLVED	DUPLICATE	Regression (Used to work)	2015-10-09
9055	pTest - acl (NUC) failed tested on NUC 5425WYK for YP 2.1_M2.rc1	Medium	Robert Yang	Francisco Pedraza	2.4	RESOLVED	FIXED	Regression (Used to work)	2016-02-04
9042	pTest - strace fails on NUC 5425WYK for YP 2.1_M2.rc1	Medium	Changqing Li		2.7 M2	RESOLVED	FIXED	Regression (Used to work)	2016-02-03
8832	pTest - bash tests fail for 2.1_M1.rc1	Medium	Changqing Li		2.7 M2	RESOLVED	WORKSFORME	---	2015-12-17
8831	pTest - busybox tests fail for 2.1_M1.rc1	Medium	dengke		Future	RESOLVED	DUPLICATE	---	2015-12-17

results

< Back
Ran on a NUC and compared with commit 573c646d4cc62dcd0c230381df4940bdf314d495

FULL LOG

Failures

% increased

% decreased

Results
Package Name	Tests ran	Passed	Failed	Passrate	Last Passrate
acl	380	354	26	93.20%	93.20%
attr	143	143	0	100.00%	100.00%
bash	76	72	4	94.70%	94.70%
bluez5	25	25	0	100.00%	100.00%
busybox	518	509	9	98.30%	98.60%
bzip2	6	6	0	100.00%	100.00%
dbus-test	15	4	11	26.70%	26.70%
diffstat	440	440	0	100.00%	100.00%
diffutils	17	17	0	100.00%	100.00%
e2fsprogs	1	1	0	100.00%	0.00%
gawk	298	297	1	99.70%	99.70%
gdbm	22	22	0	100.00%	100.00%
gdk-pixbuf	18	18	0	100.00%	100.00%
glib-2.0	250	250	0	100.00%	97.90%
kbd	7	7	0	100.00%	100.00%
libpcre	3	3	0	100.00%	100.00%
libxml2	85	82	3	96.50%	96.50%
lttng-tools	2057	1988	69	96.60%	99.80%
nettle	88	87	1	98.90%	98.90%
openssh	56	56	0	100.00%	100.00%
openssl	52	52	0	100.00%	96.20%
pango	12	12	0	100.00%	100.00%
perl	2286	2285	1	100.00%	100.00%
perl5	2286	2285	1	100.00%	100.00%
python	32058	31932	126	99.60%	99.60%
quilt	57	57	0	100.00%	98.20%
strace	278	273	5	98.20%	99.60%
tcl	148	147	1	99.30%	100.00%
util-linux	109	107	2	98.20%	98.50%
valgrind	553	2	551	0.40%	0.00%
zlib	2	2	0	100.00%	100.00%

Compliance Report

POSIX results

General results for POSIX execution can be found POSIX_result and greater details on each architecture can be found POSIX_details

Below you will see the last three execution results for comparison:

Yocto 2.# RC3 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	##	##	##	##

Yocto 2.# RC2 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	##	##	##	##

Yocto 2.# RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	##	##	##	##

LTP results

General results for LTP can be found here LTP_results and for greater information click on each architecture to be redirected to their details.

Yocto 2.# RC3 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
total	####	####	####	####
failures	##	##	##	##

Yocto 2.# RC2 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
total	####	####	####	####
failures	##	##	##	##

Yocto 2.# RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
total	####	####	####	####
failures	##	##	##	##

Detailed Report

Distro Testing

Status {{#testopia: |run_id={{#var:distro}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:distro}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:distro}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Core Build System

Bitbake

Status {{#testopia: |run_id={{#var:bitbake}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:bitbake}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:bitbake}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Meta-Yocyo

Status {{#testopia: |run_id={{#var:meta-yocto}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:meta-yocto}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:meta-yocto}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

OE-Core

Status {{#testopia: |run_id={{#var:oe-core}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:oe-core}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:oe-core}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

eSDK

Status {{#testopia: |run_id={{#var:eSDK}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:eSDK}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:eSDK}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

BSP

genericx86: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:genericx86}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:genericx86}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:genericx86}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

genericx86_64: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:genericx86_64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:genericx86_64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:genericx86_64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Beaglebone Black: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:beaglebone_black}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:beaglebone_black}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:beaglebone_black}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

EdgeRouter: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:edgerouter}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:edgerouter}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:edgerouter}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

MPC8315e-rdb: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:mpc8315e-rdb}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:mpc8315e-rdb}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:mpc8315e-rdb}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

P1022ds-rdb: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:p1022ds-rdb}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:p1022ds-rdb}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:p1022ds-rdb}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Qemu

qemux86: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemux86}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemux86}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemux86}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemux86-64: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemux86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemux86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemux86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemuarm: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemuarm}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemuarm}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemuarm}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemumips: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemumips}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemumips}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemumips}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemuppc: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemuppc}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemuppc}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemuppc}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Toaster

Status {{#testopia: |run_id={{#var:toaster}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:toaster}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:toaster}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Runtime Testing

Status {{#testopia: |run_id={{#var:runtime}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:runtime}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:runtime}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Detailed result at: Build performance metrics

Build-appliance

Status {{#testopia: |run_id={{#var:build-appliance}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:build-appliance}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:build-appliance}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

ADT

ADT on: Fedora i686

Status {{#testopia: |run_id={{#var:adt_fedora_i686}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:adt_fedora_i686}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:adt_fedora_i686}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

ADT on: Fedora i686

Status {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

ADT on: Ubuntu x86-64

Status {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Eclipse Plugin

Status {{#testopia: |run_id={{#var:eclipseplugin}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:eclipseplugin}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:eclipseplugin}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Kernel

Status {{#testopia: |run_id={{#var:kernel}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:kernel}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:kernel}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

@@ Line 1: / Line 1: @@
+TITLE : WW## - YYYY-MM-DD - Full Test Cycle 2.x Mx rcx
 {|
 {{#vardefine:adt_fedora_i686|     }}
@@ Line 24: / Line 26: @@
 {{#vardefine:build-appliance|     }}
 {{#vardefine:kernel|              }}
+{{#vardefine:AutBuild-Testing|    }}
+{{#vardefine:CROPS|               }}
 |}
@@ Line 39: / Line 43: @@
 =Test Run Summary Report=
+AutBuild-Testing
 {|
 |{{#testopia:
-|run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:kernel}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:adt_ubuntu_1310_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}}
+|run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:kernel}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:adt_ubuntu_1310_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}}, {{#var:AutBuild-Testing}}, {{#var:CROPS}}
 |report_id=concatstats}}
 <br>
 {{#testopia:
-|run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:kernel}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}}  |chart=bstatus
+|run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:kernel}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}}, {{#var:AutBuild-Testing}}, {{#var:CROPS}}   |chart=bstatus
    |report_id=bugs
    |charttype=bar
@@ Line 57: / Line 63: @@
 <br>
 {{#testopia:
-|run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:kernel}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}}
+|run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:kernel}}, {{#var:eSDK}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}}, {{#var:AutBuild-Testing}}, {{#var:CROPS}}
 |charttype=bar
 |report_id=status}}
@@ Line 177: / Line 183: @@
 {{:Ptest 62d7d4130202d8ede16abf9e7d779361ca70847e}}
+=Compliance Report=
+==POSIX results==
+General results for POSIX execution can be found [https://wiki.yoctoproject.org/wiki/Posix_result POSIX_result]
+and greater details on each architecture can be found [https://wiki.yoctoproject.org/wiki/POSIX-results POSIX_details]
+Below you will see the last three execution results for comparison:
+===Yocto 2.# RC3 build===
+{|border="1"
+|| '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb'''
+|-
+|| failures || ## || ## || ## || ##
+|}
+===Yocto 2.# RC2 build===
+{|border="1"
+|| '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb'''
+|-
+|| failures || ## || ## || ## || ##
+|}
+===Yocto 2.# RC1 build===
+{|border="1"
+|| '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb'''
+|-
+|| failures || ## || ## || ## || ##
+|}
+==LTP results==
+General results for LTP can be found here [https://wiki.yoctoproject.org/wiki/LTP_result LTP_results]
+and for greater information click on each architecture to be redirected to their details.
+===Yocto 2.# RC3 build===
+{|border="1"
+|| '''Status''' || '''[[genericx86-64-ltp|genericx86-64]]''' || '''[[beaglebone-ltp|beaglebone]]''' || '''[[edgerouter-ltp|edgerouter]]''' || '''[[mpc8315e-rdb-ltp|mpc8315e-rdb]]'''
+|-
+|| total || #### || #### || #### || ####
+|-
+|| failures || ## || ## || ## || ##
+|}
+===Yocto 2.# RC2 build===
+{|border="1"
+|| '''Status''' || '''[[genericx86-64-ltp|genericx86-64]]''' || '''[[beaglebone-ltp|beaglebone]]''' || '''[[edgerouter-ltp|edgerouter]]''' || '''[[mpc8315e-rdb-ltp|mpc8315e-rdb]]'''
+|-
+|| total || #### || #### || #### || ####
+|-
+|| failures || ## || ## || ## || ##
+|}
+===Yocto 2.# RC1 build===
+{|border="1"
+|| '''Status''' || '''[[genericx86-64-ltp|genericx86-64]]''' || '''[[beaglebone-ltp|beaglebone]]''' || '''[[edgerouter-ltp|edgerouter]]''' || '''[[mpc8315e-rdb-ltp|mpc8315e-rdb]]'''
+|-
+|| total || #### || #### || #### || ####
+|-
+|| failures || ## || ## || ## || ##
+|}
 =Detailed Report=

QA REPORT TEMPLATE: Difference between revisions

Latest revision as of 21:08, 5 December 2017

Testing data

Test Run Summary Report

Bugs Found during QA Test

New issues Found during QA Test & Community

High and M+ issues Found during QA Test (NOT NEW)

Other Bugs found during QA Test (NOT NEW)

Current community Open Bugs - Medium+/High

Bug Trend

Build Performance Test

Performance Charts

Performance Charts

pTest for genericx86-64 on NUC

open bugs

results

Compliance Report

POSIX results

Yocto 2.# RC3 build

Yocto 2.# RC2 build

Yocto 2.# RC1 build

LTP results

Yocto 2.# RC3 build

Yocto 2.# RC2 build

Yocto 2.# RC1 build

Detailed Report

Distro Testing

Core Build System

Bitbake

Meta-Yocyo

OE-Core

eSDK

BSP

genericx86: core-image-sato-sdk

genericx86_64: core-image-sato-sdk

Beaglebone Black: core-image-sato-sdk

EdgeRouter: core-image-sato-sdk

MPC8315e-rdb: core-image-sato-sdk

P1022ds-rdb: core-image-sato-sdk

Qemu

qemux86: core-image-sato-sdk

qemux86-64: core-image-sato-sdk

qemuarm: core-image-sato-sdk

qemumips: core-image-sato-sdk

qemuppc: core-image-sato-sdk

Toaster

Runtime Testing

Build-appliance

ADT

ADT on: Fedora i686

ADT on: Fedora i686

ADT on: Ubuntu x86-64

Eclipse Plugin

Kernel

Navigation menu

Search