1.9 QA Status TEMPLATE

From Yocto Project
Revision as of 13:24, 25 August 2015 by Alexandru.c.georgescu (talk | contribs) (Created page with "{| {{#vardefine:adt_fedora_i686| }} {{#vardefine:adt_fedora_x86-64| }} {{#vardefine:adt_ubuntu_x86-64| }} {{#vardefine:beaglebone_black| }} {...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search
{{#vardefine:adt_fedora_i686| }} {{#vardefine:adt_fedora_x86-64| }} {{#vardefine:adt_ubuntu_x86-64| }} {{#vardefine:beaglebone_black| }} {{#vardefine:edgerouter| }} {{#vardefine:genericx86| }} {{#vardefine:genericx86_64| }} {{#vardefine:mpc8315e-rdb| }} {{#vardefine:p1022ds-rdb| }} {{#vardefine:qemux86| }} {{#vardefine:qemuarm| }} {{#vardefine:qemumips| }} {{#vardefine:qemuppc| }} {{#vardefine:qemux86-64| }} {{#vardefine:eclipseplugin| }} {{#vardefine:meta-yocto| }} {{#vardefine:hob| }} {{#vardefine:toaster| }} {{#vardefine:oe-core| }} {{#vardefine:distro| }} {{#vardefine:bitbake| }} {{#vardefine:runtime| }} {{#vardefine:build-appliance| }}

Testing data

 Test type:  Full Pass Release Test
 Branch: master
 Build name: 
 poky commit: 
 Autobuilder images repository: 
 Distributions tested: Ubuntu 15.04 64 bit, Ubuntu 14.04 64bit, Fedora 21 64bit, Fedora 22 64bit, Opensuse 13.2 64bit, CentOS 7 64 bit 

Test Run Summary Report

{{#testopia: run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:hob}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:adt_ubuntu_1310_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}} report_id=concatstats}}


{{#testopia:

chart=bstatus report_id=bugs charttype=bar sortable=true nudechart=true chartpos=top

}}


{{#testopia:

run_id=0, {{#var:bitbake}}, {{#var:oe-core}}, {{#var:meta-yocto}}, {{#var:hob}}, {{#var:nuc}}, {{#var:atom-pc}}, {{#var:genericx86}}, {{#var:genericx86_64}}, {{#var:beaglebone_black}}, {{#var:edgerouter}}, {{#var:mpc8315e-rdb}}, {{#var:p1022ds-rdb}}, {{#var:qemux86}}, {{#var:qemux86-64}}, {{#var:qemuarm}}, {{#var:qemumips}}, {{#var:qemuppc}}, {{#var:runtime}}, {{#var:build-appliance}}, {{#var:toaster}}, {{#var:adt_opensuse_123_x86-64}}, {{#var:adt_fedora18_i686}}, {{#var:adt_fedora19_x86-64}}, {{#var:adt_fedora19_i686}}, {{#var:adt_fedora_i686}}, {{#var:adt_ubuntu_x86-64}}, {{#var:eclipseplugin}}, {{#var:distro}} charttype=bar report_id=status}}



Bugs Found during QA Test

New and notable issues Found during QA Test & Community

IDSummary (5 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
8012connection errors on genericx86-64-lsb on NUCMedium+Lucian MusatCristina Agurida1.9 M3VERIFIEDFIXEDNew (Never tested)2015-07-15
Eclipse Plugin
7996Create a new BitBake commander project failedMedium+Todor MinchevValentin Hangan1.9 M3VERIFIEDFIXED---2015-07-14
Toaster
7992Package details page throws an errorHighAlexandru DamianCristina Agurida1.9 M3VERIFIEDFIXED---2015-07-14
7810The Toaster directory structure does not expand / collapseMedium+David ReynaCristina Agurida1.9 M3VERIFIEDFIXED---2015-05-27
8011Sorting columns breaks ToasterMedium+Michael WoodMihail Stanciu2.0VERIFIEDDUPLICATERegression (Used to work)2015-07-15


Other Bugs found during QA Test (including regressions)

IDSummary (13 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
8062ptest - quilt ptest is broken - has many failuresMediumMaxin B. John (inactive)Alexandru Georgescu1.9RESOLVEDFIXED---2015-07-28
Runtime Testing
8057ptest - E2fsprogs Oprofile Openssh Bash and Kmod ptest packages are skippedMediumRoss BurtonLucian Musat1.9 M3VERIFIEDINVALID---2015-07-28
8063ptest - Valgrind ptest is broken - has many failuresMediumDave LernerAlexandru Georgescu1.9RESOLVEDFIXED---2015-07-28
8059ptest - some openssl ptests failMedium+Maxin B. JohnAlexandru Georgescu2.1VERIFIEDFIXED---2015-07-28
8021Ptest-runner hangs on certain stepMedium+Aníbal LimónCristina Agurida2.1 M3VERIFIEDFIXED---2015-07-16
New (Never tested)
7897Dmesg error BAR 14 and BAR 15 failed to assign in Minnow MaxMediumSaul WoldCristina Agurida2.1 M3RESOLVEDFIXEDNew (Never tested)2015-06-17
Regression (Found by QA test)
6584dmesg uvesafb: probe of uvesafb.0 failed with error -22 on genericx86/genericx86-64 on NUC and MinnowMaxMediumJianxun ZhangCristina Agurida2.1RESOLVEDFIXEDRegression (Found by QA test)2014-07-28
Regression (Used to work)
6156[crosstap] no trace message output on qemuarmMediumTom Zanussi 2.2VERIFIEDNOTABUGRegression (Used to work)2014-04-14
Toaster
8148Sorting the content of the Layer/Machines table doesn't workMediumMichael WoodAlexandru Roman2.1VERIFIEDFIXEDRegression (Used to work)2015-08-11
8131All builds page "Errors"/"Warnings" filters doesn't workMediumMichael WoodLibertad2.2RESOLVEDOBSOLETERegression (Used to work)2015-08-10
8126Messages are missing when "IMAGE_FSTYPES" field is not properly editedMediumDavid ReynaDavid Lopez Barriba2.4 M4RESOLVEDFIXEDRegression (Used to work)2015-08-07
8125Filter does not work as expected after clearing the search resultsMediumMichael WoodCristina Agurida2.2RESOLVEDDUPLICATERegression (Used to work)2015-08-07
7198Hiding a sorted column does not revert to default sortingMediumMichael WoodLibertad2.2RESOLVEDDUPLICATERegression (Used to work)2015-01-20

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestone or release which are Medium+ or High and are not targeted for the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of this week's Yocto Bug Trend charts can be accessed on the following page

pTest results for genericx86-64 on NUC

  • Commit 1.9_M2.rc2: c75cefe8a382a63f625123c156137782db118f64
  • Previous Commit 1.9_M2.rc1: f9ea480e6c26fea46c3011c4e9d0e2de3d5c81b4


Results
Package Name Tests ran Passed Failed Passrate Last Passrate Run difference
valgrind 561 351 210 63.0 63.0 No
lttng-tools 787 781 6 99.0 99.0 No
beecrypt 21 21 0 100.0 100.0 No
diffstat 439 439 0 100.0 100.0 No
strace 30 24 6 80.0 80.0 No
acl 211 197 14 93.0 93.0 No
glib-2.0 215 214 1 100.0 100.0 No
libpcre 3 3 0 100.0 100.0 No
quilt 55 3 52 5.0 5.0 No
pango 11 10 1 91.0 91.0 No
gdbm 11 11 0 100.0 100.0 No
busybox 487 473 14 97.0 97.0 No
dbus-test 15 15 0 100.0 100.0 No
openssl 48 19 29 40.0 40.0 No
libxml2 85 85 0 100.0 100.0 No
udev 139 139 0 100.0 100.0 No
diffutils 12 12 0 100.0 100.0 No
gdk-pixbuf 9 9 0 100.0 100.0 No
tcl 148 145 3 98.0 98.0 No
python 14187 14131 56 100.0 100.0 No
e2fsprogs 1 0 1 0.0 0.0 No
attr 108 108 0 100.0 100.0 No
gawk 282 281 1 100.0 100.0 No
perl 2266 2257 9 100.0 100.0 No
bzip2 6 6 0 100.0 100.0 No
zlib 2 2 0 100.0 100.0 No

Build Performance Test

Detailed description on how performance test are run can be found on Performance Test wiki. The below charts are displaying the build performance trend for 1.8 by milestone starting 1.6 on Master. The charts by Milestone can be accessed at this link where you can view more detailed information or by clicking on the images which will open a new window with the interactive chart.


Performance Charts

For Bitbake core-image-sato with rm_work

Ww33-2k15 - rmwork.JPG

Building the Virtual kernel

Ww33-2k15 - kernel.JPG

Building the rootfs

Ww33-2k15 - rootfs.JPG

Detailed Report

Distro Testing

Status {{#testopia: |run_id={{#var:distro}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:distro}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:distro}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Core Build System

Bitbake

Status {{#testopia: |run_id={{#var:bitbake}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:bitbake}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:bitbake}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Meta-Yocyo

Status {{#testopia: |run_id={{#var:meta-yocto}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:meta-yocto}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:meta-yocto}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

OE-Core

Status {{#testopia: |run_id={{#var:oe-core}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:oe-core}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:oe-core}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Hob

Status {{#testopia: |run_id={{#var:hob}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:hob}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:hob}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

BSP

genericx86: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:genericx86}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:genericx86}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:genericx86}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

genericx86_64: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:genericx86_64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:genericx86_64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:genericx86_64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Beaglebone Black: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:beaglebone_black}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:beaglebone_black}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:beaglebone_black}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

EdgeRouter: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:edgerouter}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:edgerouter}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:edgerouter}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

MPC8315e-rdb: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:mpc8315e-rdb}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:mpc8315e-rdb}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:mpc8315e-rdb}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

P1022ds-rdb: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:p1022ds-rdb}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:p1022ds-rdb}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:p1022ds-rdb}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}


Qemu

qemux86: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemux86}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemux86}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemux86}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemux86-64: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemux86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemux86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemux86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemuarm: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemuarm}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemuarm}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemuarm}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemumips: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemumips}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemumips}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemumips}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

qemuppc: core-image-sato-sdk

Status {{#testopia: |run_id={{#var:qemuppc}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:qemuppc}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:qemuppc}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Toaster

Status {{#testopia: |run_id={{#var:toaster}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:toaster}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:toaster}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Runtime Testing

Status {{#testopia: |run_id={{#var:runtime}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:runtime}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:runtime}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Detailed result at: Build performance metrics

Build-appliance

Status {{#testopia: |run_id={{#var:build-appliance}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:build-appliance}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:build-appliance}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

ADT

ADT on: Fedora i686

Status {{#testopia: |run_id={{#var:adt_fedora_i686}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:adt_fedora_i686}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:adt_fedora_i686}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

ADT on: Fedora i686

Status {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:adt_fedora_x86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

ADT on: Ubuntu x86-64

Status {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:adt_ubuntu_x86-64}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}

Eclipse Plugin

Status {{#testopia: |run_id={{#var:eclipseplugin}} |showhide=true |hidden=true |report_id=status |charttype=pie3}}

Bugs {{#testopia: |run_id={{#var:eclipseplugin}} |report_id=bugs |chart=bstatus,prio,severity |charttype=pie,pie,pie |showhide=true |hidden=true}}

Agenda {{#testopia: |run_id={{#var:eclipseplugin}} |report_id=agenda |showhide=true |hidden=true |reportoptions=withdependency=false}}