WW51 - 2017-12-20- Full Test Cycle - 2.3.3 rc1

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 
 Test type: Full Pass Test
 Branch: pyro
 Build name: yocto-2.3.3.rc1
 poky commit: 022df4653576c72752156bef8fda0e1aa46733d2 
 Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.3.3.rc1
 Distributions tested: 

     CentOS 7.1 x86_64
     Debian 8 x86_64
     Fedora 25 x86_64
     Fedora 26 x86_64
     OpenSuse 42.3 x86_64	
     OpenSuse 422 x86_64
     ubuntu 16.04 x86_64
     Ubuntu 17.04 x86-64


  
Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
8791 8792 eSDK: pyro branchCentos 7.1100.0%none0.0%none0.0%none100.0%
8788 CROPS: pyro branchCentos 7.1100.0%none0.0%none0.0%none100.0%
8748 8749 ADT: pyro branchFedora 25 i686100.0%none0.0%none0.0%none100.0%
8750 8751 ADT: pyro branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
8794 Kernel: pyro branchFedora 24100.0%none0.0%none0.0%none100.0%
8790 Eclipse-Plugin: pyro branchNeon on Ubuntu 16.04100.0%none0.0%none0.0%none100.0%
8789 Eclipse-Plugin: pyro branchOxygen - Fedora24100.0%none0.0%none0.0%none100.0%
8795 Meta-yocto: pyro branchUbuntu 14.04 x86_64100.0%none0.0%none0.0%none100.0%
8805 Build-Appliance: pyro branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
8806 Toaster: pyro branchDebian 4.3.0-0.bpo.1-amd64100.0%12436 12437 0.0%none0.0%none100.0%
8796 OE-Core: pyro branchCentOS 7.1 x86_64100.0%none0.0%none0.0%none100.0%
8797 OE-Core: pyro branchDebian 8 x86_64100.0%none0.0%none0.0%none100.0%
8798 OE-Core: pyro branchFedora 25 x86_64100.0%none0.0%none0.0%none100.0%
8804 OE-Core: pyro branchFedora 26 x86_6499.5%none0.5%12440 0.0%none100.0%
8800 OE-Core: pyro branchOpenSuse 42.3 x86_64100.0%none0.0%none0.0%none100.0%
8801 OE-Core: pyro branchOpenSuse 422 x86_64100.0%none0.0%none0.0%none100.0%
8802 OE-Core: pyro branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
8803 OE-Core: pyro branchUbuntu 17.04 x86-6499.5%none0.5%12022 0.0%none100.0%
8752 BitBake: pyro branchUbuntu 15.04 x86_64100.0%none0.0%none0.0%none100.0%
8783 BSP/QEMU: pyro branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
8755 BSP/QEMU: pyro branchBeaglebone-Black100.0%none0.0%none0.0%none100.0%
8756 8784 BSP/QEMU: pyro branchEdgeRouter100.0%none0.0%none0.0%none100.0%
8776 BSP/QEMU: pyro branchgenericx86 - on MMAX32bit97.8%none2.2%none0.0%none100.0%
8777 BSP/QEMU: pyro branchgenericx86-64 on MMAX6497.8%none2.2%10455 0.0%none100.0%
8778 BSP/QEMU: pyro branchgenericx86-64 on NUC97.8%none2.2%10455 0.0%none100.0%
8786 BSP/QEMU: pyro branchgenericx86-64-lsb on MMAX6497.9%none2.1%10455 0.0%none100.0%
8785 BSP/QEMU: pyro branchgenericx86-64-lsb on NUC97.9%none2.1%10455 0.0%none100.0%
8779 BSP/QEMU: pyro branchgenericx86-64-WIC on MMAX6497.8%none2.2%10455 0.0%none100.0%
8780 BSP/QEMU: pyro branchgenericx86-64-WIC on NUC97.8%none2.2%9267 0.0%none100.0%
8754 BSP/QEMU: pyro branchGenericx86-64_NUC100.0%none0.0%none0.0%none100.0%
8787 BSP/QEMU: pyro branchgenericx86-lsb on MMAX32bit100.0%none0.0%none0.0%none100.0%
8781 BSP/QEMU: pyro branchgenericx86-WIC on MMAX3297.8%none2.2%10455 0.0%none100.0%
8753 BSP/QEMU: pyro branchGenericx86_MMAX32100.0%none0.0%none0.0%none100.0%
8757 8782 BSP/QEMU: pyro branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
8758 8765 BSP/QEMU: pyro branchqemu-x86100.0%none0.0%none0.0%none100.0%
8759 8766 BSP/QEMU: pyro branchqemuarm100.0%none0.0%none0.0%none100.0%
8760 8767 BSP/QEMU: pyro branchqemuarm64100.0%none0.0%none0.0%none100.0%
8761 8768 BSP/QEMU: pyro branchqemumips100.0%none0.0%none0.0%none100.0%
8762 8769 BSP/QEMU: pyro branchqemumips64100.0%none0.0%none0.0%none100.0%
8763 8770 BSP/QEMU: pyro branchqemuppc100.0%none0.0%none0.0%none100.0%
8764 BSP/QEMU: pyro branchqemux86-64100.0%none0.0%none0.0%none100.0%
8771 BSP/QEMU: pyro branchqemux86-64,100.0%none0.0%none0.0%none100.0%
8774 BSP/QEMU: pyro branchqemux86-lsb100.0%none0.0%none0.0%none100.0%
8775 BSP/QEMU: pyro branchqemux86_64-lsb100.0%none0.0%none0.0%none100.0%
8793 Runtime: pyro branchuser-preference (any supported environment)100.0%none0.0%none0.0%none100.0%
Total: 45


Bug Status


Test Case Status Report
StatusCount
FAILED10
PASSED2975
RUNNING2
Total: 2987

Bugs Found during QA Test

New issues Found during QA Test & Community

IDSummary (4 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
12440test_qemu_efi (oeqa.selftest.wic.Wic) fails on Fedora26Medium+Armin Kuster 2.3.4RESOLVEDFIXED---2017-12-20
Test Plans/Suite
12441ptest comparison script is not showing correct resultsMedium+Sudhir Sharmayeoh ee peng2.7 M2RESOLVEDFIXED---2017-12-21
Toaster
12436[Test Case 1958] Project names cannot be blankMediumDavid ReynaLibertad2.99NEW---2017-12-19
12437[Test Case 1958] Toaster exploratory testingMediumDavid ReynaLibertad2.99NEW---2017-12-19


High and M+ issuesFound during QA Test (NOT NEW)

Build Testing
12022signing.Signing.test_signing_packages hangs on ubuntu1704.yocto.ioMedium+Alexander Kanavin 2.5 M2RESOLVEDFIXED---2017-08-31

Other Bugs found during QA Test (NOT NEW)

IDSummary (2 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
10455[morty][Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) giving error: blk_update_request: I/O error, dev loop0MediumCalifornia Sullivanyeoh ee peng2.5 M3RESOLVEDFIXED---2016-10-18
Regression (Found by QA test)
9267test_parselogs (oeqa.runtime.parselogs.ParseLogsTest) failed on NUC 6i5SYH, firmware load for i915/skl_guc_ver4.bin failedMediumCalifornia SullivanApoorv2.5RESOLVEDFIXEDRegression (Found by QA test)2016-03-15

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestoneor release which are Medium+ or High and are not targetedfor the next milestones

IDPSummary (2 tasks) AssigneeQAMilestoneStatuscf_regression_typeCreated
  0      
BitBake
10984Medium+[Test Case 1058] bitbake can use alternate download methods than gitRichard Purdieoscar lopez arandas2.3 M3CLOSED---2017-01-27
  1      
OE-Core
12059High[pyro-next] adwaita-icon-theme buildspec issueArmin Kuster 2.3.2CLOSED---2017-09-06
  1      

Bug Trend

An overview of thisweek's Yocto Bug Trend charts can be accessed on thefollowing [1]

Build Performance Test

Detailed description on how performance test is run canbe found on the Performance Test wiki. The below charts are displaying the build performance trend for 2.1M4 to 2.3.3. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and from 2.1 to 2.4 executed in GDC. There are differences between HW used in bothlocations, hence the numbers are different.

Below chars are showing the latestperformance results taken from the current milestone.

Performance Charts

Results Ww51-2k17 - table.png

For Bitbake core-image-sato withrm_work

Ww51-2k17 - rmwork.png

Building the Virtual kernel

Ww51-2k17 - kernel.png

Building the rootfs

Ww51-2k17 - rootfs.png

Building core-image-sato

Ww51-2k17 - sato.png

eSDK Deploy time

Ww51-2k17 - esdk.png

pTest for genericx86-64 on NUC

open bugs

Test Plans/Suite
12441ptest comparison script is not showing correct resultsMedium+Sudhir Sharmayeoh ee peng2.7 M2RESOLVEDFIXED---2017-12-21

results

< Archive


Ran on a NUC and compared with 2.3.2_rc1.log

FULL_LOG

Failures
 % increased
 % decreased
Module not executed

Detailed Report

ADT

Status

Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.0 100.0
attr 143 143 0 100.0 100.0
bash 76 72 4 94.74 93.67 Up rate.png
bluez5 25 25 0 100.0 100.0
busybox 518 508 10 98.07 98.07
bzip2 6 6 0 100.0 100.0
dbus-test 15 4 11 26.67 26.67
diffstat 440 440 0 100.0 100.0
diffutils 17 17 0 100.0 100.0
e2fsprogs 1 0 1 0.0 0.0
flex 114 114 0 100.0 100.0
gawk 298 297 1 99.66 99.66
gdbm 22 22 0 100.0 100.0
gdk-pixbuf 20 20 0 100.0 100.0
glib-2.0 252 251 1 99.6 100.0 Down rate.png
gzip 18 18 0 0 100.0 Down rate.png
kbd 7 7 0 100.0 100.0
libpcre 3 3 0 100.0 100.0
libxml2 39 36 3 92.31 92.31
lttng-tools 2057 1988 69 96.65 96.65
nettle 88 87 1 98.86 98.86
openssh 34 33 1 97.06 100.0 Down rate.png
openssl 52 52 0 100.0 100.0
pango 12 12 0 100.0 100.0
perl 2286 2285 1 99.96 99.96
perl5 2286 2285 1 99.96 99.96
python 32230 32104 126 99.61 0 Up rate.png
quilt 57 57 0 100.0 100.0
slang 97 97 0 0 100.0 Down rate.png
strace 335 334 1 99.7 99.72 Down rate.png
tcl 148 146 2 98.65 99.32 Down rate.png
util-linux 108 106 2 98.15 82.93 Up rate.png
valgrind 553 296 257 53.53 53.49 Up rate.png
zlib 2 2 0 100.0 100.0

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

BitBake

Status

Bugs

**No results**

Agenda

BSPs

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

CROPS

Status

Bugs

**No results**

Agenda

Eclipse Plugin

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

eSDK

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

General Runtime

Status

Bugs

**No results**

Agenda

Kernel

Status

Bugs

**No results**

Agenda

Meta-yocto

Status

Bugs

**No results**

Agenda

OE-Core

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda