WW33 - 2018-08-16- Full Test Cycle - 2.5.1 rc1

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 
 Test type: Full Pass Test
 Branch: master 
 Build name: yocto-2.5.1.rc1
 poky commit: 24a833f40d837407baa4e8192f372d998a776b6c 
 Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.5.1.rc1
 Distributions tested: 
      CentOs 7
      Debian 9
      Fedora 26
      Fedora 27
      OpenSuse 42.3
      Ubuntu 16.04
      Ubuntu 17.10
  
Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
9914 9915 eSDK: sumo branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
9911 CROPS: sumo branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
9909 ADT: sumo branchFedora 26 i686100.0%none0.0%none0.0%none100.0%
9902 9919 ADT: sumo branchqemu-x86100.0%none0.0%none0.0%none100.0%
9903 ADT: sumo branchqemu-x86-64100.0%none0.0%none0.0%none100.0%
9904 9920 ADT: sumo branchqemuarm100.0%none0.0%none0.0%none100.0%
9906 9921 ADT: sumo branchqemumips100.0%none0.0%none0.0%none100.0%
9907 ADT: sumo branchqemumips64100.0%none0.0%none0.0%none100.0%
9908 9922 ADT: sumo branchqemuppc100.0%none0.0%none0.0%none100.0%
9910 ADT: sumo branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
9916 Kernel: sumo branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
9913 Eclipse-Plugin: sumo branchNeon - Fedora 24100.0%none0.0%none0.0%none100.0%
9912 Eclipse-Plugin: sumo branchOxygen - Fedora26100.0%none0.0%none0.0%none100.0%
9891 Meta-yocto: sumo branchUbuntu 16.04 x86_6487.5%none12.5%12802 0.0%none100.0%
9899 Build-Appliance: sumo branchVMWare Player on Ubuntu0.0%none100.0%12804 12894 0.0%none100.0%
9900 9901 Toaster: sumo branchUbuntu 16.04 x86_6495.7%none4.3%12813 8860 12887 0.0%none100.0%
9892 OE-Core: sumo branchCentOS 7 x86_64100.0%none0.0%none0.0%none100.0%
9893 OE-Core: sumo branchDebian 9 x86_6499.3%none0.7%12885 0.0%none100.0%
9894 OE-Core: sumo branchFedora 26 x86_64100.0%none0.0%none0.0%none100.0%
9895 OE-Core: sumo branchFedora 27 x86_64100.0%none0.0%none0.0%none100.0%
9896 OE-Core: sumo branchOpenSuse 42.3 x86_64100.0%none0.0%none0.0%none100.0%
9897 OE-Core: sumo branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
9898 OE-Core: sumo branchUbuntu 17.10 x86_64100.0%none0.0%none0.0%none100.0%
9851 BitBake: sumo branchUbuntu 16.04 x86_6485.7%none14.3%12670 0.0%none100.0%
9854 9875 BSP/QEMU: sumo branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
9855 9876 BSP/QEMU: sumo branchEdgeRouter100.0%none0.0%none0.0%none100.0%
9871 BSP/QEMU: sumo branchgenericx86 on MTURBOT3289.6%none6.3%12861 4.2%none100.0%
9853 9872 BSP/QEMU: sumo branchgenericx86-64 on MTURBOT6492.3%none5.5%12666 2.2%none100.0%
9873 BSP/QEMU: sumo branchgenericx86-64 on NUC697.9%none2.1%12861 0.0%none100.0%
9852 9874 BSP/QEMU: sumo branchgenericx86-64 on NUC796.7%none3.3%12893 12861 0.0%none100.0%
9879 BSP/QEMU: sumo branchgenericx86-64-WIC on MTURBOT6497.9%none0.0%none2.1%none100.0%
9880 BSP/QEMU: sumo branchgenericx86-64-WIC on NUC697.9%none0.0%none2.1%none100.0%
9881 BSP/QEMU: sumo branchgenericx86-64-WIC on NUC797.9%none0.0%none2.1%none100.0%
9878 BSP/QEMU: sumo branchgenericx86-WIC on MTURBOT3297.9%none0.0%none2.1%none100.0%
9856 9877 BSP/QEMU: sumo branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
9857 9864 9884 BSP/QEMU: sumo branchqemu-x86100.0%none0.0%none0.0%none100.0%
9858 9865 9885 BSP/QEMU: sumo branchqemu-x86-64100.0%none0.0%none0.0%none100.0%
9883 BSP/QEMU: sumo branchqemu-x86-64-WIC97.9%none0.0%none0.0%none97.9%
9882 BSP/QEMU: sumo branchqemu-x86-WIC97.9%none0.0%none0.0%none97.9%
9859 9866 9886 BSP/QEMU: sumo branchqemuarm96.3%none3.7%12499 0.0%none100.0%
9860 9867 9887 BSP/QEMU: sumo branchqemuarm6498.1%none0.0%none0.0%none98.1%
9861 9868 9888 BSP/QEMU: sumo branchqemumips94.4%none0.0%none0.0%none94.4%
9862 9869 9889 BSP/QEMU: sumo branchqemumips6494.4%none0.0%none0.0%none94.4%
9863 9870 9890 BSP/QEMU: sumo branchqemuppc100.0%none0.0%none0.0%none100.0%
9917 Runtime: sumo branchuser-preference (any supported environment)100.0%none0.0%none0.0%none100.0%
9918 Package-Management-Issues: sumo branchAny Supported Distro100.0%none0.0%none0.0%none100.0%
Total: 46


Bug Status


Test Case Status Report
StatusCount
BLOCKED8
FAILED26
IDLE9
PASSED3351
Total: 3394

Bugs Found during QA Test

New issues found during QA Test & Community

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
12893[QA 2.5.1.rc1][BSP][NUC7][Case 266 & 267]HDMI audio for .wav and .ogg file not audible on monitor's audio port with HDMIMedium+Kevin Hao 2.5.2RESOLVEDNOTABUG---2018-08-17
Runtime Testing
12885[2.5.1 rc1] selftest wic test cases produce random results on debian9Medium+Anuj MittalMohamad Noor Alim2.5.2RESOLVEDFIXED---2018-08-14
Toaster
12887[QA 2.5.1.rc1 ][Toaster]:Search for rpi-basic-image, click on 'add layer' buttonMediumDavid ReynaToaster default assignee2.6 M4RESOLVEDFIXED---2018-08-14


High and M+ issues found during QA Test (NOT NEW)

IDSummary (2 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
12666[2.5 M3 RC1] BSP-QEMU:core-image-sato-sdk-genericx86-64.hddimg unable to connect to console with serial on Minnowboard TurbotMedium+Anuj Mittal 2.6 M2RESOLVEDFIXED---2018-04-08
Runtime
12804[QA 2.6 M1 rc1 ][Build Appliance]: proxy issue in VM wareMedium+Anuj Mittal 2.6 M3RESOLVEDFIXEDRegression (Used to work)2018-06-20

Other Bugs found during QA Test (NOT NEW)

IDSummary (7 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
12670[2.5 M3 RC1]MANUAL_BitBake:test dependancy explorer is launched with failed logsUndecidedStephano Cetola 2.6RESOLVEDFIXED---2018-04-10
BSPs
12861[2.6 M2RC1] [Meta-Intel][ corei7-64_WIC_MTURBOT64][MTURBOT64] [test_parselogs] inotify_add_watch(6, /dev/loop0, 10) failed: No such file or directoryMediumAnuj Mittal 2.6RESOLVEDFIXED---2018-07-23
Meta-yocto
12802[Yocto-2.6_M1.RC1] Crosstap doesn't work on 2.6 M1 RC1MediumManish Das 2.7RESOLVEDFIXED---2018-06-20
OE-Core
12499[2.5 M1 RC3] qemuarm: failed to shutdownLowJon Mason FutureNEW---2018-01-18
Toaster
8860The 'clear search' icon in ToasterTable tables should clear any applied filterLowToaster default assignee FutureNEW---2015-12-23
12813[QA 2.6 M1 rc1 ][Toaster]:Build stopped without error, to terminate by ctrl+cMediumDavid ReynaToaster default assignee2.6 M4RESOLVEDFIXED---2018-06-25
Regression (Found by QA test)
12836[2.6 M1 RC1] busybox ptest failedMedium+Manish Das 2.6 M2RESOLVEDNOTABUGRegression (Found by QA test)2018-07-02

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestoneor release which are Medium+ or High and are not targetedfor the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of thisweek's Yocto Bug Trend charts can be accessed on thefollowing [1]

Build Performance Test

Detailed description on how performance test is run canbe found on the Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.3 M3. Performance execution has had 3 stages, one executed from 1.6 to 2.1 in RO, second executed from 2.1 to 2.5 m1 rc3 in GDC, third executed from 2.5 m2 rc1 onward in Linux Foundation, there are differences between HW used in all locations, hence the numbers are different.

Below chars are showing the latestperformance results taken from the current milestone.

Performance Charts

Ww33-2k18 - table.png

pTest for genericx86-64 on NUC

open bugs

OE-Core
12836[2.6 M1 RC1] busybox ptest failedMedium+Manish Das 2.6 M2RESOLVEDNOTABUGRegression (Found by QA test)2018-07-02

results

< Archive


Ran on a NUC and compared with 2.5.m4.rc1

< FULL_LOG

Failures
 % increased
 % decreased
Module not executed
Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.0 100.0
attr 143 143 0 100.0 100.0
bash 79 77 2 97.47 96.2 Up rate.png
bluez5 7 7 0 100.0 100.0
busybox 524 510 14 97.33 98.09 Down rate.png
bzip2 6 6 0 100.0 100.0
dbus-test 21 19 2 90.48 90.48
diffstat 440 440 0 100.0 100.0
diffutils 20 20 0 100.0 100.0
e2fsprogs 309 309 0 100.0 100.0
flex 114 114 0 100.0 100.0
gawk 344 344 0 100.0 100.0
gdbm 25 25 0 100.0 100.0
gdk-pixbuf 20 20 0 100.0 100.0
glib-2.0 253 253 0 100.0 100.0
gzip 21 21 0 100.0 100.0
kbd 7 7 0 100.0 100.0
libpcre 3 3 0 100.0 100.0
libxml-parser-perl 15 15 0 100.0 100.0
libxml2 134 134 0 100.0 100.0
lttng-tools 2089 2020 69 96.7 96.65 Up rate.png
lzo 5 5 0 100.0 100.0
nettle 93 93 0 100.0 100.0
openssh 60 60 0 100.0 100.0
openssl 53 53 0 100.0 100.0
pango 12 12 0 100.0 100.0
perl 2287 2287 0 100.0 100.0
python 30547 30361 186 99.39 99.37 Up rate.png
quilt 57 57 0 100.0 100.0
sed 86 86 0 100.0 100.0
slang 97 97 0 100.0 100.0
strace 374 344 30 91.98 91.98
tcl 148 148 0 100.0 98.65 Up rate.png
util-linux 480 468 12 97.5 96.82 Up rate.png
valgrind 559 300 259 53.67 53.49 Up rate.png
zlib 2 2 0 100.0 100.0


Detailed Report

BitBake

Status

Bugs

Agenda

BSPs

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Meta-yocto

Status

Bugs

Agenda

OE-Core

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda