WW49 - 2018-12-06- Full Test Cycle - 2.5.2 rc1

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 
 Test type: Full Pass Test
 Branch: Sumo
 Build name: yocto-2.5.2.rc1
 poky commit:  95ebfb33e4c0aa9669f318e061a4d7540b84f4ac 
 Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.5.2.rc1
 Distributions tested: 
      CentOs 7
      Debian 9
      Fedora 26
      Fedora 27
      OpenSuse 42.3
      Ubuntu 16.04
      Ubuntu 17.10
  
Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
11111 11112 eSDK: sumo branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
11108 CROPS: sumo branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
11102 ADT: sumo branchFedora 26 i686100.0%none0.0%none0.0%none100.0%
11095 11104 ADT: sumo branchqemu-x86100.0%none0.0%none0.0%none100.0%
11096 ADT: sumo branchqemu-x86-64100.0%none0.0%none0.0%none100.0%
11097 11105 ADT: sumo branchqemuarm100.0%none0.0%none0.0%none100.0%
11098 ADT: sumo branchqemuarm64100.0%none0.0%none0.0%none100.0%
11099 11106 ADT: sumo branchqemumips100.0%none0.0%none0.0%none100.0%
11100 ADT: sumo branchqemumips64100.0%none0.0%none0.0%none100.0%
11101 11107 ADT: sumo branchqemuppc100.0%none0.0%none0.0%none100.0%
11103 ADT: sumo branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
11113 Kernel: sumo branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
11110 Eclipse-Plugin: sumo branchNeon on Ubuntu 16.04100.0%none0.0%none0.0%none100.0%
11109 Eclipse-Plugin: sumo branchOxygen - Fedora26100.0%none0.0%none0.0%none100.0%
11092 Build-Appliance: sumo branchVMWare Player on Ubuntu75.0%none25.0%12991 0.0%none100.0%
11093 11094 Toaster: sumo branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
11085 OE-Core: sumo branchCentOS 7 x86_64100.0%none0.0%none0.0%none100.0%
11086 OE-Core: sumo branchDebian 9 x86_64100.0%none0.0%none0.0%none100.0%
11087 OE-Core: sumo branchFedora 26 x86_64100.0%none0.0%none0.0%none100.0%
11088 OE-Core: sumo branchFedora 27 x86_64100.0%none0.0%none0.0%none100.0%
11089 OE-Core: sumo branchOpenSuse 42.3 x86_64100.0%none0.0%none0.0%none100.0%
11090 OE-Core: sumo branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
11091 OE-Core: sumo branchUbuntu 17.10 x86_64100.0%none0.0%none0.0%none100.0%
11116 BitBake: sumo branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
11048 11069 BSP/QEMU: sumo branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
11049 11070 BSP/QEMU: sumo branchEdgeRouter100.0%none0.0%none0.0%none100.0%
11065 BSP/QEMU: sumo branchgenericx86 on MTURBOT3297.9%none0.0%none2.1%none100.0%
11047 11066 BSP/QEMU: sumo branchgenericx86-64 on MTURBOT6493.4%none6.6%12914 12666 0.0%none100.0%
11067 BSP/QEMU: sumo branchgenericx86-64 on NUC6100.0%none0.0%none0.0%none100.0%
11046 11068 BSP/QEMU: sumo branchgenericx86-64 on NUC798.9%none1.1%12914 0.0%none100.0%
11073 BSP/QEMU: sumo branchgenericx86-64-WIC on MTURBOT6497.9%none0.0%none2.1%none100.0%
11074 BSP/QEMU: sumo branchgenericx86-64-WIC on NUC697.9%none0.0%none2.1%none100.0%
11075 BSP/QEMU: sumo branchgenericx86-64-WIC on NUC797.9%none0.0%none2.1%none100.0%
11072 BSP/QEMU: sumo branchgenericx86-WIC on MTURBOT3297.9%none0.0%none2.1%none100.0%
11050 11071 BSP/QEMU: sumo branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
11051 11058 11078 BSP/QEMU: sumo branchqemu-x86100.0%none0.0%none0.0%none100.0%
11052 11059 11079 BSP/QEMU: sumo branchqemu-x86-64100.0%none0.0%none0.0%none100.0%
11077 BSP/QEMU: sumo branchqemu-x86-64-WIC100.0%none0.0%none0.0%none100.0%
11076 BSP/QEMU: sumo branchqemu-x86-WIC100.0%none0.0%none0.0%none100.0%
11053 11060 11080 BSP/QEMU: sumo branchqemuarm96.3%none3.7%12499 0.0%none100.0%
11054 11061 11081 BSP/QEMU: sumo branchqemuarm64100.0%none0.0%none0.0%none100.0%
11055 11062 11082 BSP/QEMU: sumo branchqemumips100.0%none0.0%none0.0%none100.0%
11056 11063 11083 BSP/QEMU: sumo branchqemumips64100.0%none0.0%none0.0%none100.0%
11057 11064 11084 BSP/QEMU: sumo branchqemuppc100.0%none0.0%none0.0%none100.0%
11114 Runtime: sumo branchuser-preference (any supported environment)100.0%none0.0%none0.0%none100.0%
11045 Package-Management-Issues: sumo branchAny Supported Distro100.0%none0.0%none0.0%none100.0%
Total: 46


Bug Status


Test Case Status Report
StatusCount
BLOCKED5
FAILED10
PASSED3374
Total: 3389

Bugs Found during QA Test

New issues Found during QA Test & Community

"None"


High and M+ issuesFound during QA Test (NOT NEW)

BSPs
12666[2.5 M3 RC1] BSP-QEMU:core-image-sato-sdk-genericx86-64.hddimg unable to connect to console with serial on Minnowboard TurbotMedium+Anuj Mittal 2.6 M2RESOLVEDFIXED---2018-04-08
BSPs
12914[2.6 M2 RC1][Meta-Intel 10.0][NUC7 & Mturbot][Test case 197] reboot command missing while installing imageMedium+Armin Kuster 2.5.3ACCEPTED---2018-09-04
Runtime
12991[2.6 M4 RC1][Build-Appliance] Bitbake build-appliance-image getting failed during building image due to webkitgtk packageMedium+Richard Purdie 2.7 M3ACCEPTED---2018-11-09

Other Bugs found during QA Test (NOT NEW)

---
12499[2.5 M1 RC3] qemuarm: failed to shutdownLowJon Mason FutureNEW---2018-01-18

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestoneor release which are Medium+ or High and are not targetedfor the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of thisweek's Yocto Bug Trend charts can be accessed on thefollowing [1]

Build Performance Test

Detailed description on how performance test is run can be found on the Performance Test wiki. The below charts are displaying the build performance trend for 2.5.1rc to 2.5.2 rc1. Performance execution has had 2 stages, one executed from 1.6 to 2.1 in RO and executed from 1.6 to 2.1in RO and are differences between HW used in bothlocations, hence the numbers are different.

Below chars are showing the latestperformance results taken from the current milestone.

Performance Charts

WW50-2k18 - table 2.5.2.rc1.PNG

pTest for genericx86-64 on NUC

open bugs

OE-Core
13075[2.5.2 RC1] perl ptest failureMedium+Ross Burton 2.5.3NEW---2018-12-12
OE-Core
13076[2.5.2 RC1] python ptest failureUndecidedRoss Burton ---RESOLVEDWORKSFORME---2018-12-12
OE-Core
13077[2.5.2 RC1] tcl ptest failureMediumRoss Burton 2.5.3NEW---2018-12-12

results

< Archive


Ran on a NUC and compared with 2.5.1_rc1

< FULL_LOG

Failures
 % increased
 % decreased
Module not executed
Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.0 100.0
attr 143 143 0 100.0 100.0
bash 79 77 2 97.47 97.47
bluez5 7 7 0 100.0 100.0
busybox 524 514 10 98.09 97.33 Up rate.png
bzip2 6 6 0 100.0 100.0
dbus-test 21 19 2 90.48 90.48
diffstat 440 440 0 100.0 100.0
diffutils 20 20 0 100.0 100.0
e2fsprogs 309 309 0 100.0 100.0
flex 114 114 0 100.0 100.0
gawk 344 344 0 100.0 100.0
gdbm 25 25 0 100.0 100.0
gdk-pixbuf 20 20 0 100.0 100.0
glib-2.0 253 253 0 100.0 100.0
gzip 21 21 0 100.0 100.0
kbd 7 7 0 100.0 100.0
libpcre 3 3 0 100.0 100.0
libxml-parser-perl 15 15 0 100.0 100.0
libxml2 134 134 0 100.0 100.0
lttng-tools 2089 2020 69 96.7 96.7
lzo 5 5 0 100.0 100.0
nettle 93 93 0 100.0 100.0
openssh 60 60 0 100.0 100.0
openssl 54 54 0 100.0 100.0
pango 12 12 0 100.0 100.0
perl 2286 2284 2 99.91 100.0 Down rate.png
python 15327 15229 98 99.36 99.39 Down rate.png
quilt 57 57 0 100.0 100.0
sed 86 86 0 100.0 100.0
slang 97 97 0 100.0 100.0
strace 374 344 30 91.98 91.98
tcl 148 147 1 99.32 100.0 Down rate.png
util-linux 481 469 12 97.51 97.5 Up rate.png
valgrind 559 300 259 53.67 53.67
zlib 2 2 0 100.0 100.0

Detailed Report

BitBake

Status

Bugs

**No results**

Agenda

Package Management Issues

Status

Bugs

**No results**

Agenda

BSPs

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

OE-Core

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda