WW25 - 2018-06-22- Full Test Cycle - 2.3.4 rc1

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 
 Test type: Full Pass Test
 Branch: pyro
 Build name: yocto-2.3.4.rc1
 poky commit: ebb42af2829edfca1a23c7a51a431c656ffc2090 
 Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.3.4.rc1
 Distributions tested: 
      CentOs 7
      Debian 9
      Fedora 26
      Fedora 27
      OpenSuse 42.3
      Ubuntu 16.04
      Ubuntu 17.10
  
Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
9555 9556 eSDK: pyro branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
9553 9554 CROPS: pyro branchUbuntu 16.0488.9%none11.1%11431 0.0%none100.0%
9549 9551 ADT: pyro branchFedora 26 i68690.9%none5.8%12784 3.3%none100.0%
9550 9552 ADT: pyro branchUbuntu 16.04 x86_6490.9%none5.8%12784 3.3%none100.0%
9557 Kernel: pyro branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
9560 Eclipse-Plugin: pyro branchNeon on Ubuntu 1604100.0%none0.0%none0.0%none100.0%
9559 Eclipse-Plugin: pyro branchOxygen on Ubuntu 1604100.0%none0.0%none0.0%none100.0%
9539 Meta-yocto: pyro branchUbuntu 16.04 x86_6491.3%none8.7%12846 0.0%none100.0%
9547 Build-Appliance: pyro branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
9548 Toaster: pyro branchUbuntu 16.04 x86_6498.6%none1.4%12591 0.0%none100.0%
9540 OE-Core: pyro branchCentOS 7 x86_64100.0%none0.0%none0.0%none100.0%
9541 OE-Core: pyro branchDebian 9 x86_64100.0%none0.0%none0.0%none100.0%
9542 OE-Core: pyro branchFedora 26 x86_64100.0%none0.0%none0.0%none100.0%
9543 OE-Core: pyro branchFedora 27 x86_64100.0%none0.0%none0.0%none100.0%
9544 OE-Core: pyro branchOpenSuse 42.3 x86_64100.0%none0.0%none0.0%none100.0%
9545 OE-Core: pyro branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
9546 OE-Core: pyro branchUbuntu 17.10 x86_64100.0%none0.0%none0.0%none100.0%
9508 BitBake: pyro branchUbuntu 16.04 x86_6475.0%none25.0%12771 12670 0.0%none100.0%
9511 9531 BSP/QEMU: pyro branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
9512 9532 BSP/QEMU: pyro branchEdgeRouter100.0%none0.0%none0.0%none100.0%
9510 9528 BSP/QEMU: pyro branchgenericx86 on MMAX32bit96.7%none3.3%12590 12783 12770 0.0%none100.0%
9529 BSP/QEMU: pyro branchgenericx86-64 on MMAX64bit97.8%none2.2%6584 0.0%none100.0%
9509 9530 BSP/QEMU: pyro branchgenericx86-64 on NUC696.6%none3.4%12590 12783 6584 0.0%none100.0%
9535 BSP/QEMU: pyro branchgenericx86-64-WIC on MMAX64bit100.0%none0.0%none0.0%none100.0%
9536 BSP/QEMU: pyro branchgenericx86-64-WIC on NUC695.8%none2.1%6584 2.1%none100.0%
9534 BSP/QEMU: pyro branchgenericx86-WIC on MMAX32bit100.0%none0.0%none0.0%none100.0%
9513 9533 BSP/QEMU: pyro branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
9514 9521 BSP/QEMU: pyro branchqemu-x8698.1%none1.9%12783 0.0%none100.0%
9515 9522 BSP/QEMU: pyro branchqemu-x86-6498.1%none1.9%12783 0.0%none100.0%
9538 BSP/QEMU: pyro branchqemu-x86-64-WIC0.0%none100.0%12812 0.0%none100.0%
9537 BSP/QEMU: pyro branchqemu-x86-WIC0.0%none100.0%12812 0.0%none100.0%
9516 9523 BSP/QEMU: pyro branchqemuarm94.4%none5.6%12499 12783 0.0%none100.0%
9517 9524 BSP/QEMU: pyro branchqemuarm6496.2%none3.8%11288 12783 0.0%none100.0%
9518 9525 BSP/QEMU: pyro branchqemumips98.1%none1.9%12783 0.0%none100.0%
9519 9526 BSP/QEMU: pyro branchqemumips6498.1%none1.9%12783 0.0%none100.0%
9520 9527 BSP/QEMU: pyro branchqemuppc98.1%none1.9%12783 0.0%none100.0%
9558 Runtime: pyro branchuser-preference (any supported environment)88.9%none11.1%12849 0.0%none100.0%
Total: 37


Bug Status


Test Case Status Report
StatusCount
BLOCKED9
FAILED135
PASSED2527
Total: 2671

Bugs Found during QA Test

New issues Found during QA Test & Community

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
Meta-yocto
12846Runqemu MACHINE values need better error reportingMediumJagadeesh Krishnanjanappa 2.6RESOLVEDFIXED---2018-07-09
OE-Core
12847[2.3.4 rc1] valgrind ptest failedUndecidedRoss Burton ---RESOLVEDDUPLICATE---2018-07-09
12848[2.3.4 rc1] glib-2.0 ptest failedUndecidedRoss Burton ---RESOLVEDFIXED---2018-07-09


High and M+ issues Found during QA Test (NOT NEW)

IDSummaryPAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
12590[2.4.2 rc2] No. of Chars/sec For Yocto Image is very Low To that of UbuntuMedium+Ross BurtonAshwin Soman2.4.4RESOLVEDFIXED---2018-03-08

Other Bugs found during QA Test (NOT NEW)

IDSummary (6 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
12670[2.5 M3 RC1]MANUAL_BitBake:test dependancy explorer is launched with failed logsUndecidedStephano Cetola 2.6RESOLVEDFIXED---2018-04-10
BSPs
12770[2.4.3 RC2][Test case-1059]AUTO_BSP:Check logs for errors and failuresUndecidedBruce Ashfield ---RESOLVEDDUPLICATE---2018-06-04
CROPS
11431Crops-ESDK: devtool is unable to add recipe for nodejs-native recipeMediumAnuj Mittalnagarajagopal2.7IN PROGRESS DESIGN COMPLETE---2017-04-27
OE-Core
12499[2.5 M1 RC3] qemuarm: failed to shutdownLowJon Mason FutureNEW---2018-01-18
Other YP Layers
12849[Bug][2.3.4.rc1][compliance test][LSB Test] "lsb-setup-4.1.0-1.noarch.rpm" not getting downloaded, LSB Test has failedUndecidedPaul Eggleton ---RESOLVEDDUPLICATE---2018-07-09
Toaster
12591[2.4.2 rc2] Run again button from all builds page must run the specified taskMediumDavid ReynaToaster default assignee2.99NEW---2018-03-08

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestoneor release which are Medium+ or High and are not targetedfor the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of thisweek's Yocto Bug Trend charts can be accessed on thefollowing [1]

Build Performance Test

Detailed description on how performance test is run canbe found on the Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.3 M3. Performance execution has had 3 stages, one executed from 1.6 to 2.1 in RO, second executed from 2.1 to 2.5 m1 rc3 in GDC, third executed from 2.5 m2 rc1 onward in Linux Foundation, there are differences between HW used in all locations, hence the numbers are different.

Below chars are showing the latestperformance results taken from the current milestone.

Performance Charts

Note: performance test results in below are both executed from the same machines in Linux Foundation.

Ww28-2k18 - table.png

pTest for genericx86-64 on NUC

open bugs

IDSummary (2 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
12847[2.3.4 rc1] valgrind ptest failedUndecidedRoss Burton ---RESOLVEDDUPLICATE---2018-07-09
12848[2.3.4 rc1] glib-2.0 ptest failedUndecidedRoss Burton ---RESOLVEDFIXED---2018-07-09

results

< Archive


Ran on a NUC and compared with 2.3.3.rc1

< FULL_LOG

Failures
 % increased
 % decreased
Module not executed
Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.0 100.0
attr 143 143 0 100.0 100.0
bash 76 72 4 94.74 94.74
bluez5 25 25 0 100.0 100.0
busybox 487 480 7 98.56 98.07 Up rate.png
bzip2 6 6 0 100.0 100.0
dbus-test 15 4 11 26.67 26.67
diffstat 440 440 0 100.0 100.0
diffutils 17 17 0 100.0 100.0
e2fsprogs 1 0 1 0.0 0.0
flex 114 114 0 100.0 100.0
gawk 298 297 1 99.66 99.66
gdbm 22 22 0 100.0 100.0
gdk-pixbuf 20 20 0 100.0 100.0
glib-2.0 283 276 7 97.53 99.6 Down rate.png
kbd 7 7 0 100.0 100.0
libpcre 3 3 0 100.0 100.0
libxml2 39 36 3 92.31 92.31
lttng-tools 2089 2020 69 96.7 96.65 Up rate.png
nettle 88 87 1 98.86 98.86
openssh 34 33 1 97.06 97.06
openssl 53 53 0 100.0 100.0
pango 12 12 0 100.0 100.0
perl 2286 2285 1 99.96 99.96
perl5 2286 2285 1 99.96 99.96
python 32230 32112 118 99.63 99.61 Up rate.png
quilt 57 57 0 100.0 100.0
strace 335 334 1 99.7 99.7
tcl 148 147 1 99.32 98.65 Up rate.png
util-linux 108 106 2 98.15 98.15
valgrind 553 295 258 53.35 53.53 Down rate.png
zlib 2 2 0 100.0 100.0


Detailed Report

BitBake

Status

Bugs

Agenda

BSPs

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Meta-yocto

Status

Bugs

Agenda

OE-Core

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda