WW19 - 2018-05-10- Full Test Cycle - 2.5 M4 rc1

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 
 Test type: Full Pass Test
 Branch: master 
 Build name: yocto-2.5_M4.rc1
 poky commit: da3625c52e1ab8985fba4fc3d133edf92142f182
 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.5.rc1/
 Distributions tested: 
      CentOs 7
      Debian 9
      Fedora 26
      Fedora 27
      OpenSuse 42.3
      Ubuntu 16.04
      Ubuntu 17.10
  
Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
9424 9425 eSDK: master branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
9421 CROPS: master branchUbuntu 16.0483.3%12726 16.7%11431 0.0%none100.0%
9436 ADT: master branchFedora 26 i686100.0%none0.0%none0.0%none100.0%
9412 9438 ADT: master branchqemu-x86100.0%none0.0%none0.0%none100.0%
9413 ADT: master branchqemu-x86-64100.0%none0.0%none0.0%none100.0%
9414 9439 ADT: master branchqemuarm100.0%none0.0%none0.0%none100.0%
9415 ADT: master branchqemuarm64100.0%none0.0%none0.0%none100.0%
9416 9440 ADT: master branchqemumips100.0%none0.0%none0.0%none100.0%
9417 ADT: master branchqemumips64100.0%none0.0%none0.0%none100.0%
9418 9441 ADT: master branchqemuppc100.0%none0.0%none0.0%none100.0%
9437 ADT: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
9426 Kernel: master branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
9423 Eclipse-Plugin: master branchNeon - Fedora 24100.0%none0.0%none0.0%none100.0%
9422 Eclipse-Plugin: master branchOxygen - Fedora26100.0%none0.0%none0.0%none100.0%
9401 Meta-yocto: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
9409 Build-Appliance: master branchVMWare Player on Ubuntu100.0%none0.0%none0.0%none100.0%
9410 9411 Toaster: master branchUbuntu 16.04 x86_6498.9%none1.1%12591 0.0%none100.0%
9402 OE-Core: master branchCentOS 7 x86_6499.3%none0.3%12736 0.0%none99.7%
9403 OE-Core: master branchDebian 9 x86_6499.7%none0.0%none0.0%none99.7%
9404 OE-Core: master branchFedora 26 x86_6499.7%none0.0%none0.0%none99.7%
9405 OE-Core: master branchFedora 27 x86_6499.7%none0.0%none0.0%none99.7%
9406 OE-Core: master branchOpenSuse 42.3 x86_6499.7%none0.0%none0.0%none99.7%
9407 OE-Core: master branchUbuntu 16.04 x86_6499.7%none0.0%none0.0%none99.7%
9408 OE-Core: master branchUbuntu 17.10 x86_6499.7%none0.0%none0.0%none99.7%
9361 BitBake: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
9364 9385 BSP/QEMU: master branchBeaglebone Black100.0%12712 0.0%none0.0%none100.0%
9365 9386 BSP/QEMU: master branchEdgeRouter100.0%none0.0%none0.0%none100.0%
9381 BSP/QEMU: master branchgenericx86 on MTURBOT3297.9%none2.1%12602 0.0%none100.0%
9363 9382 BSP/QEMU: master branchgenericx86-64 on MTURBOT6496.7%none3.3%12666 0.0%none100.0%
9383 BSP/QEMU: master branchgenericx86-64 on NUC6100.0%none0.0%none0.0%none100.0%
9362 9384 BSP/QEMU: master branchgenericx86-64 on NUC797.8%none0.0%none2.2%none100.0%
9389 BSP/QEMU: master branchgenericx86-64-WIC on MTURBOT6495.8%none2.1%12602 2.1%none100.0%
9390 BSP/QEMU: master branchgenericx86-64-WIC on NUC695.8%none2.1%12602 2.1%none100.0%
9391 BSP/QEMU: master branchgenericx86-64-WIC on NUC795.8%none2.1%12602 2.1%none100.0%
9388 BSP/QEMU: master branchgenericx86-WIC on MTURBOT3297.9%none0.0%none2.1%none100.0%
9366 9387 BSP/QEMU: master branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
9367 9374 9394 BSP/QEMU: master branchqemu-x86100.0%none0.0%none0.0%none100.0%
9368 9375 9395 BSP/QEMU: master branchqemu-x86-64100.0%none0.0%none0.0%none100.0%
9393 BSP/QEMU: master branchqemu-x86-64-WIC97.9%none0.0%none0.0%none97.9%
9392 BSP/QEMU: master branchqemu-x86-WIC97.9%none0.0%none0.0%none97.9%
9369 9376 9396 BSP/QEMU: master branchqemuarm96.3%none3.7%12499 0.0%none100.0%
9370 9377 9397 BSP/QEMU: master branchqemuarm64100.0%none0.0%none0.0%none100.0%
9371 9378 9398 BSP/QEMU: master branchqemumips100.0%none0.0%none0.0%none100.0%
9372 9379 9399 BSP/QEMU: master branchqemumips64100.0%none0.0%none0.0%none100.0%
9373 9380 9400 BSP/QEMU: master branchqemuppc100.0%none0.0%none0.0%none100.0%
9427 Runtime: master branchuser-preference (any supported environment)100.0%none0.0%none0.0%none100.0%
9428 Package-Management-Issues: master branchAny Supported Distro100.0%none0.0%none0.0%none100.0%
Total: 47


Bug Status


Test Case Status Report
StatusCount
BLOCKED6
FAILED12
IDLE9
PASSED3380
Total: 3407

Bugs Found during QA Test

New issues Found during QA Test & Community

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
12712[beagleboneblack] demsage error: platform regulatory.0: Direct firmware load for regulatory.db failed with error -2Medium+Kevin HaoYi Zhao0.0.0RESOLVEDNOTABUGRegression (Used to work)2018-04-25
eSDK
12725devtool sdk-install unable to add component for libxml2MediumPaul EggletonFrancisco Pedraza2.7NEW---2018-05-01
Runtime Testing
127362.5 rc1: CentOS 7 oe-selftest fail at runtime_test.TestImage.test_testimage_dnfMedium+Mohamad Noor AlimMohamad Noor Alim2.6 M1RESOLVEDNOTABUG---2018-05-07


High and M+ issuesFound during QA Test (NOT NEW)

IDSummary (3 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
12602[meta-intel 2.4.2 rc2] [test_parselogs] Error Present in dmesg logs for core-image-lsb-sdk image on NUC6Medium+nagarajagopalnagarajagopal2.4.4RESOLVEDFIXED---2018-03-13
12666[2.5 M3 RC1] BSP-QEMU:core-image-sato-sdk-genericx86-64.hddimg unable to connect to console with serial on Minnowboard TurbotMedium+Anuj Mittal 2.6 M2RESOLVEDFIXED---2018-04-08
CROPS
11431Crops-ESDK: devtool is unable to add recipe for nodejs-native recipeMediumAnuj Mittalnagarajagopal2.7IN PROGRESS DESIGN COMPLETE---2017-04-27

Other Bugs found during QA Test (NOT NEW)

IDSummary (2 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
12499[2.5 M1 RC3] qemuarm: failed to shutdownLowJon Mason FutureNEW---2018-01-18
Toaster
12591[2.4.2 rc2] Run again button from all builds page must run the specified taskMediumDavid ReynaToaster default assignee2.99NEW---2018-03-08

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestoneor release which are Medium+ or High and are not targetedfor the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of thisweek's Yocto Bug Trend charts can be accessed on thefollowing [1]

Build Performance Test

Detailed description on how performance test is run canbe found on the Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.3 M3. Performance execution has had 3 stages, one executed from 1.6 to 2.1 in RO, second executed from 2.1 to 2.5 m1 rc3 in GDC, third executed from 2.5 m2 rc1 onward in Linux Foundation, there are differences between HW used in all locations, hence the numbers are different.

Below chars are showing the latestperformance results taken from the current milestone.

Performance Charts

Note: performance test results in below are both executed from the same machines in Linux Foundation.

Ww19-2k18 - table.png

pTest for genericx86-64 on NUC

open bugs

"None"

results

< Archive


Ran on a NUC and compared with 2.5_m3_rc1

< FULL_LOG

Failures
 % increased
 % decreased
Module not executed
Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.0 100.0
attr 143 143 0 100.0 100.0
bash 79 76 3 96.2 96.2
bluez5 7 7 0 100.0 100.0
busybox 524 514 10 98.09 97.33 Up rate.png
bzip2 6 6 0 100.0 100.0
dbus-test 21 19 2 90.48 90.48
diffstat 440 440 0 100.0 100.0
diffutils 20 20 0 100.0 100.0
e2fsprogs 309 309 0 100.0 100.0
flex 114 114 0 100.0 100.0
gawk 339 339 0 100.0 100.0
gdbm 25 25 0 100.0 100.0
gdk-pixbuf 20 20 0 100.0 100.0
glib-2.0 253 253 0 100.0 100.0
gzip 21 21 0 100.0 100.0
kbd 7 7 0 100.0 100.0
libpcre 3 3 0 100.0 66.67 Up rate.png
libxml-parser-perl 15 15 0 100.0 100.0
libxml2 134 134 0 100.0 100.0
lttng-tools 2057 1988 69 96.65 96.65
lzo 5 5 0 100.0 100.0
nettle 93 93 0 100.0 100.0
openssh 60 60 0 100.0 100.0
openssl 53 53 0 100.0 100.0
pango 12 12 0 100.0 100.0
perl 2287 2287 0 100.0 100.0
python 30552 30358 194 99.37 99.39 Down rate.png
quilt 57 57 0 100.0 100.0
sed 86 86 0 100.0 100.0
slang 97 97 0 100.0 100.0
strace 374 344 30 91.98 91.98
tcl 148 146 2 98.65 99.32 Down rate.png
util-linux 377 365 12 96.82 96.81 Up rate.png
valgrind 559 299 260 53.49 53.49
zlib 2 2 0 100.0 100.0


Detailed Report

BitBake

Status

Bugs

**No results**

Agenda

BSPs

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Meta-yocto

Status

Bugs

**No results**

Agenda

OE-Core

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda