WW15 - 2018-04-09- Full Test Cycle - 2.5 M3 rc1

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 
 Test type: Full Pass Test
 Branch: master
 Build name: yocto-2.5_M3.rc1
 poky commit: f49ee61422c867516db252054e993df29f775136
 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/yocto-2.5_M3.rc1/ 
 Distributions tested: 
      CentOs 7
      Debian 9
      Fedora 26
      Fedora 27
      OpenSuse 42.3
      Ubuntu 16.04
      Ubuntu 17.10
  
Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
9311 9312 eSDK: master branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
9308 CROPS: master branchUbuntu 16.0483.3%none16.7%11431 0.0%none100.0%
9317 ADT: master branchFedora 26 i686100.0%none0.0%none0.0%none100.0%
9299 9320 ADT: master branchqemu-x86100.0%none0.0%none0.0%none100.0%
9300 ADT: master branchqemu-x86-64100.0%none0.0%none0.0%none100.0%
9301 9321 ADT: master branchqemuarm100.0%none0.0%none0.0%none100.0%
9302 ADT: master branchqemuarm64100.0%none0.0%none0.0%none100.0%
9303 9322 ADT: master branchqemumips100.0%none0.0%none0.0%none100.0%
9304 ADT: master branchqemumips64100.0%none0.0%none0.0%none100.0%
9305 9323 ADT: master branchqemuppc100.0%none0.0%none0.0%none100.0%
9316 ADT: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
9313 Kernel: master branchUbuntu 16.0488.9%none11.1%12641 0.0%none100.0%
9310 9325 Eclipse-Plugin: master branchNeon - Fedora 26100.0%none0.0%none0.0%none100.0%
9309 9324 Eclipse-Plugin: master branchOxygen - Fedora26100.0%none0.0%none0.0%none100.0%
9288 Meta-yocto: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
9296 9326 Build-Appliance: master branchVMWare Player on Ubuntu50.0%none50.0%12637 0.0%none100.0%
9297 9298 Toaster: master branchUbuntu 16.04 x86_6498.9%none1.1%12591 0.0%none100.0%
9289 OE-Core: master branchCentOS 7 x86_64100.0%none0.0%none0.0%none100.0%
9290 OE-Core: master branchDebian 9 x86_64100.0%12655 12647 0.0%none0.0%none100.0%
9291 OE-Core: master branchFedora 26 x86_64100.0%none0.0%none0.0%none100.0%
9292 OE-Core: master branchFedora 27 x86_64100.0%none0.0%none0.0%none100.0%
9293 OE-Core: master branchOpenSuse 42.3 x86_6498.9%none1.1%12665 0.0%none100.0%
9294 OE-Core: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
9295 OE-Core: master branchUbuntu 17.10 x86_64100.0%none0.0%none0.0%none100.0%
9264 BitBake: master branchUbuntu 16.04 x86_6485.7%none14.3%12670 0.0%none100.0%
9257 9279 9329 9350 BSP/QEMU: master branchBeaglebone Black100.0%none0.0%none0.0%none100.0%
9258 9280 9330 9351 BSP/QEMU: master branchEdgeRouter100.0%none0.0%none0.0%none100.0%
9275 9346 BSP/QEMU: master branchgenericx86 on MTURBOT32100.0%none0.0%none0.0%none100.0%
9256 9276 9328 9347 BSP/QEMU: master branchgenericx86-64 on MTURBOT6495.6%none4.4%12590 12666 0.0%none100.0%
9277 9348 BSP/QEMU: master branchgenericx86-64 on NUC699.0%none1.0%12444 0.0%none100.0%
9255 9278 9327 9349 BSP/QEMU: master branchgenericx86-64 on NUC798.9%12048 1.1%12590 0.0%none100.0%
9283 9354 BSP/QEMU: master branchgenericx86-64-WIC on MTURBOT6499.0%none0.0%none0.0%none99.0%
9284 9355 BSP/QEMU: master branchgenericx86-64-WIC on NUC697.9%none0.0%none0.0%none97.9%
9285 9356 BSP/QEMU: master branchgenericx86-64-WIC on NUC799.0%none0.0%none1.0%none100.0%
9282 9353 BSP/QEMU: master branchgenericx86-WIC on MTURBOT320.0%none2.1%12633 97.9%none100.0%
9259 9281 9331 9352 BSP/QEMU: master branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
9260 9268 9332 BSP/QEMU: master branchqemu-x86100.0%none0.0%none0.0%none100.0%
9261 9269 9333 BSP/QEMU: master branchqemu-x86-64100.0%none0.0%none0.0%none100.0%
9287 BSP/QEMU: master branchqemu-x86-64-WIC97.9%none0.0%none0.0%none97.9%
9286 BSP/QEMU: master branchqemu-x86-WIC100.0%none0.0%none0.0%none100.0%
9262 9270 9334 BSP/QEMU: master branchqemuarm93.8%none6.3%12499 0.0%none100.0%
9271 9318 9335 BSP/QEMU: master branchqemuarm6496.9%none3.1%11288 0.0%none100.0%
9265 9272 9336 BSP/QEMU: master branchqemumips100.0%none0.0%none0.0%none100.0%
9266 9273 9337 BSP/QEMU: master branchqemumips64100.0%none0.0%none0.0%none100.0%
9267 9274 9338 BSP/QEMU: master branchqemuppc100.0%none0.0%none0.0%none100.0%
9314 9359 Runtime: master branchuser-preference (any supported environment)100.0%none0.0%none0.0%none100.0%
9315 Package-Management-Issues: master branchAny Supported Distro100.0%none0.0%none0.0%none100.0%
Total: 47


Bug Status


Test Case Status Report
StatusCount
BLOCKED95
FAILED30
IDLE4
PASSED3909
Total: 4038

Bugs Found during QA Test

New issues Found during QA Test & Community

IDSummary (8 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BitBake
12670[2.5 M3 RC1]MANUAL_BitBake:test dependancy explorer is launched with failed logsUndecidedStephano Cetola 2.6RESOLVEDFIXED---2018-04-10
BSPs
12633[2.5 M3 RC1] BSP-QEMU:core-image-lsb-sdk-x86.wic unable to boot on Minnowboard TurbotMedium+California Sullivanyeoh ee peng2.5 M4VERIFIEDFIXED---2018-03-29
12666[2.5 M3 RC1] BSP-QEMU:core-image-sato-sdk-genericx86-64.hddimg unable to connect to console with serial on Minnowboard TurbotMedium+Anuj Mittal 2.6 M2RESOLVEDFIXED---2018-04-08
Build Testing
12647[2.5 M3 rc1][OE-Core:Debian9] recipe xcursor-transport-theme went backwards, breaks package feedsHighPaul EggletonAaron Chan2.5 M4RESOLVEDFIXEDRegression (Found by QA test)2018-04-04
12655[2.5 M3 rc1][OE Core:Debian9] layerappend.LayerAppendTests.test_layer_appends - Testcase 1196: ERRORHighPaul EggletonAaron Chan2.5 M4RESOLVEDFIXEDRegression (Found by QA test)2018-04-04
Kernel
12641[2.5 M3 RC1] Kernel Development Build external modules (hello-mod) test case: Function failed do_rootfsMedium+Juro Bystricky 0.0.0RESOLVEDFIXED---2018-04-02
Runtime
12637[2.5 M3 RC1] Build appliance failed on glibc-initial do_configure when bitbake core-image-minimalMedium+Richard Purdie 2.99RESOLVEDFIXED---2018-03-30
Runtime Testing
12665[2.5 M3 rc1] OpenSUSE42.3 runtime test fail at runtime_test.TextExport, runtime_test.TestImage, runtime_test.TestImage and oe_runmake.HighPaul EggletonMohamad Noor Alim2.5 M4RESOLVEDFIXED---2018-04-08


High and M+ issues Found during QA Test (NOT NEW)

IDSummary (4 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
12444[Test Case 1542] test_2_logrotate fails in MinnowMax with core-image-lsb-sdk-genericx86-64 imageMedium+yeoh ee peng 2.5 M2RESOLVEDINVALID---2017-12-21
12590[2.4.2 rc2] No. of Chars/sec For Yocto Image is very Low To that of UbuntuMedium+Ross BurtonAshwin Soman2.4.4RESOLVEDFIXED---2018-03-08
CROPS
11431Crops-ESDK: devtool is unable to add recipe for nodejs-native recipeMediumAnuj Mittalnagarajagopal2.6RESOLVEDWONTFIX---2017-04-27
Sato
11288X Server fails to start on qemuarm64 (need CONFIG_FB?)Medium+Jon Mason 2.7ACCEPTED---2017-04-03

Other Bugs found during QA Test (NOT NEW)

IDSummary (2 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
12499[2.5 M1 RC3] qemuarm: failed to shutdownLowJon Mason FutureNEW---2018-01-18
Toaster
12591[2.4.2 rc2] Run again button from all builds page must run the specified taskMediumDavid ReynaToaster default assignee2.99NEW---2018-03-08

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestoneor release which are Medium+ or High and are not targetedfor the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of thisweek's Yocto Bug Trend charts can be accessed on thefollowing [1]

Build Performance Test

Detailed description on how performance test is run canbe found on the Performance Test wiki. The belowcharts are displaying the build performance trend for 2.1M4 to 2.3 M3. Performance execution has had 3 stages, one executed from 1.6 to 2.1 in RO, second executed from 2.1 to 2.5 m1 rc3 in GDC, third executed from 2.5 m2 rc1 onward in Linux Foundation, there are differences between HW used in all locations, hence the numbers are different.

Below chars are showing the latestperformance results taken from the current milestone.

Performance Charts

Note: performance test results in below are both executed from the same machines in Linux Foundation.

Ww15-2k18 - table.png

pTest for genericx86-64 on NUC

open bugs

"None"

results

< Archive


Ran on a NUC and compared with 2.5_m2_rc1

< FULL_LOG

Failures
 % increased
 % decreased
Module not executed
Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.0 100.0
attr 143 143 0 100.0 100.0
bash 79 76 3 96.2 96.2
bluez5 7 7 0 100.0 100.0
busybox 524 510 14 97.33 98.09 Down rate.png
bzip2 6 6 0 100.0 100.0
dbus-test 21 19 2 90.48 26.67 Up rate.png
diffstat 440 440 0 100.0 100.0
diffutils 20 20 0 100.0 100.0
e2fsprogs 309 309 0 100.0 85.76 Up rate.png
flex 114 114 0 100.0 100.0
gawk 339 339 0 100.0 100.0
gdbm 25 25 0 100.0 100.0
gdk-pixbuf 20 20 0 100.0 100.0
glib-2.0 253 253 0 100.0 100.0
gzip 21 21 0 100.0 100.0
kbd 7 7 0 100.0 100.0
libpcre 3 2 1 66.67 66.67
libxml-parser-perl 15 15 0 100.0 0 Up rate.png
libxml2 134 134 0 100.0 100.0
lttng-tools 2057 1988 69 96.65 96.65
lzo 5 5 0 100.0 100.0
nettle 93 93 0 100.0 100.0
openssh 60 60 0 100.0 100.0
openssl 53 53 0 100.0 100.0
pango 12 12 0 100.0 100.0
perl 2287 2287 0 100.0 100.0
python 30548 30362 186 99.39 99.35 Up rate.png
quilt 57 57 0 100.0 100.0
sed 86 86 0 100.0 100.0
slang 97 97 0 100.0 100.0
strace 374 344 30 91.98 99.73 Down rate.png
tcl 148 147 1 99.32 99.32
util-linux 376 364 12 96.81 96.81
valgrind 559 299 260 53.49 53.49
zlib 2 2 0 100.0 100.0


Detailed Report

BSPs

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

BitBake

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Meta-yocto

Status

Bugs

**No results**

Agenda

OE-Core

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda