WW27 - 2018-07-02 - Full Test Cycle 2.6 M1 rc1

From Yocto Project
Jump to: navigation, search

Contents

Testing data

 
 Test type: Full Pass Test
 Branch: master 
 Build name: yocto-2.6_M1.rc1
 poky commit: ddbd7b0cd6580ee26e11aa87e426fb294b372d64 
 Autobuilder images repository: https://autobuilder.yocto.io/pub/releases/yocto-2.6_M1.rc1
 Distributions tested: 
      CentOs 7
      Debian 9
      Fedora 26
      Fedora 27
      OpenSuse 42.3
      Ubuntu 16.04
      Ubuntu 17.10
  
Testing Summary Report
Test RunTest PlanEnvironmentPassedOther issuesFailedFailing bugsBlockedBlocking bugsComplete
9759 9760 eSDK: master branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
9756 CROPS: master branchUbuntu 16.0483.3%none16.7%11431 0.0%none100.0%
9754 ADT: master branchFedora 26 i686100.0%none0.0%none0.0%none100.0%
9747 9764 ADT: master branchqemu-x8690.0%none0.0%none0.0%none90.0%
9748 ADT: master branchqemu-x86-6485.7%none0.0%none0.0%none85.7%
9749 9765 ADT: master branchqemuarm90.0%none0.0%none0.0%none90.0%
9750 ADT: master branchqemuarm6485.7%none0.0%none0.0%none85.7%
9751 9766 ADT: master branchqemumips90.0%none0.0%none0.0%none90.0%
9752 ADT: master branchqemumips6485.7%none0.0%none0.0%none85.7%
9753 9767 ADT: master branchqemuppc90.0%none0.0%none0.0%none90.0%
9755 ADT: master branchUbuntu 16.04 x86_64100.0%none0.0%none0.0%none100.0%
9761 Kernel: master branchUbuntu 16.04100.0%none0.0%none0.0%none100.0%
9758 Eclipse-Plugin: master branchNeon - Fedora 24100.0%none0.0%none0.0%none100.0%
9757 Eclipse-Plugin: master branchOxygen - Fedora26100.0%none0.0%none0.0%none100.0%
9736 Meta-yocto: master branchUbuntu 16.04 x86_6480.0%none20.0%12803 12802 0.0%none100.0%
9744 Build-Appliance: master branchVMWare Player on Ubuntu0.0%none100.0%12804 0.0%none100.0%
9745 9746 Toaster: master branchUbuntu 16.04 x86_6496.8%none3.2%12813 12591 0.0%none100.0%
9737 OE-Core: master branchCentOS 7 x86_6499.7%none0.0%none0.0%none99.7%
9738 OE-Core: master branchDebian 9 x86_6499.7%none0.0%none0.0%none99.7%
9739 OE-Core: master branchFedora 26 x86_6499.7%none0.0%none0.0%none99.7%
9740 OE-Core: master branchFedora 27 x86_6499.7%none0.0%none0.0%none99.7%
9741 OE-Core: master branchOpenSuse 42.3 x86_6499.7%none0.0%none0.0%none99.7%
9742 OE-Core: master branchUbuntu 16.04 x86_6499.7%none0.0%none0.0%none99.7%
9743 OE-Core: master branchUbuntu 17.10 x86_6499.7%none0.0%none0.0%none99.7%
9696 BitBake: master branchUbuntu 16.04 x86_6466.7%none16.7%12670 16.7%none100.0%
9699 9720 BSP/QEMU: master branchBeaglebone Black91.4%none8.6%12795 0.0%none100.0%
9700 9721 BSP/QEMU: master branchEdgeRouter0.0%none0.0%none100.0%12790 100.0%
9716 BSP/QEMU: master branchgenericx86 on MTURBOT3297.9%none2.1%12763 0.0%none100.0%
9698 9717 BSP/QEMU: master branchgenericx86-64 on MTURBOT6486.8%none12.1%12832 12666 12763 1.1%none100.0%
9718 BSP/QEMU: master branchgenericx86-64 on NUC697.9%none2.1%12763 0.0%none100.0%
9697 9719 BSP/QEMU: master branchgenericx86-64 on NUC781.1%none13.3%12832 12763 5.6%none100.0%
9724 BSP/QEMU: master branchgenericx86-64-WIC on MTURBOT64100.0%none0.0%none0.0%none100.0%
9725 BSP/QEMU: master branchgenericx86-64-WIC on NUC697.9%none0.0%none2.1%none100.0%
9726 BSP/QEMU: master branchgenericx86-64-WIC on NUC797.9%none0.0%none2.1%none100.0%
9723 BSP/QEMU: master branchgenericx86-WIC on MTURBOT32100.0%none0.0%none0.0%none100.0%
9701 9722 BSP/QEMU: master branchMPC8315e-rdb100.0%none0.0%none0.0%none100.0%
9702 9709 9729 BSP/QEMU: master branchqemu-x86100.0%none0.0%none0.0%none100.0%
9703 9710 9730 BSP/QEMU: master branchqemu-x86-64100.0%none0.0%none0.0%none100.0%
9728 BSP/QEMU: master branchqemu-x86-64-WIC97.9%none0.0%none0.0%none97.9%
9727 BSP/QEMU: master branchqemu-x86-WIC97.9%none0.0%none0.0%none97.9%
9704 9711 9731 BSP/QEMU: master branchqemuarm96.4%none3.6%12499 0.0%none100.0%
9705 9712 9732 BSP/QEMU: master branchqemuarm6496.4%12801 1.8%11288 0.0%none98.2%
9706 9713 9733 BSP/QEMU: master branchqemumips94.5%none0.0%none0.0%none94.5%
9707 9714 9734 BSP/QEMU: master branchqemumips6494.5%none0.0%none0.0%none94.5%
9708 9715 9735 BSP/QEMU: master branchqemuppc98.2%12806 1.8%12806 0.0%none100.0%
9762 Runtime: master branchuser-preference (any supported environment)100.0%none0.0%none0.0%none100.0%
9763 Package-Management-Issues: master branchAny Supported Distro100.0%none0.0%none0.0%none100.0%
Total: 47


Bug Status


Test Case Status Report
StatusCount
BLOCKED71
FAILED46
IDLE23
PASSED3275
RUNNING2
Total: 3417

Bugs Found during QA Test

New issues Found during QA Test & Community

IDSummary (9 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
BSPs
12790[2.6 M1] Edgerouter can not bootUndecidedKevin HaoYi Zhao---VERIFIEDFIXEDRegression (Used to work)2018-06-15
12806[2.6 M1 rc1 ][BSP][Test Run 9708]: Qemuppc image was not booting with graphic even though runtime/xorg.py test passedMedium+Randy MacLeod 2.6 M3RESOLVEDWORKSFORME---2018-06-21
12832[ 2.6 M1 rc1 ][BSP][Test case 267]: audio and video does not play in media player[Mturbot x86-64 and NUC7]Medium+Anuj Mittal 2.6 M2RESOLVEDFIXED---2018-06-29
General Runtime
12795[2.6 M1] gcc/g++ doesn't work on beaglebone blackUndecidedKhem RajYi Zhao---VERIFIEDFIXEDRegression (Used to work)2018-06-19
Meta-yocto
12802[Yocto-2.6_M1.RC1] Crosstap doesn't work on 2.6 M1 RC1MediumManish Das 2.7RESOLVEDFIXED---2018-06-20
OE-Core
12836[2.6 M1 RC1] busybox ptest failedMedium+Manish Das 2.6 M2RESOLVEDNOTABUGRegression (Found by QA test)2018-07-02
12837[2.6 M1 RC1] valgrind ptest failedMedium+Randy MacLeod 2.6 M4RESOLVEDFIXEDRegression (Found by QA test)2018-07-02
Runtime
12804[QA 2.6 M1 rc1 ][Build Appliance]: proxy issue in VM wareMedium+Anuj Mittal 2.6 M3RESOLVEDFIXEDRegression (Used to work)2018-06-20
Toaster
12813[QA 2.6 M1 rc1 ][Toaster]:Build stopped without error, to terminate by ctrl+cMediumDavid ReynaToaster default assignee2.6 M4RESOLVEDFIXED---2018-06-25


High and M+ issuesFound during QA Test (NOT NEW)

"None"

Other Bugs found during QA Test (NOT NEW)

IDSummary (4 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
---
12763[Meta-Intel 2.5 M4 RC1] [test_parselogs] Error present in parselog for core-image-sato-sdk image on NUC7MediumAnuj Mittal 2.6RESOLVEDFIXED---2018-05-31
BSPs
12666[2.5 M3 RC1] BSP-QEMU:core-image-sato-sdk-genericx86-64.hddimg unable to connect to console with serial on Minnowboard TurbotMedium+Anuj Mittal 2.6 M2RESOLVEDFIXED---2018-04-08
CROPS
11431Crops-ESDK: devtool is unable to add recipe for nodejs-native recipeMediumAnuj Mittalnagarajagopal2.7IN PROGRESS DESIGN COMPLETE---2017-04-27
OE-Core
12499[2.5 M1 RC3] qemuarm: failed to shutdownLowJon Mason FutureNEW---2018-01-18

Current community Open Bugs - Medium+/High

Overview of the open bugs opened against current milestoneor release which are Medium+ or High and are not targetedfor the next milestones

IDPSummaryAssigneeQAMilestoneStatuscf_regression_typeCreated
  0      

Bug Trend

An overview of thisweek's Yocto Bug Trend charts can be accessed on thefollowing [1]

Build Performance Test

Detailed description on how performance test is run canbe found on the Performance Test wiki. The below charts are displaying the build performance trend for 2.1 M4 to 2.3 M3. Performance execution has had 3 stages, one executed from 1.6 to 2.1 in RO, second executed from 2.1 to 2.5 m1 rc3 in GDC, third executed from 2.5 m2 rc1 onward in Linux Foundation, there are differences between HW used in all locations, hence the numbers are different.

Below chars are showing the latestperformance results taken from the current milestone.

Performance Charts

Note: performance test results in below are both executed from the same machines in Linux Foundation.

Ww27-2k18 - table.png

pTest for genericx86-64 on NUC

open bugs

IDSummary (2 tasks) PAssigneeQAMilestoneStatusResolutioncf_regression_typeCreated
OE-Core
12836[2.6 M1 RC1] busybox ptest failedMedium+Manish Das 2.6 M2RESOLVEDNOTABUGRegression (Found by QA test)2018-07-02
12837[2.6 M1 RC1] valgrind ptest failedMedium+Randy MacLeod 2.6 M4RESOLVEDFIXEDRegression (Found by QA test)2018-07-02

results

< Archive


Ran on a NUC and compared with 2.5_M4_rc1

< FULL_LOG

Failures
 % increased
 % decreased
Module not executed
Results
Package Name Tests ran Passed Failed Passrate Last Passrate
acl 380 380 0 100.0 100.0
attr 143 143 0 100.0 100.0
bash 79 76 3 96.2 96.2
bluez5 7 7 0 100.0 100.0
busybox 524 510 14 97.33 98.09 Down rate.png
bzip2 6 6 0 100.0 100.0
dbus-test 21 19 2 90.48 90.48
diffstat 440 440 0 100.0 100.0
diffutils 20 20 0 100.0 100.0
e2fsprogs 309 309 0 100.0 100.0
flex 114 114 0 100.0 100.0
gawk 344 344 0 100.0 100.0
gdbm 25 25 0 100.0 100.0
gdk-pixbuf 20 20 0 100.0 100.0
glib-2.0 256 256 0 100.0 100.0
gzip 21 21 0 100.0 100.0
kbd 7 7 0 100.0 100.0
libpcre 3 3 0 100.0 100.0
libxml-parser-perl 15 15 0 100.0 100.0
libxml2 134 134 0 100.0 100.0
lttng-tools 2089 2020 69 96.7 96.65 Up rate.png
lzo 5 5 0 100.0 100.0
nettle 93 93 0 100.0 100.0
openssh 50 50 0 100.0 100.0
openssl 53 53 0 100.0 100.0
pango 12 12 0 100.0 100.0
perl 2298 2298 0 100.0 100.0
python 30540 30354 186 99.39 99.37 Up rate.png
quilt 57 57 0 100.0 100.0
sed 86 86 0 100.0 100.0
slang 97 97 0 100.0 100.0
strace 389 375 14 96.4 91.98 Up rate.png
tcl 45 45 0 100.0 98.65 Up rate.png
util-linux 479 466 13 97.29 96.82 Up rate.png
valgrind 558 274 284 49.1 53.49 Down rate.png
zlib 2 2 0 100.0 100.0


Detailed Report

BitBake

Status

Bugs

Agenda

BSPs

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Meta-yocto

Status

Bugs

Agenda

OE-Core

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda

Status

Bugs

**No results**

Agenda