WW16 - 2019-04-18 - Full Test Cycle 2.5.3 RC3: Difference between revisions
From Yocto Project
Jump to navigationJump to search
No edit summary |
(→==) |
||
| Line 168: | Line 168: | ||
QA cycle report for 2.5.3 RC3: | QA cycle report for 2.5.3 RC3: | ||
1. No high milestone defects. | 1. No high milestone defects. | ||
2. Test results are available at following location: | 2. Test results are available at following location: | ||
• For results of all automated tests, refer to results at public AB [1]. | • For results of all automated tests, refer to results at public AB [1]. | ||
• For other test results, refer to attachment [2]. | • For other test results, refer to attachment [2]. | ||
• For test report for test cases run by Intel and WR team, refer attachment [3] | • For test report for test cases run by Intel and WR team, refer attachment [3] | ||
• For full test report, refer attachment [4] | • For full test report, refer attachment [4] | ||
• For ptest results, please refer to results at public AB [5] | • For ptest results, please refer to results at public AB [5] | ||
• For ptest report, refer to attachment [6] | • For ptest report, refer to attachment [6] | ||
3. No new defects are found in this cycle. | 3. No new defects are found in this cycle. | ||
4. Number of existing issues observed in this release is 2- toaster [7] and Build-appliance [8] | 4. Number of existing issues observed in this release is 2- toaster [7] and Build-appliance [8] | ||
5. For ptest, regression data is not available for this release. No timeout issues. | 5. For ptest, regression data is not available for this release. No timeout issues. | ||
6. Test result report on Public AB shows no failures. | 6. Test result report on Public AB shows no failures. | ||
======= Links ======== | ======= Links ======== | ||
Revision as of 03:39, 24 April 2019
Test Result Status Summary (Counts/Percentages sorted by testseries, ID)
Test Series | ID | Passed | Failed | Skipped
beaglebone | manual_bsps-hw_20190415101630 | 22 (48%) | 0 (0%) | 24 (52%) beaglebone | manual_compliance-test_20190415102409 | 4 (67%) | 0 (0%) | 2 (33%) beaglebone | runtime_core-image-sato-sdk_beaglebone-yocto_20190410043531 | 48 (79%) | 1 (2%) | 12 (20%) bitbake | manual_oe-core_20190411155828 | 7 (100%) | 0 (0%) | 0 (0%) build-appliance | manual_build-appliance_20190412091610 | 3 (75%) | 1 (25%) | 0 (0%) crops | sdkext_core-image-sato-sdk_x86_64_qemux86_20190417070541 | 20 (100%) | 0 (0%) | 0 (0%) eclipse-plugin | manual_eclipse-plugin_20190417104509 | 6 (100%) | 0 (0%) | 0 (0%) eclipse-plugin | manual_eclipse-plugin_20190417132839 | 6 (100%) | 0 (0%) | 0 (0%) edgerouter | manual_bsps-hw_20190415105842 | 11 (24%) | 0 (0%) | 35 (76%) edgerouter | manual_compliance-test_20190415110648 | 2 (33%) | 0 (0%) | 4 (67%) edgerouter | runtime_core-image-sato-sdk_edgerouter_20190411013527 | 47 (77%) | 2 (3%) | 12 (20%) kernel-dev | manual_kernel-configuration_20190416165928 | 9 (100%) | 0 (0%) | 0 (0%) minnowboard | manual_bsps-hw_20190416094344 | 43 (100%) | 0 (0%) | 0 (0%) minnowboard | runtime_core-image-lsb-sdk_genericx86-64_20190416030314 | 48 (79%) | 0 (0%) | 13 (21%) minnowboard | runtime_core-image-lsb-sdk_genericx86_20190416094619 | 48 (79%) | 0 (0%) | 13 (21%) minnowboard | runtime_core-image-sato-sdk_genericx86-64_20190416012138 | 49 (80%) | 0 (0%) | 12 (20%) minnowboard | runtime_core-image-sato-sdk_genericx86_20190416072759 | 49 (80%) | 0 (0%) | 12 (20%) mpc8315e-rdb | manual_bsps-hw_20190415102615 | 15 (33%) | 0 (0%) | 31 (67%) mpc8315e-rdb | manual_compliance-test_20190415103232 | 2 (33%) | 0 (0%) | 4 (67%) mpc8315e-rdb | runtime_core-image-sato-sdk_mpc8315e-rdb_20190410070846 | 47 (77%) | 2 (3%) | 12 (20%) multilib | runtime_core-image-minimal_qemumips64_20190328012724 | 1 (2%) | 0 (0%) | 57 (98%) multilib | runtime_core-image-sato_qemux86-64_20190328005537 | 36 (57%) | 0 (0%) | 27 (43%) multilib | runtime_core-image-sato_qemux86-64_20190328010523 | 15 (24%) | 0 (0%) | 48 (76%) multilib | runtime_lib32-core-image-minimal_qemux86-64_20190328004331 | 1 (100%) | 0 (0%) | 0 (0%) multilib | runtime_lib32-core-image-minimal_qemux86-64_20190328004626 | 1 (100%) | 0 (0%) | 0 (0%) multilib | sdk_core-image-minimal_x86_64_qemumips64_20190328012724 | 9 (82%) | 0 (0%) | 2 (18%) musl-qemux86-64 | runtime_core-image-full-cmdline_qemux86-64_20190328005407 | 12 (20%) | 0 (0%) | 48 (80%) musl-qemux86-64 | runtime_core-image-minimal_qemux86-64_20190328005407 | 1 (2%) | 0 (0%) | 59 (98%) musl-qemux86-64 | runtime_core-image-sato-sdk_qemux86-64_20190328005407 | 48 (80%) | 0 (0%) | 12 (20%) musl-qemux86 | runtime_core-image-full-cmdline_qemux86_20190328005325 | 12 (20%) | 0 (0%) | 48 (80%) musl-qemux86 | runtime_core-image-minimal_qemux86_20190328005325 | 1 (2%) | 0 (0%) | 59 (98%) musl-qemux86 | runtime_core-image-sato-sdk_qemux86_20190328005325 | 48 (80%) | 0 (0%) | 12 (20%) no-x11 | runtime_core-image-full-cmdline_qemux86-64_20190328004745 | 12 (20%) | 0 (0%) | 49 (80%) no-x11 | runtime_core-image-weston_qemux86-64_20190328004745 | 32 (52%) | 0 (0%) | 29 (48%) nuc6 | manual_compliance-test_20190416170925 | 4 (67%) | 0 (0%) | 2 (33%) nuc6 | runtime_core-image-lsb-sdk_genericx86-64_20190415062722 | 48 (79%) | 0 (0%) | 13 (21%) nuc6 | runtime_core-image-sato-sdk_genericx86-64_20190415051153 | 49 (80%) | 0 (0%) | 12 (20%) nuc7 | manual_bsps-hw_20190415085927 | 43 (100%) | 0 (0%) | 0 (0%) nuc7 | runtime_core-image-lsb-sdk_genericx86-64_20190416070445 | 48 (79%) | 0 (0%) | 13 (21%) nuc7 | runtime_core-image-sato-sdk_genericx86-64_20190412085847 | 49 (80%) | 0 (0%) | 12 (20%) oe-selftest-centos | oeselftest_centos-7_qemux86-64_20190328005014 | 330 (100%) | 0 (0%) | 1 (0%) oe-selftest-centos | runtime_core-image-full-cmdline_qemux86-64_20190328024414 | 4 (100%) | 0 (0%) | 0 (0%) oe-selftest-centos | runtime_core-image-full-cmdline_qemux86-64_20190328024557 | 4 (100%) | 0 (0%) | 0 (0%) oe-selftest-debian | oeselftest_debian-9_qemux86-64_20190328005049 | 330 (100%) | 0 (0%) | 1 (0%) oe-selftest-debian | runtime_core-image-full-cmdline_qemux86-64_20190328022241 | 4 (100%) | 0 (0%) | 0 (0%) oe-selftest-debian | runtime_core-image-full-cmdline_qemux86-64_20190328022404 | 4 (100%) | 0 (0%) | 0 (0%) oe-selftest-fedora | oeselftest_fedora-28_qemux86-64_20190328010400 | 330 (100%) | 0 (0%) | 1 (0%) oe-selftest-fedora | runtime_core-image-full-cmdline_qemux86-64_20190328024233 | 4 (100%) | 0 (0%) | 0 (0%) oe-selftest-ubuntu | oeselftest_ubuntu-18.04_qemux86-64_20190328001839 | 330 (100%) | 0 (0%) | 1 (0%) oe-selftest-ubuntu | runtime_core-image-full-cmdline_qemux86-64_20190328015546 | 4 (100%) | 0 (0%) | 0 (0%) oe-selftest-ubuntu | runtime_core-image-full-cmdline_qemux86-64_20190328015706 | 4 (100%) | 0 (0%) | 0 (0%) pkgman-deb-non-deb | runtime_core-image-minimal_qemux86_20190328003313 | 1 (2%) | 0 (0%) | 60 (98%) pkgman-deb-non-deb | runtime_core-image-sato-sdk_qemux86_20190328003313 | 30 (49%) | 0 (0%) | 31 (51%) pkgman-deb-non-deb | runtime_core-image-sato_qemux86_20190328003313 | 15 (25%) | 0 (0%) | 46 (75%) pkgman-non-rpm | runtime_core-image-minimal_qemux86_20190328002823 | 1 (2%) | 0 (0%) | 60 (98%) pkgman-non-rpm | runtime_core-image-minimal_qemux86_20190328004538 | 1 (2%) | 0 (0%) | 60 (98%) pkgman-non-rpm | runtime_core-image-sato-sdk_qemux86_20190328002823 | 31 (51%) | 0 (0%) | 30 (49%) pkgman-non-rpm | runtime_core-image-sato-sdk_qemux86_20190328004538 | 30 (49%) | 0 (0%) | 31 (51%) pkgman-non-rpm | runtime_core-image-sato_qemux86_20190328002823 | 14 (23%) | 0 (0%) | 47 (77%) pkgman-non-rpm | runtime_core-image-sato_qemux86_20190328004538 | 15 (25%) | 0 (0%) | 46 (75%) pkgman-rpm-non-rpm | runtime_core-image-minimal_qemux86_20190328003915 | 1 (2%) | 0 (0%) | 60 (98%) pkgman-rpm-non-rpm | runtime_core-image-sato-sdk_qemux86_20190328003915 | 49 (80%) | 0 (0%) | 12 (20%) pkgman-rpm-non-rpm | runtime_core-image-sato_qemux86_20190328003915 | 35 (57%) | 0 (0%) | 26 (43%) qa-extras2 | runtime_core-image-sato_qemux86-64_20190328003056 | 37 (61%) | 0 (0%) | 24 (39%) qa-extras2 | runtime_core-image-sato_qemux86-64_20190328003507 | 36 (59%) | 0 (0%) | 25 (41%) qa-extras2 | runtime_core-image-sato_qemux86-64_20190328003902 | 37 (59%) | 0 (0%) | 26 (41%) qa-extras2 | runtime_core-image-sato_qemux86-64_20190328004342 | 42 (69%) | 0 (0%) | 19 (31%) qa-extras2 | runtime_core-image-sato_qemux86-64_20190328004728 | 35 (57%) | 0 (0%) | 26 (43%) qa-extras2 | runtime_core-image-sato_qemux86-64_20190328005231 | 42 (69%) | 0 (0%) | 19 (31%) qemuarm-lsb | runtime_core-image-lsb-sdk_qemuarm_20190328010037 | 48 (79%) | 0 (0%) | 13 (21%) qemuarm-lsb | runtime_core-image-lsb_qemuarm_20190328010037 | 34 (56%) | 0 (0%) | 27 (44%) qemuarm64 | manual_bsps-qemu_20190416173043 | 5 (62%) | 0 (0%) | 3 (38%) qemuarm64 | runtime_core-image-minimal_qemuarm64_20190328005246 | 1 (2%) | 0 (0%) | 59 (98%) qemuarm64 | runtime_core-image-sato-sdk_qemuarm64_20190328005246 | 48 (80%) | 0 (0%) | 12 (20%) qemuarm64 | runtime_core-image-sato_qemuarm64_20190328005246 | 34 (57%) | 0 (0%) | 26 (43%) qemuarm64 | sdk_core-image-sato_i686_qemuarm64_20190328005246 | 11 (100%) | 0 (0%) | 0 (0%) qemuarm64 | sdkext_core-image-minimal_x86_64_qemuarm64_20190328015152 | 15 (75%) | 0 (0%) | 5 (25%) qemuarm64 | sdkext_core-image-sato_x86_64_qemuarm64_20190328015152 | 15 (75%) | 0 (0%) | 5 (25%) qemuarm | manual_bsps-qemu_20190416172910 | 5 (62%) | 0 (0%) | 3 (38%) qemuarm | runtime_core-image-minimal_qemuarm_20190328002950 | 1 (2%) | 0 (0%) | 60 (98%) qemuarm | runtime_core-image-sato-sdk_qemuarm_20190328002950 | 49 (80%) | 0 (0%) | 12 (20%) qemuarm | runtime_core-image-sato_qemuarm_20190328002950 | 35 (57%) | 0 (0%) | 26 (43%) qemuarm | sdk_core-image-sato_i686_qemuarm_20190328002950 | 11 (100%) | 0 (0%) | 0 (0%) qemuarm | sdkext_core-image-minimal_x86_64_qemuarm_20190328015353 | 15 (75%) | 0 (0%) | 5 (25%) qemuarm | sdkext_core-image-sato_x86_64_qemuarm_20190328015353 | 15 (75%) | 0 (0%) | 5 (25%) qemumips-lsb | runtime_core-image-lsb-sdk_qemumips_20190328010135 | 45 (78%) | 0 (0%) | 13 (22%) qemumips-lsb | runtime_core-image-lsb_qemumips_20190328010135 | 34 (59%) | 0 (0%) | 24 (41%) qemumips64 | manual_bsps-qemu_20190416172522 | 5 (62%) | 0 (0%) | 3 (38%) qemumips64 | runtime_core-image-minimal_qemumips64_20190328010018 | 1 (2%) | 0 (0%) | 57 (98%) qemumips64 | runtime_core-image-sato-sdk_qemumips64_20190328010018 | 46 (79%) | 0 (0%) | 12 (21%) qemumips64 | runtime_core-image-sato_qemumips64_20190328010018 | 35 (60%) | 0 (0%) | 23 (40%) qemumips64 | sdk_core-image-sato_i686_qemumips64_20190328010018 | 11 (100%) | 0 (0%) | 0 (0%) qemumips64 | sdkext_core-image-minimal_x86_64_qemumips64_20190328023044 | 15 (75%) | 0 (0%) | 5 (25%) qemumips64 | sdkext_core-image-sato_x86_64_qemumips64_20190328023044 | 15 (75%) | 0 (0%) | 5 (25%) qemumips | manual_bsps-qemu_20190416172259 | 5 (62%) | 0 (0%) | 3 (38%) qemumips | runtime_core-image-minimal_qemumips_20190328005803 | 1 (2%) | 0 (0%) | 57 (98%) qemumips | runtime_core-image-sato-sdk_qemumips_20190328005803 | 46 (79%) | 0 (0%) | 12 (21%) qemumips | runtime_core-image-sato_qemumips_20190328005803 | 35 (60%) | 0 (0%) | 23 (40%) qemumips | sdk_core-image-sato_i686_qemumips_20190328005803 | 11 (100%) | 0 (0%) | 0 (0%) qemumips | sdkext_core-image-minimal_x86_64_qemumips_20190328023934 | 15 (75%) | 0 (0%) | 5 (25%) qemumips | sdkext_core-image-sato_x86_64_qemumips_20190328023934 | 15 (75%) | 0 (0%) | 5 (25%) qemuppc-lsb | runtime_core-image-lsb-sdk_qemuppc_20190328011857 | 48 (79%) | 0 (0%) | 13 (21%) qemuppc-lsb | runtime_core-image-lsb_qemuppc_20190328011857 | 34 (56%) | 0 (0%) | 27 (44%) qemuppc | manual_bsps-qemu_20190416172725 | 5 (62%) | 0 (0%) | 3 (38%) qemuppc | runtime_core-image-minimal_qemuppc_20190328010415 | 1 (2%) | 0 (0%) | 60 (98%) qemuppc | runtime_core-image-sato-sdk_qemuppc_20190328010415 | 49 (80%) | 0 (0%) | 12 (20%) qemuppc | runtime_core-image-sato_qemuppc_20190328010415 | 35 (57%) | 0 (0%) | 26 (43%) qemuppc | sdk_core-image-sato_i686_qemuppc_20190328010415 | 11 (100%) | 0 (0%) | 0 (0%) qemuppc | sdkext_core-image-minimal_x86_64_qemuppc_20190328020155 | 15 (75%) | 0 (0%) | 5 (25%) qemuppc | sdkext_core-image-sato_x86_64_qemuppc_20190328020155 | 15 (75%) | 0 (0%) | 5 (25%) qemux86-64-x32 | runtime_core-image-minimal_qemux86-64_20190328010011 | 1 (2%) | 0 (0%) | 60 (98%) qemux86-64 | manual_bsps-qemu_20190416172006 | 5 (62%) | 0 (0%) | 3 (38%) qemux86-64 | runtime_core-image-minimal_qemux86-64_20190328010727 | 1 (2%) | 0 (0%) | 60 (98%) qemux86-64 | runtime_core-image-sato-sdk_qemux86-64_20190328010727 | 49 (80%) | 0 (0%) | 12 (20%) qemux86-64 | runtime_core-image-sato_qemux86-64_20190328010727 | 35 (57%) | 0 (0%) | 26 (43%) qemux86-64 | sdk_core-image-sato_i686_qemux86-64_20190328010727 | 11 (100%) | 0 (0%) | 0 (0%) qemux86-64 | sdk_core-image-sato_x86_64_qemux86-64_20190328011455 | 11 (100%) | 0 (0%) | 0 (0%) qemux86-64 | sdkext_core-image-minimal_x86_64_qemux86-64_20190328011455 | 15 (75%) | 0 (0%) | 5 (25%) qemux86-64 | sdkext_core-image-sato_x86_64_qemux86-64_20190328011455 | 15 (75%) | 0 (0%) | 5 (25%) qemux86-lsb | runtime_core-image-lsb-sdk_qemux86_20190328010015 | 48 (79%) | 0 (0%) | 13 (21%) qemux86-lsb | runtime_core-image-lsb_qemux86_20190328010015 | 34 (56%) | 0 (0%) | 27 (44%) qemux86 | manual_bsps-qemu_20190416171331 | 8 (100%) | 0 (0%) | 0 (0%) qemux86 | runtime_core-image-minimal_qemux86_20190328010207 | 1 (2%) | 0 (0%) | 60 (98%) qemux86 | runtime_core-image-sato-sdk_qemux86_20190328010207 | 49 (80%) | 0 (0%) | 12 (20%) qemux86 | runtime_core-image-sato_qemux86_20190328010207 | 35 (57%) | 0 (0%) | 26 (43%) qemux86 | sdk_core-image-sato_i686_qemux86_20190328010207 | 11 (100%) | 0 (0%) | 0 (0%) qemux86 | sdk_core-image-sato_x86_64_qemux86_20190328011153 | 11 (100%) | 0 (0%) | 0 (0%) qemux86 | sdkext_core-image-minimal_x86_64_qemux86_20190328011153 | 15 (75%) | 0 (0%) | 5 (25%) qemux86 | sdkext_core-image-sato_x86_64_qemux86_20190328011153 | 15 (75%) | 0 (0%) | 5 (25%) sdk-extras | manual_sdk_20190411165245 | 1 (100%) | 0 (0%) | 0 (0%) sdk-extras | manual_sdk_20190416175031 | 1 (100%) | 0 (0%) | 0 (0%) sdk-extras | oeselftest_ubuntu-16.04_qemuarm_20190411094931 | 3 (100%) | 0 (0%) | 0 (0%) sdk-extras | oeselftest_ubuntu-16.04_qemumips_20190411101805 | 3 (100%) | 0 (0%) | 0 (0%) sdk-extras | oeselftest_ubuntu-16.04_qemuppc_20190411105505 | 3 (100%) | 0 (0%) | 0 (0%) sdk-extras | oeselftest_ubuntu-16.04_qemux86_20190410105457 | 3 (100%) | 0 (0%) | 0 (0%) toaster | manual_toaster-managed-mode_20190417143733 | 62 (95%) | 3 (5%) | 0 (0%) toaster | manual_toaster-unmanaged-mode_20190417110922 | 28 (100%) | 0 (0%) | 0 (0%)
There was no ptest data
==================================================================================================
Failed test cases (sorted by testseries, ID)
==================================================================================================
testseries | result_id : beaglebone | runtime_core-image-sato-sdk_beaglebone-yocto_20190410043531
parselogs.ParseLogsTest.test_parselogs
testseries | result_id : build-appliance | manual_build-appliance_20190412091610
build-appliance.build-appliance.Bitbake_build-appliance-image
testseries | result_id : edgerouter | runtime_core-image-sato-sdk_edgerouter_20190411013527
xorg.XorgTest.test_xorg_running parselogs.ParseLogsTest.test_parselogs
testseries | result_id : mpc8315e-rdb | runtime_core-image-sato-sdk_mpc8315e-rdb_20190410070846
xorg.XorgTest.test_xorg_running parselogs.ParseLogsTest.test_parselogs
testseries | result_id : toaster | manual_toaster-managed-mode_20190417143733
toaster-managed-mode.toaster-managed.Adding_packages_with_dependencies toaster-managed-mode.toaster-managed.Removing_packages_without_and_with_dependencies__from_custom_images toaster-managed-mode.toaster-managed.Adding_packages_without_dependencies_from_custom_images
QA cycle report for 2.5.3 RC3:
1. No high milestone defects. 2. Test results are available at following location: • For results of all automated tests, refer to results at public AB [1]. • For other test results, refer to attachment [2]. • For test report for test cases run by Intel and WR team, refer attachment [3] • For full test report, refer attachment [4] • For ptest results, please refer to results at public AB [5] • For ptest report, refer to attachment [6] 3. No new defects are found in this cycle. 4. Number of existing issues observed in this release is 2- toaster [7] and Build-appliance [8] 5. For ptest, regression data is not available for this release. No timeout issues. 6. Test result report on Public AB shows no failures.
= Links ==
Link to testresults:
[1] - https://autobuilder.yocto.io/pub/releases/yocto-2.5.3.rc3/testresults/
[2] - 2.5.3_RC3_results_merged.zip
[3] - 2. 5.3_rc3_intel_wr_merged_report
[4] - 2. 5.3_rc3_full_report
[5] - https://autobuilder.yocto.io/pub/non-release/20190328-7/testresults/qemux86-64-ptest/
[6] - 2. 5.3_rc3_ptest_report
Previous Bugs observed in this release:
[7] Bug 13191 – [Test Case 1439] Build Failure after adding or removing packages with and without dependencies https://bugzilla.yoctoproject.org/show_bug.cgi?id=13191
[8] Bug 12991 - [Bug] [2.6 M4 RC1][Build-Appliance] Bitbake build-appliance-image getting failed during building image due to webkitgtk package. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12991
