Yocto 1.3 Milestone Test Report: Difference between revisions

From Yocto Project
Jump to navigationJump to search
No edit summary
 
(23 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This page is to hold Milestone Test Report for Yocto 1.3
This page is to hold Milestone Test Report for Yocto 1.3
====Yocto 1.3 M1 Build Test Report====
====Yocto 1.3 M1 Build Test Report====
This is the Fullpass test report for Yocto 1.3 20120606 build. There are 15 bugs found in this test circle. We find some meta-intel BSPs failed to build with "Weekly build 1.3_M1 Week 4", one is for connman-gnome and another is for gst-ffmpeg, Due to the issue, QA testing against Sugarbay/eMenlow/Crownbay/FRI2/Huron River is blocked. The 2 bugs we reported for these build error are bug 2561 and 2562.
This is the Fullpass test report for Yocto 1.3 M1 20120613 build on meta-intel platform. QA found one critical issue that is grub-efi-native fails due to help2man script which blocked all cases on FRI2, reported as bug 2527. For Hob, we found two minor issues, they are Incorrect total image size in "View Package" page and can’t delete layer in special situation. For core build system, self-hosted-image are still running, no new issue found till now, For Distribution Support, Yocto build on OpenSuse 12.1 64 bit and Ubuntu 12.04 64 bit could pass, but beecrypt-native failed to compile on Fedora17(final) 64bit report as 2631.
For Hob, a  minor issue is that Incorrect image shown when loading template file, bug 2576.  
Other features perform well and no new issue found except the syslog daemon can't startup on qemu and boards.
For Core Build System, there are 9 issues, Mulitilb could not work with lsb , sato build and lib32-connman-gnome build;  two bugs for Yocto-bsp tools, X can't start in sato image built by yocto-bsp tool; fail to build core-image-sato when add patch into the kernel; qemu-native build failed in self-hosted-image with vmdk; lib64-core-image-sato build failed due to sanity check.
For Build performance, one qemux86 sato build on a Core i7 machine costs 95 minutes.
For Distribution testing, we tested on three distributions, they are Fedora 16,OpenSuse 12.1 and Ubuntu 11.10, and no issue found with Yocto build.
For ADT, we found two issues that are Segmentation fault on qemuarm from /usr/lib/libust.so.0 and  ADT-Installer failed if the folder /opt/poky/1.2+snapshot exist. Other features perform well.
For Build performance, one qemux86 sato build on a Core i7 machine costs 108 minutes, which is almost the same as the result in 1.2 M4 RC5 build(111 minutes)


'''Test Summary:'''
'''Test Summary:'''


[[File:Fullpass_Yocto1.3_M1_RC1_Test_Result_Summary.png]]
[[File:Fullpass_Yocto1.3_M1_RC1_Test_Result_Summary3.png]]


'''Test detail Summary:'''
'''Test detail Summary:'''
Line 19: Line 16:
* The failed/blocked case number is listed with failed cases’ bug number.
* The failed/blocked case number is listed with failed cases’ bug number.


'''Distribution Test:'''
[[File:Fullpass_Yocto1.3_M1_RC1_Detailed_Test_Result_.png]]
 
'''Commit Information:'''
 
Tree/Branch: poky/master
Commit id: 7f017cab0b3e4d5a4fc356eccd8eb8429b0531d0
 
Meta branch: master
Commit id: bd00e28dd54b06a1f01f186be168d3af872f96d4


[[File:Fullpass_Yocto1.3_M1_RC1_Test_Distribution_Test.png]]
 
'''Issue Summary:'''
 
[[File:Fullpass_Yocto1.3_M1_RC1_Issue_Summary_.png]]
 
 
====Yocto 1.3 M2 Build Test Report====
 
Here are the results for the Full Pass testing on 1.3 M2 build with commit id: eb0cb7e8234f5d2e5623406e9660be91cf52f65e
For HOB testing the commit id is: 4bd0338972a1a64829734c43501a3a74c510421a
The link for tested images is: http://autobuilder.yoctoproject.org/pub/nightly/20120712-1
 
Summary: Everything looks ok on qemu (both x86 and non-x86), mpc8315e-rdb-sato-sdk tests are blocked by bug 2775, FRI2 has 4 tests cases failing. For Core Build System there are 16 test cases failed (6 bugs) and 2 test cases blocked (by 2 bugs) – most of the tests cases fail due  to multilb issues. In ADT there are 10 test cases failed (4 bugs)
 
The build time for sato image on an i7 processor increased to 127m37.820s (aprox 10%)
 
 
'''Test Summary:'''
 
[[File:Fullpass_Yocto1.3_M2_RC1_Test_Result_Summary.png]]
 
'''Test detail Summary:'''
 
[[File:Fullpass_Yocto1.3_M1_RC1_Detailed_Test_Result.png]]
 
* You can check the detailed test result in attachment for each target.
* The failed/blocked case number is listed with failed cases’ bug number.
 
[[File:Fullpass_Yocto1.3_M2_RC1_Detailed_Test_Result.png]]
 
'''Issue Summary:'''
 
[[File:Fullpass_Yocto1.3_M2_RC1_Issue_Summary.png]]
 
 
This page is to hold Milestone Test Report for Yocto 1.3
====Yocto 1.3 M3 Build Test Report====
Here are the results for the full pas tests on 1.3 M3 RC2. The commit used for testing is 8b8748c8f963900b83dc0fdd7757556f917fe4fd.
Some details about the encountered issues below:
 
BSP – Sudoku-savant project build issue (2878)
ADT – the relocatable sdk issue (2980) causes 13 test cases to be on faile/blocked state , also the Clutter C template issue is unsolved (2577)
Core Build System – x32 is still an issue (2888), cleaning sstate issue is still not solved (2897), incremental RPM image generation (2969), source archiving (2619), the kvm issue was reproduced by another colleague (2790) Yocto BSP creation via JSON (2693) or for qemu (2991) fails, multilib issue (2918 – this requires a little more investigation from QA),
HOB - all seems ok for RC2
Self-hosted-image  - cannot start on Virtual Box (X issue), it is very slow on qemu and it has a m4 package build (3005) issue on VMWare. If the self-hosted-image is used on machine with internet connectivity via proxy there will be an initial sanity check failure, but this is not a blocking issue.
A mention for the performance testing: on a Ubbuntu 12.04  i7 machine using 8 threads the build time was 83 minutes (with prior fetching).
 
'''Test Summary:'''
 
[[File:Fullpass_Yocto1.3_M3_RC2_Detailed_Test_Result.png]]
 
 
'''Test detail Summary:'''
 
[[File:Fullpass_Yocto1.3_M1_RC1_Detailed_Test_Result.png]]
 
* You can check the detailed test result in attachment for each target.
* The failed/blocked case number is listed with failed cases’ bug number.
 
[[File:Fullpass_Yocto1.3_M3_RC2_Test_Result_Summary3.png]]


'''Commit Information:'''
'''Commit Information:'''


Tree/Branch: denzil
Tree/Branch: poky/master
Commit id: 8b8748c8f963900b83dc0fdd7757556f917fe4fd
 
'''Issue Summary:'''
 
[[File:Fullpass_Yocto1.3_M3_RC2_Issue_Summary.png]]
 
 
====Yocto 1.3 M4 Build Test Report====
 
Here are the results for 1.3 M4 RC3 full pass testing.
The commit id is: e5c3513ca8abb21eb5e9b8c19d1b9199b14ae07c.
 
The following issues are encountered:
 
*For the BSPs:
**For e-menlow the X server is not starting (3091)
**For FRI2 – connman does not provide a 3g configuration utility (2415), there is dmesg PowerButton error (2989) and the system cannot enter S3 standby mode (2037)
**For qemu x86_64 images – the zypper refresh fails
**For all the boards there is common issue with the Sudoku-savant application (2878) and with the warnings at boot time (2399)
*For core build system:
**There are still errors with the do_rootfs for RPM incremental build (3047), with multilib builds with ipk (2590), the do_rootfs fails for lib64-core-image-sato-sdk (2918) and there is a small issue with the sstate-cache-management.sh
*For ADT
**There are issues with the lttng (3100) and meta-toolchain-support.
 
The build for core-image-sato with pre-fetched sources takes '''85 minutes'''
 
'''Test Summary:'''
 
[[File:Report_summary-ww39.JPG‎]]
*You can check the detailed test result in attachment for each target.
*The failed/blocked case number is listed with failed cases’ bug number.
 
[[File:Bsp-ww39.JPG]]
 
 
 
'''Test detailed Summary:'''
 
[[File:Bsp-detailed-ww39.JPG]]
 
 
 
'''Issue Summary:'''
 
[[File:Other-ww39.JPG]]
 
 
====Yocto 1.3 M5 Build Test Report====
 
Here are the results for 1.3 M5 RC4 full pass testing.
The commit id is: c9de24d3f4845251b01e2eab0aeeb7472badc5bb.
 
The following issues are encountered. Also updated with comments from Jessica and Saul:
 
*For the BSPs:
**For qemu x86_64 images - the zypper refresh fails (2694).
*For core build system:
**There are still errors with the do_rootfs fails for lib64-core-image-sato-sdk (2918).
*For ADT
**GL support still available on qemuarm (3287).
 
 
 
'''Test Summary:'''
 
 
[[File:FullPass_Yocto_1.3_RC4_Test_Result_Summary_v2.JPG]]
 
 
'''Test detailed Summary:'''
 
 
[[File:Fullpass_Yocto1.3_M1_RC1_Detailed_Test_Result.png]]
*You can check the detailed test result in attachment for each target.
*The failed/blocked case number is listed with failed cases’ bug number.
 


Commit id: 348da93e283b2b8454fd9cde80d4b17378e2a168


Meta branch :master
[[File:FullPass_Yocto_1.3_RC4_Test_Result_Summary_v3.JPG]]


Commit id: 8bc2fa135060e941bcbb4dcfb664e175a3eb871d




'''Issue Summary:'''
'''Issue Summary:'''


[[File:Fullpass_Yocto1.3_M1_RC1_Issue_Summary.png]]
[[File:FullPass_Yocto_1.3_RC4_Issue_Summary_v2.JPG]]

Latest revision as of 13:50, 23 October 2012

This page is to hold Milestone Test Report for Yocto 1.3

Yocto 1.3 M1 Build Test Report

This is the Fullpass test report for Yocto 1.3 M1 20120613 build on meta-intel platform. QA found one critical issue that is grub-efi-native fails due to help2man script which blocked all cases on FRI2, reported as bug 2527. For Hob, we found two minor issues, they are Incorrect total image size in "View Package" page and can’t delete layer in special situation. For core build system, self-hosted-image are still running, no new issue found till now, For Distribution Support, Yocto build on OpenSuse 12.1 64 bit and Ubuntu 12.04 64 bit could pass, but beecrypt-native failed to compile on Fedora17(final) 64bit report as 2631. Other features perform well and no new issue found except the syslog daemon can't startup on qemu and boards. For Build performance, one qemux86 sato build on a Core i7 machine costs 95 minutes.

Test Summary:

Fullpass Yocto1.3 M1 RC1 Test Result Summary3.png

Test detail Summary:

Fullpass Yocto1.3 M1 RC1 Detailed Test Result.png

  • You can check the detailed test result in attachment for each target.
  • The failed/blocked case number is listed with failed cases’ bug number.

Fullpass Yocto1.3 M1 RC1 Detailed Test Result .png

Commit Information:

Tree/Branch: poky/master Commit id: 7f017cab0b3e4d5a4fc356eccd8eb8429b0531d0

Meta branch: master Commit id: bd00e28dd54b06a1f01f186be168d3af872f96d4


Issue Summary:

Fullpass Yocto1.3 M1 RC1 Issue Summary .png


Yocto 1.3 M2 Build Test Report

Here are the results for the Full Pass testing on 1.3 M2 build with commit id: eb0cb7e8234f5d2e5623406e9660be91cf52f65e For HOB testing the commit id is: 4bd0338972a1a64829734c43501a3a74c510421a The link for tested images is: http://autobuilder.yoctoproject.org/pub/nightly/20120712-1

Summary: Everything looks ok on qemu (both x86 and non-x86), mpc8315e-rdb-sato-sdk tests are blocked by bug 2775, FRI2 has 4 tests cases failing. For Core Build System there are 16 test cases failed (6 bugs) and 2 test cases blocked (by 2 bugs) – most of the tests cases fail due to multilb issues. In ADT there are 10 test cases failed (4 bugs)

The build time for sato image on an i7 processor increased to 127m37.820s (aprox 10%)


Test Summary:

Fullpass Yocto1.3 M2 RC1 Test Result Summary.png

Test detail Summary:

Fullpass Yocto1.3 M1 RC1 Detailed Test Result.png

  • You can check the detailed test result in attachment for each target.
  • The failed/blocked case number is listed with failed cases’ bug number.

Fullpass Yocto1.3 M2 RC1 Detailed Test Result.png

Issue Summary:

Fullpass Yocto1.3 M2 RC1 Issue Summary.png


This page is to hold Milestone Test Report for Yocto 1.3

Yocto 1.3 M3 Build Test Report

Here are the results for the full pas tests on 1.3 M3 RC2. The commit used for testing is 8b8748c8f963900b83dc0fdd7757556f917fe4fd. Some details about the encountered issues below:

BSP – Sudoku-savant project build issue (2878) ADT – the relocatable sdk issue (2980) causes 13 test cases to be on faile/blocked state , also the Clutter C template issue is unsolved (2577) Core Build System – x32 is still an issue (2888), cleaning sstate issue is still not solved (2897), incremental RPM image generation (2969), source archiving (2619), the kvm issue was reproduced by another colleague (2790) Yocto BSP creation via JSON (2693) or for qemu (2991) fails, multilib issue (2918 – this requires a little more investigation from QA), HOB - all seems ok for RC2 Self-hosted-image - cannot start on Virtual Box (X issue), it is very slow on qemu and it has a m4 package build (3005) issue on VMWare. If the self-hosted-image is used on machine with internet connectivity via proxy there will be an initial sanity check failure, but this is not a blocking issue. A mention for the performance testing: on a Ubbuntu 12.04 i7 machine using 8 threads the build time was 83 minutes (with prior fetching).

Test Summary:

Fullpass Yocto1.3 M3 RC2 Detailed Test Result.png


Test detail Summary:

Fullpass Yocto1.3 M1 RC1 Detailed Test Result.png

  • You can check the detailed test result in attachment for each target.
  • The failed/blocked case number is listed with failed cases’ bug number.

Fullpass Yocto1.3 M3 RC2 Test Result Summary3.png

Commit Information:

Tree/Branch: poky/master Commit id: 8b8748c8f963900b83dc0fdd7757556f917fe4fd

Issue Summary:

Fullpass Yocto1.3 M3 RC2 Issue Summary.png


Yocto 1.3 M4 Build Test Report

Here are the results for 1.3 M4 RC3 full pass testing. The commit id is: e5c3513ca8abb21eb5e9b8c19d1b9199b14ae07c.

The following issues are encountered:

  • For the BSPs:
    • For e-menlow the X server is not starting (3091)
    • For FRI2 – connman does not provide a 3g configuration utility (2415), there is dmesg PowerButton error (2989) and the system cannot enter S3 standby mode (2037)
    • For qemu x86_64 images – the zypper refresh fails
    • For all the boards there is common issue with the Sudoku-savant application (2878) and with the warnings at boot time (2399)
  • For core build system:
    • There are still errors with the do_rootfs for RPM incremental build (3047), with multilib builds with ipk (2590), the do_rootfs fails for lib64-core-image-sato-sdk (2918) and there is a small issue with the sstate-cache-management.sh
  • For ADT
    • There are issues with the lttng (3100) and meta-toolchain-support.

The build for core-image-sato with pre-fetched sources takes 85 minutes

Test Summary:

Report summary-ww39.JPG

  • You can check the detailed test result in attachment for each target.
  • The failed/blocked case number is listed with failed cases’ bug number.

Bsp-ww39.JPG


Test detailed Summary:

Bsp-detailed-ww39.JPG


Issue Summary:

Other-ww39.JPG


Yocto 1.3 M5 Build Test Report

Here are the results for 1.3 M5 RC4 full pass testing. The commit id is: c9de24d3f4845251b01e2eab0aeeb7472badc5bb.

The following issues are encountered. Also updated with comments from Jessica and Saul:

  • For the BSPs:
    • For qemu x86_64 images - the zypper refresh fails (2694).
  • For core build system:
    • There are still errors with the do_rootfs fails for lib64-core-image-sato-sdk (2918).
  • For ADT
    • GL support still available on qemuarm (3287).


Test Summary:


FullPass Yocto 1.3 RC4 Test Result Summary v2.JPG


Test detailed Summary:


Fullpass Yocto1.3 M1 RC1 Detailed Test Result.png

  • You can check the detailed test result in attachment for each target.
  • The failed/blocked case number is listed with failed cases’ bug number.


FullPass Yocto 1.3 RC4 Test Result Summary v3.JPG


Issue Summary:

FullPass Yocto 1.3 RC4 Issue Summary v2.JPG

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Yocto_1.3_Milestone_Test_Report&oldid=7671"