|
|
(111 intermediate revisions by 15 users not shown) |
Line 1: |
Line 1: |
| ==QA== | | ==Current Release QA Trackers== |
| ===Milestone Test Report for Yocto===
| |
|
| |
|
| ====Fullpass Test Report for M3-20100911 ====
| | *Bugs that need to be implemented by QA Team |
| [https://oldwiki.pokylinux.org/share/QA_docs/milestone%20test/M3/Fullpass%20Test%20Report%20for%20Yocto%20M3%2020100911%20Build.htm Fullpass Test Report for M3-20100911] | | **[[2.2 qa assigned bugs]] |
|
| |
|
| ==== Sanity Test Report for M3-20100911 ====
| | *Bugs that need to be verified by QA Team |
| [https://oldwiki.pokylinux.org/share/QA_docs/milestone%20test/M3/Yocto%20Sanity%20Test%20Report%20for%20M3-20100911.htm Sanity Test Report for M3-20100911] | | **[[2.2 qa owned_bugs]] |
|
| |
|
| ==== Sanity Test Report for M3-20100909 ====
| | *Features to verify |
| [https://oldwiki.pokylinux.org/share/QA_docs/milestone%20test/M3/Yocto%20Sanity%20Test%20Report%20for%20M3-20100909.htm Sanity Test Report for M3-20100911] | | **[[2.2 qa_owned features to verify]] |
|
| |
|
| ==== Partial Sanity Test Report for M3 20100905 ====
| | *Features to implement |
| [https://oldwiki.pokylinux.org/share/QA_docs/milestone%20test/M3/Yocto%20Partial%20Sanity%20Test%20Report%20for%20M3%2020100905.htm Partial Test Report for M3 20100905] | | **[[2.2 qa owned features]] |
|
| |
|
| ==== Fullpass Test for M2-20100731 ====
| | *Old Bugs |
| [https://git.pokylinux.org/share/QA_docs/milestone%20test/M2/Fullpass%20Test%20Report%20for%20M2%2020100731.htm Fullpass Test Report for M2-20100731] | | **[[Old resolved bugs and features]] |
| ==== M2-20100725 & Bug Update for M2-20100728 ====
| |
| I finished testing on QEMU/Netbook for M2-20100725 and verified some bugs with M2-20100728 build. Against M2-20100725 build, I reported 12 new bugs and verified 2 bugs as fixed. With the second build M2-20100728, I verified old critical bugs and reported another 2 new bugs. 2 old bugs are verified as fixed with this build.
| |
|
| |
|
| ==== M3 Test Execution Plan ==== | | ==Current Release QA Test Plans== |
| [https://oldwiki.pokylinux.org/share/QA_docs/Test%20Plan/M3-Test%20Plan.doc M3 fullpass test plan]. | | * [[QA Master Test Plan]] |
| | * [[Yocto Project 2.2_Release Test Plan]] |
| | * [[2.2 QA Status]] |
| | * [[Toaster testing plan]] |
| | * [[Extensible SDK Test Plan (eSDK)]] |
| | * [[Distro Testing Plan]] |
| | * [[BSP Test Plan]] |
|
| |
|
| ==== M4 Test Execution Plan ==== | | ==Current Release QA Test Cases== |
| [https://oldwiki.pokylinux.org/share/QA_docs/Test%20Plan/M4-Test_Plan.doc M4 fullpass test plan]. | | * [[Yocto 2.2 Test Cases]] |
|
| |
|
| | ==Test Automation== |
| | ===Autobuilder=== |
|
| |
|
| | The [[AutoBuilder]] is the automated build and test framework used by Yocto Project QA. |
|
| |
|
| There are still 5 open bugs for M2, 4 of them are CRITICAL. And 23 open bugs are marked to M3. As a summary, QEMU targets (x86/arm/mips/ppc) can work without critical bugs. Toolchain has one critical bug. Meanwhile, emenlow/netbook have critical issue with/within X environment.
| | ===Adding Automated Tests=== |
|
| |
|
| | Tests for a given component can be automated in the AutoBuilder. With that purpose, follow the [[AutoBuilder/Adding Automated Tests|Adding Automated Tests]] to the Autobuilder Guide. |
|
| |
|
| '''Open Bugs for M2:
| | A list of tests that are automated [[QA#List_of_Automated_Tests|can be seen here]]. |
| * Wrong qemu binary is used if qemu-native isn't built -- (Richard, is the issue going to be fixed in M2?)
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=123
| |
| * Mouse/Keyboard cannot work with poky-image-sato-live on netbook -- (It should be fixed by Jeff, but mouse/kbd still not work because of bug 171)
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=141
| |
| * [emenlow] X not bootable of image poky-image-sato-live-emenlow -- (This bug is with 0728 image)
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=159
| |
| * [toolchain] Cannot build LTP with ppc/arm toolchain
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=161
| |
| * [Netbook] Xserver version mismatch makes mouse/keyboard not work
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=171
| |
|
| |
|
| '''Other Open Bugs reported, target for M3 or later:
| | ==Reporting== |
| * It takes long time to boot qemuppc image at the first time -- Enhance/Low
| | *[[Bug reporting and Information levels]] |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=143
| | *[[Testopia]], the test manager |
| * avahi-daemon fail to start with qemumips sdk image -- Minor/Medium (Target on M3)
| | *[[QA/Test_Reporting_Tool|The Test Reporting Tool]] |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=144
| | *[http://errors.yoctoproject.org/Errors/ error report web] |
| * nm-applet segfault with qemux86 sato/sdk image -- Normal/Medium (Not our focus)
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=145
| |
| * keystroke combination ctrl-alt-del not work on netbook -- Normal/Low
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=146
| |
| * [Netbook] pcmanfm no response after I click File Manager icon -- Major/Medium (Target on M3) | |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=148
| |
| * [All] There is no way to exit matchbox-keyboard in X -- Normal/Medium (Target on M3)
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=149
| |
| * [Netbook] Music Player cannot play WAV format file -- Major/Undecided | |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=151
| |
| * [emenlow] command reboot cannot make system reboot entirely -- Medium/Normal (Target on M3)
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=160
| |
| * [LTP] LTP default test suite cannot fully pass -- Normal/Low (Target on M4)
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=169 | |
| * [POSIX] POSIX default test suite cannot fully pass -- Normal/Low (Target on M4) | |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=172
| |
|
| |
|
| '''Verified Fixed Issues:
| | ==Performance testing== |
| * qemuppc images cannot boot up
| | [[Performance Test]] |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=101
| |
| * Some icons broken with qemumips images
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=106
| |
| * qemumips poky-image-sato/sdk fail to boot -- (Fixed in 0728 build)
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=138
| |
| * emenlow live image cannot boot/install -- (Fixed in 0728 build)
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=142
| |
|
| |
|
| '''Existing Old Issues: | | ==QA Resources== |
| * No mouse action response in X window with qemumips images -- Major/High (Target on M3) | | *[[Rpm's Repository Setup for QA]] |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=99 | | *[[Testopia]] |
| * qemuarm and qemuppc images cannot shutdown entirely -- Normal/High (Target on M3) | | *[[Testing Cycle]] |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=100
| | *[[qa-tools|qa-tools Git Repository]] |
|
| |
|
| ==== M1 Test Report 2010/7/10 ==== | | = Archive = |
| '''Summary & Issue List'''
| |
|
| |
|
| I finished sanitytest for M1 20100710 build. There is 2 new issues found in this testing. qemuppc boot up issue is fixed but qemu will report OOM error when booting qemuppc images. There is no mouse action response after X startup with qemumips images. qemuppc and qemuarm cannot shutdown entirely.
| | You can find the previous QA work by release in the [[QA/Archive|Yocto Project QA Archive]]. |
| There are totally 4 open bugs in this testing.
| |
|
| |
|
| BTW, This time I still build qemu-native from WR's branch (git.pokylinux.org:poky-wr +yocto-kernel-clean).
| | = Other Relevant Data= |
| | * [[Yocto Bug Trend]] |
| | * Compliance Test Result |
| | ** [[LTP result]] |
| | ** [[Posix result]] |
| | ** [[LSB Result]] |
| | * [[ADT Testing]] |
| | * [[Regression Test]] |
| | * [[Performance Test]] |
| | * [[Distro Test]] |
| | * [[Distribution Support]] |
| | * [[QA BKM sharing]] |
| | * [[LAVA server vs Yocto HW automation testing]] |
| | ** Note: The LAVA framework usage stopped in favor of testing in the AutoBuilder in early 2016. |
|
| |
|
| '''Issue List:'''
| | ==List of Automated Tests== |
| | *[[Distribution Support]] |
| | * add ptest wiki |
| | * add piglit test wiki |
| | * add kernel test wiki |
| | *[[LSB]] |
| | *[[LSB Result]] |
| | *[[LTP]] |
| | *[[LTP result]] |
| | *[[POSIX]] |
| | *[[Posix result]] |
| | *[[POSIX-results]] |
| | *[[POSIX History Results]] |
| | *[[Automated package upgrade testing]] |
|
| |
|
| * No mouse action response in X window with qemumips images
| | [[Category:QA]] |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=99
| |
| * qemuarm and qemuppc images cannot shutdown entirely
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=100
| |
| * OOM error when booting qemuppc images
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=102
| |
| * Some icons broken with qemumips images
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=106
| |
| | |
| '''Test Result Matrix:'''
| |
| | |
| ''SanityTest Result''
| |
| {| border="1"
| |
| |-
| |
| ! Machine
| |
| ! Image
| |
| ! Boot
| |
| ! Network(ping)
| |
| ! X startup
| |
| ! SSH Service
| |
| ! Shutdown
| |
| ! Comments
| |
| |-
| |
| ! rowspan="2"| qemux86
| |
| ! poky-image-sato
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green" | PASS
| |
| |
| |
| |-
| |
| ! poky-image-sdk
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green" | PASS
| |
| |
| |
| |-
| |
| ! rowspan="2"| qemuxarm
| |
| ! poky-image-sato
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green"| PASS
| |
| | style="color:red" | FAIL
| |
| |
| |
| |-
| |
| ! poky-image-sdk
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green"| PASS
| |
| | style="color:red" | FAIL
| |
| |
| |
| |-
| |
| ! rowspan="2"| qemumips
| |
| ! poky-image-sato
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green"| PASS
| |
| | style="color:red" | No mouse action response, some icons broken
| |
| |-
| |
| ! poky-image-sdk
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green"| PASS
| |
| | style="color:red" | No mouse action response, some icons broken
| |
| |-
| |
| ! rowspan="2"| qemuppc
| |
| ! poky-image-sato
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | N/A
| |
| | style="color:green" | PASS
| |
| | style="color:red"| FAIL
| |
| |
| |
| |-
| |
| ! poky-image-sdk
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | N/A
| |
| | style="color:green" | PASS
| |
| | style="color:red"| FAIL
| |
| |
| |
| |}
| |
| | |
| '''Detail Test Result'''
| |
| | |
| Test Result can be found here [https://git.pokylinux.org/share/QA_docs/milestone%20test/M1-Testing-20100710.xls Milestone Test Result for 20100710]
| |
| | |
| '''Testing Env'''
| |
| : Host: Ubuntu 10.04 LTS
| |
| : Kernel: 2.6.32-21-generic-pae
| |
| : Poky gitinfo(used to build qemu-native): aa416b6133bd55b5ba4d2fb6f5faef274ba511a0 (git.pokylinux.org:poky-wr +yocto-kernel-clean)
| |
| : Build Image Download URL: http://autobuilder.pokylinux.org/milestone/20100710-1/
| |
| | |
| | |
| ==== M1 Test Report 2010/7/8 ====
| |
| '''Summary & Issue List'''
| |
| | |
| I finished sanitytest for M1 20100708 build. There are 3 issues found in this testing.
| |
| The major issue is that there is no cursor in qemuarm and qemumips images. I also tried qemuppc because there is some fixing in latest tree, but qemuppc still fails to startup.
| |
| 5 issues found in last time testing are fixed. Qemuarm can boot up now. Qemuarm and qemumips can start X now.
| |
| | |
| BTW, This time I still build qemu-native from WR's branch (git.pokylinux.org:poky-wr +yocto-kernel-clean).
| |
| | |
| '''Issue List:'''
| |
| | |
| * No cursor in X window with qemuarm and qemumips images
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=99
| |
| * qemuarm images cannot shutdown entirely
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=100
| |
| * qemuppc images cannot boot up
| |
| ** http://bugzilla.pokylinux.org/show_bug.cgi?id=101
| |
| | |
| '''Test Result Matrix:'''
| |
| | |
| ''SanityTest Result''
| |
| {| border="1"
| |
| |-
| |
| ! Machine
| |
| ! Image
| |
| ! Boot
| |
| ! Network(ping)
| |
| ! X startup
| |
| ! SSH Service
| |
| ! Shutdown
| |
| ! Comments
| |
| |-
| |
| ! rowspan="2"| qemux86
| |
| ! poky-image-sato
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green" | PASS
| |
| |
| |
| |-
| |
| ! poky-image-sdk
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green" | PASS
| |
| |
| |
| |-
| |
| ! rowspan="2"| qemuxarm
| |
| ! poky-image-sato
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green"| PASS
| |
| | style="color:red" | FAIL
| |
| | style="color:red" | No cursor after X startup
| |
| |-
| |
| ! poky-image-sdk
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green"| PASS
| |
| | style="color:red" | FAIL
| |
| | style="color:red" | No cursor after X startup
| |
| |-
| |
| ! rowspan="2"| qemumips
| |
| ! poky-image-sato
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green"| PASS
| |
| | style="color:red" | No cursor after X startup
| |
| |-
| |
| ! poky-image-sdk
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | style="color:green"| PASS
| |
| | style="color:green"| PASS
| |
| | style="color:red" | No cursor after X startup
| |
| |}
| |
| | |
| '''Detail Test Result'''
| |
| | |
| Test Result can be found here [https://git.pokylinux.org/share/QA_docs/milestone%20test/M1-Testing-20100708.xls Milestone Test Result for 20100708]
| |
| | |
| '''Testing Env'''
| |
| : Host: Ubuntu 10.04 LTS
| |
| : Kernel: 2.6.32-21-generic-pae
| |
| : Poky gitinfo(used to build qemu-native): aa416b6133bd55b5ba4d2fb6f5faef274ba511a0 (git.pokylinux.org:poky-wr +yocto-kernel-clean)
| |
| : Build Image Download URL: http://autobuilder.pokylinux.org/milestone/20100708-1/
| |
| | |
| ==== M1 Test Report 2010/6/29 ====
| |
| '''Summary & Issue List'''
| |
| | |
| There are 8 target images be tested, only 3 ones can boot up, while SDK image for qemumips is not built out. Testing for qemuarm and qemuppc is blocked. Network in qemux86 image is not enabled. X failure is still within SDK image. And there is no cursor within qemux86 poky-image-sato. Detail issue list as below.
| |
| | |
| For BAT testing, LTP and POSIX testing are performed against qemux86 poky-image-sdk and qemumips poky-image-sato. Compared with last time test result, LTP result for qemux86 is same, while POSIX result has one more failure.
| |
| | |
| BTW, I build qemu-native by myself from poky WR’s branch (git.pokylinux.org:poky-wr). These qemu startup scripts, like “runqemu”, also come from the branch.
| |
| | |
| '''Issue List:'''
| |
| # No network in qemux86 image, (sato/sdk). It should be caused by lack of e1000 module built in WR kernel.
| |
| # No cursor after X startup in qemux86 poky-image-sato image.
| |
| # Qemux86 poky-image-sdk image fails to start X. (Known issue as Saul pointed)
| |
| # Fail to boot up qemuarm images, (sato/sdk). It seems no root partition when booting. See attachment “qemuarm-poky-image.log” for full log when booting QEMU. [https://git.pokylinux.org/share/QA_docs/milestone%20test/log-20100629/qemuarm-poky-image.log qemuarm log]
| |
| # Fail to boot up qemuppc images, (sato/sdk). qemu reports “could not load PPC PREP bios ‘powerpc_rom.bin’”. See attachment “qemuppc-poky-image.log” for full log. [https://git.pokylinux.org/share/QA_docs/milestone%20test/log-20100629/qemuppc-poky-image.log qemuppc log]
| |
| # By default, script “runqemu” does not start X for qemuarm, qemuppc and qemumips. I find there is no wnidow popup even I remove the X disable option, "-nographic" for qemumips.
| |
| | |
| '''Test Result Matrix:'''
| |
| | |
| ''SanityTest Result''
| |
| {| border="1"
| |
| |-
| |
| ! Machine
| |
| ! Image
| |
| ! Boot
| |
| ! Network(ping)
| |
| ! X startup
| |
| ! SSH Service
| |
| ! Shutdown
| |
| ! Comments
| |
| |-
| |
| ! rowspan="2"| qemux86
| |
| ! poky-image-sato
| |
| | style="color:green" | PASS
| |
| | style="color:red" | FAIL
| |
| | style="color:green"| PASS
| |
| | N/T
| |
| | style="color:green" | PASS
| |
| | style="color:red" | No cursor after X startup with show-cursor option
| |
| |-
| |
| ! poky-image-sdk
| |
| | style="color:green" | PASS
| |
| | style="color:red" | FAIL
| |
| | style="color:red"| FAIL
| |
| | N/T
| |
| | style="color:green" | PASS
| |
| |
| |
| |-
| |
| ! rowspan="2"| qemuxarm
| |
| ! poky-image-sato
| |
| | style="color:red" | FAIL
| |
| | style="color:red" | BLOCK
| |
| | style="color:red"| BLOCK
| |
| | style="color:red"| BLOCK
| |
| | style="color:red"| BLOCK
| |
| |
| |
| |-
| |
| ! poky-image-sdk
| |
| | style="color:red" | FAIL
| |
| | style="color:red" | BLOCK
| |
| | style="color:red"| BLOCK
| |
| | style="color:red"| BLOCK
| |
| | style="color:red"| BLOCK
| |
| |
| |
| |-
| |
| ! rowspan="2"| qemumips
| |
| ! poky-image-sato
| |
| | style="color:green" | PASS
| |
| | style="color:green" | PASS
| |
| | N/A
| |
| | style="color:green"| PASS
| |
| | style="color:green"| PASS
| |
| | style="color:red" | option "-nographic" is set in script "poky-qemu-internal" for qemumips. There is no window popup even I remove the option.
| |
| |-
| |
| ! poky-image-sdk
| |
| | N/A
| |
| | N/A
| |
| | N/A
| |
| | N/A
| |
| | N/A
| |
| | style="color:red" | No SDK images for qemumips
| |
| |-
| |
| ! rowspan="2"| qemuppc
| |
| ! poky-image-sato
| |
| | style="color:red" | FAIL
| |
| | style="color:red" | BLOCK
| |
| | style="color:red" | BLOCK
| |
| | style="color:red" | BLOCK
| |
| | style="color:red" | BLOCK
| |
| |
| |
| |-
| |
| ! poky-image-sdk
| |
| | style="color:red" | FAIL
| |
| | style="color:red" | BLOCK
| |
| | style="color:red" | BLOCK
| |
| | style="color:red" | BLOCK
| |
| | style="color:red" | BLOCK
| |
| |
| |
| |}
| |
| | |
| ''BAT Result:''
| |
| {| border="1"
| |
| |-
| |
| ! Machine
| |
| ! Image
| |
| ! LTP(FAIL/TOTAL)
| |
| ! POSIX(FAIL/UNSUPPORT/TOTAL)
| |
| |-
| |
| ! qemux86
| |
| ! poky-image-sdk
| |
| | style="color:red" | 46/1120
| |
| | style="color:red" | 51/99/1644
| |
| |-
| |
| ! qemumips
| |
| ! poky-image-sato
| |
| | style="color:red" | 58/1116
| |
| | N/A
| |
| |}
| |
| | |
| '''Detail Test Result'''
| |
| | |
| Test Result can be found here [https://git.pokylinux.org/share/QA_docs/milestone%20test/Nightly-M1-Testing-20100629.xls Milestone Test Result for 20100629]
| |
| | |
| '''Testing Env'''
| |
| : Host: Ubuntu 10.04 LTS
| |
| : Kernel: 2.6.32-21-generic-pae
| |
| : Poky gitinfo(used to build qemu-native): 5280cfe8c9cd9c9a880683186c77c24d380978ed (git.pokylinux.org:poky-wr)
| |
| : Build Image Download URL: Except for SDK images for qemuppc and qemuarm (from http://autobuilder.pokylinux.org/nightly/20100629-1/), all the other images come from http://autobuilder.pokylinux.org/nightly/20100628-1/.
| |
| | |
| === Enable Automation Test in Poky ===
| |
| Currently we have implemented two automated sanity test cases in Poky. User can follow below steps to enable the automation test:
| |
| | |
| * Make sure your working tree has latest test code. The first sanity test implementation is checked in poky master tree as commit, '''fd1d661b391dfad1edaac937a17c1165f635031a.
| |
| * Make sure your machine has package "expect" installed. The package is necessary for testing. We will add a recipe for it later. Currently, you need install it by yourself. On Ubuntu host, you can use apt-get to install expect:
| |
| <pre>
| |
| $ apt-get install expect
| |
| </pre>
| |
| * Edit /etc/sudoers to allow the user run sudo without prompted for a password. For example, you can do like following for user "tester":
| |
| <pre>
| |
| $ cat /etc/sudoers
| |
| # /etc/sudoers
| |
| #
| |
| # This file MUST be edited with the 'visudo' command as root.
| |
| #
| |
| # See the man page for details on how to write a sudoers file.
| |
| #
| |
| ....
| |
| ....
| |
| ....
| |
| | |
| %tester ALL=(ALL) NOPASSWD: NOPASSWD: ALL
| |
| </pre>
| |
| * Typically, you need a X environment to start QEMU. You can use your desktop's DISPLAY. Or, alternatively, you can install vncserver, and startup a X environment in localhost:x.0. (x can be 1, 2, 3.....n)
| |
| * You need set "IMAGETEST" to "qemu" in build/conf/local.conf
| |
| <pre>
| |
| $ cat local.conf
| |
| .....
| |
| # Set IMAGETEST to qemu if you want to build testcases and start
| |
| # testing in qemu after do_rootfs.
| |
| IMAGETEST = "qemu"
| |
| .....
| |
| </pre>
| |
| * Then you can run "bitbake poky-image-xxx" (xxx can be minimal, sato, sdk). After the kernel/image are generated, you need run "bitbake poky-image-xxx -c qemuimagetest". It will trigger automation test and you will get the result after a while. :)
| |
| | |
| ===Full Pass Specification===
| |
| | |
| ==== M2 Test Execution Plan ====
| |
| Since there are still some unclear requirements for Yocto project, currently it’s hard to define test plan to cover all future milestone/weekly testing. According to the suggestion from KangKang, I defined a test execution plan for M2 testing, which only covers tests for M2 requirement.
| |
| | |
| | |
| The fullpass test plan is here, [https://git.pokylinux.org/share/QA_docs/Test%20Plan/TC_test_plan-2010-08-11.doc M2 fullpass test plan]. For each target, like qemux86, qemuarm, netbook, e-Menlow, etc, I define separate test plan for them, with sub-tests chosen from the fullpass test plan.
| |
| | |
| I setup a testlink page to track test case and test plan status, http://otc-qa.sh.intel.com/testlink_yocto/. You can use username/passwd: guest/guest, to browser test case and test plan.
| |