Milestone Test Report

Milestone Test Report for Yocto

Fullpass Test Report for M3-20100911

Fullpass Test Report for M3-20100911

Sanity Test Report for M3-20100911

Sanity Test Report for M3-20100911

Sanity Test Report for M3-20100909

Sanity Test Report for M3-20100911

Partial Sanity Test Report for M3 20100905

Partial Test Report for M3 20100905

Fullpass Test for M2-20100731

Fullpass Test Report for M2-20100731

M2-20100725 & Bug Update for M2-20100728

I finished testing on QEMU/Netbook for M2-20100725 and verified some bugs with M2-20100728 build. Against M2-20100725 build, I reported 12 new bugs and verified 2 bugs as fixed. With the second build M2-20100728, I verified old critical bugs and reported another 2 new bugs. 2 old bugs are verified as fixed with this build.

M3 Test Execution Plan

M3 fullpass test plan.

M4 Test Execution Plan

M4 fullpass test plan.

There are still 5 open bugs for M2, 4 of them are CRITICAL. And 23 open bugs are marked to M3. As a summary, QEMU targets (x86/arm/mips/ppc) can work without critical bugs. Toolchain has one critical bug. Meanwhile, emenlow/netbook have critical issue with/within X environment.

Open Bugs for M2:

• Wrong qemu binary is used if qemu-native isn't built -- (Richard, is the issue going to be fixed in M2?)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=123
• Mouse/Keyboard cannot work with poky-image-sato-live on netbook -- (It should be fixed by Jeff, but mouse/kbd still not work because of bug 171)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=141
• [emenlow] X not bootable of image poky-image-sato-live-emenlow -- (This bug is with 0728 image)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=159
• [toolchain] Cannot build LTP with ppc/arm toolchain
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=161
• [Netbook] Xserver version mismatch makes mouse/keyboard not work
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=171

Other Open Bugs reported, target for M3 or later:

• It takes long time to boot qemuppc image at the first time -- Enhance/Low
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=143
• avahi-daemon fail to start with qemumips sdk image -- Minor/Medium (Target on M3)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=144
• nm-applet segfault with qemux86 sato/sdk image -- Normal/Medium (Not our focus)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=145
• keystroke combination ctrl-alt-del not work on netbook -- Normal/Low
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=146
• [Netbook] pcmanfm no response after I click File Manager icon -- Major/Medium (Target on M3)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=148
• [All] There is no way to exit matchbox-keyboard in X -- Normal/Medium (Target on M3)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=149
• [Netbook] Music Player cannot play WAV format file -- Major/Undecided
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=151
• [emenlow] command reboot cannot make system reboot entirely -- Medium/Normal (Target on M3)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=160
• [LTP] LTP default test suite cannot fully pass -- Normal/Low (Target on M4)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=169
• [POSIX] POSIX default test suite cannot fully pass -- Normal/Low (Target on M4)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=172

Verified Fixed Issues:

• qemuppc images cannot boot up
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=101
• Some icons broken with qemumips images
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=106
• qemumips poky-image-sato/sdk fail to boot -- (Fixed in 0728 build)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=138
• emenlow live image cannot boot/install -- (Fixed in 0728 build)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=142

Existing Old Issues:

• No mouse action response in X window with qemumips images -- Major/High (Target on M3)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=99
• qemuarm and qemuppc images cannot shutdown entirely -- Normal/High (Target on M3)
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=100

M1 Test Report 2010/7/10

Summary & Issue List

I finished sanitytest for M1 20100710 build. There is 2 new issues found in this testing. qemuppc boot up issue is fixed but qemu will report OOM error when booting qemuppc images. There is no mouse action response after X startup with qemumips images. qemuppc and qemuarm cannot shutdown entirely. There are totally 4 open bugs in this testing.

BTW, This time I still build qemu-native from WR's branch (git.pokylinux.org:poky-wr +yocto-kernel-clean).

Issue List:

• No mouse action response in X window with qemumips images
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=99
• qemuarm and qemuppc images cannot shutdown entirely
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=100
• OOM error when booting qemuppc images
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=102
• Some icons broken with qemumips images
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=106

Test Result Matrix:

SanityTest Result

Machine	Image	Boot	Network(ping)	X startup	SSH Service	Shutdown	Comments
qemux86	poky-image-sato	PASS	PASS	PASS	PASS	PASS
	poky-image-sdk	PASS	PASS	PASS	PASS	PASS
qemuxarm	poky-image-sato	PASS	PASS	PASS	PASS	FAIL
	poky-image-sdk	PASS	PASS	PASS	PASS	FAIL
qemumips	poky-image-sato	PASS	PASS	PASS	PASS	PASS	No mouse action response, some icons broken
	poky-image-sdk	PASS	PASS	PASS	PASS	PASS	No mouse action response, some icons broken
qemuppc	poky-image-sato	PASS	PASS	N/A	PASS	FAIL
	poky-image-sdk	PASS	PASS	N/A	PASS	FAIL

Detail Test Result

Test Result can be found here Milestone Test Result for 20100710

Testing Env

Host: Ubuntu 10.04 LTS

Kernel: 2.6.32-21-generic-pae

Poky gitinfo(used to build qemu-native): aa416b6133bd55b5ba4d2fb6f5faef274ba511a0 (git.pokylinux.org:poky-wr +yocto-kernel-clean)

Build Image Download URL: http://autobuilder.pokylinux.org/milestone/20100710-1/

M1 Test Report 2010/7/8

Summary & Issue List

I finished sanitytest for M1 20100708 build. There are 3 issues found in this testing. The major issue is that there is no cursor in qemuarm and qemumips images. I also tried qemuppc because there is some fixing in latest tree, but qemuppc still fails to startup. 5 issues found in last time testing are fixed. Qemuarm can boot up now. Qemuarm and qemumips can start X now.

BTW, This time I still build qemu-native from WR's branch (git.pokylinux.org:poky-wr +yocto-kernel-clean).

Issue List:

• No cursor in X window with qemuarm and qemumips images
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=99
• qemuarm images cannot shutdown entirely
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=100
• qemuppc images cannot boot up
  • http://bugzilla.pokylinux.org/show_bug.cgi?id=101

Test Result Matrix:

SanityTest Result

Machine	Image	Boot	Network(ping)	X startup	SSH Service	Shutdown	Comments
qemux86	poky-image-sato	PASS	PASS	PASS	PASS	PASS
	poky-image-sdk	PASS	PASS	PASS	PASS	PASS
qemuxarm	poky-image-sato	PASS	PASS	PASS	PASS	FAIL	No cursor after X startup
	poky-image-sdk	PASS	PASS	PASS	PASS	FAIL	No cursor after X startup
qemumips	poky-image-sato	PASS	PASS	PASS	PASS	PASS	No cursor after X startup
	poky-image-sdk	PASS	PASS	PASS	PASS	PASS	No cursor after X startup

Detail Test Result

Test Result can be found here Milestone Test Result for 20100708

Testing Env

Host: Ubuntu 10.04 LTS

Kernel: 2.6.32-21-generic-pae

Poky gitinfo(used to build qemu-native): aa416b6133bd55b5ba4d2fb6f5faef274ba511a0 (git.pokylinux.org:poky-wr +yocto-kernel-clean)

Build Image Download URL: http://autobuilder.pokylinux.org/milestone/20100708-1/

M1 Test Report 2010/6/29

Summary & Issue List

There are 8 target images be tested, only 3 ones can boot up, while SDK image for qemumips is not built out. Testing for qemuarm and qemuppc is blocked. Network in qemux86 image is not enabled. X failure is still within SDK image. And there is no cursor within qemux86 poky-image-sato. Detail issue list as below.

For BAT testing, LTP and POSIX testing are performed against qemux86 poky-image-sdk and qemumips poky-image-sato. Compared with last time test result, LTP result for qemux86 is same, while POSIX result has one more failure.

BTW, I build qemu-native by myself from poky WR’s branch (git.pokylinux.org:poky-wr). These qemu startup scripts, like “runqemu”, also come from the branch.

Issue List:

1. No network in qemux86 image, (sato/sdk). It should be caused by lack of e1000 module built in WR kernel.
2. No cursor after X startup in qemux86 poky-image-sato image.
3. Qemux86 poky-image-sdk image fails to start X. (Known issue as Saul pointed)
4. Fail to boot up qemuarm images, (sato/sdk). It seems no root partition when booting. See attachment “qemuarm-poky-image.log” for full log when booting QEMU. qemuarm log
5. Fail to boot up qemuppc images, (sato/sdk). qemu reports “could not load PPC PREP bios ‘powerpc_rom.bin’”. See attachment “qemuppc-poky-image.log” for full log. qemuppc log
6. By default, script “runqemu” does not start X for qemuarm, qemuppc and qemumips. I find there is no wnidow popup even I remove the X disable option, "-nographic" for qemumips.

Test Result Matrix:

SanityTest Result

Machine	Image	Boot	Network(ping)	X startup	SSH Service	Shutdown	Comments
qemux86	poky-image-sato	PASS	FAIL	PASS	N/T	PASS	No cursor after X startup with show-cursor option
	poky-image-sdk	PASS	FAIL	FAIL	N/T	PASS
qemuxarm	poky-image-sato	FAIL	BLOCK	BLOCK	BLOCK	BLOCK
	poky-image-sdk	FAIL	BLOCK	BLOCK	BLOCK	BLOCK
qemumips	poky-image-sato	PASS	PASS	N/A	PASS	PASS	option "-nographic" is set in script "poky-qemu-internal" for qemumips. There is no window popup even I remove the option.
	poky-image-sdk	N/A	N/A	N/A	N/A	N/A	No SDK images for qemumips
qemuppc	poky-image-sato	FAIL	BLOCK	BLOCK	BLOCK	BLOCK
	poky-image-sdk	FAIL	BLOCK	BLOCK	BLOCK	BLOCK

BAT Result:

Machine	Image	LTP(FAIL/TOTAL)	POSIX(FAIL/UNSUPPORT/TOTAL)
qemux86	poky-image-sdk	46/1120	51/99/1644
qemumips	poky-image-sato	58/1116	N/A

Detail Test Result

Test Result can be found here Milestone Test Result for 20100629

Testing Env

Host: Ubuntu 10.04 LTS

Kernel: 2.6.32-21-generic-pae

Poky gitinfo(used to build qemu-native): 5280cfe8c9cd9c9a880683186c77c24d380978ed (git.pokylinux.org:poky-wr)

Build Image Download URL: Except for SDK images for qemuppc and qemuarm (from http://autobuilder.pokylinux.org/nightly/20100629-1/), all the other images come from http://autobuilder.pokylinux.org/nightly/20100628-1/.