Wiki Test Report Format: Difference between revisions

Revision as of 07:19, 3 May 2011

This page is to define Test Report Format for Yocto

Here is a test result summary for a build.

A general Test Result Summary and a detailed Test Result Summary should be provided here.

Test Result Summary

Component	Target	Status	Comments
BSP	SugarBay	GOOD	Zypper install/remove has issue; 3D game running failed;
	CrownBay	BLOCK	emgd crownbay build failed; matchbox-panel segfault; S3 fail on B0 board
	JasperForest	GOOD	The only issue is zypper install/remove bug
	Blacksand	Buggy	mouse/keyboard do not work in X window; zypper install/remove not work well
	Beagleboard	GOOD	Only one RTC issue exists
	Routerstationpro	GOOD	Only zypper install has issue
	Mpc8315e-rdb	GOOD	Only zypper install has issue
QEMU	qemux86	Buggy	Only zypper install/remove has issue
	qemux86-64	Buggy	Only zypper install/remove has issue
	qemuarm	Buggy	zypper could not work well
	qemuppc	Buggy	zypper could not work well
	qemumips	Buggy	Only zypper install has issue
SDK		BLOCK	i686 toolchain not copied to sharing folder; unfs does not work with qemuppc;
Poky		BUGGY	sstate and non-gplv3 build do not work correctly
Stress		GOOD	Sugarbay pass 18 hours stress and Jasperforest pass 12 hours stress
Compliance		GOOD	Sugarbay pass LTP and POSIX test
Documentation		GOOD

Detailed Test Result Summary

Test Target	System & Core OS	SDK	Compliance	Stress	Power/Performance	Graphics	Mulitimedia
qemux86	Buggy	Good	n/a	n/a	n/a	n/a	n/a
qemux86-64	Buggy	Good	n/a	n/a	n/a	n/a	n/a
qemuarm	Buggy	Good	n/a	n/a	n/a	n/a	n/a
qemumips	Buggy	Good	n/a	n/a	n/a	n/a	n/a
qemuppc	Buggy	Good	n/a	n/a	n/a	n/a	n/a
SugarBay	Good	n/a	Buggy	Good	Good (May add Link to P&P result page)	Good	Good
CrownBay	Good	n/a	Good	n/a	n/a	Good	Good
CrownBay noemgd	n/a	n/a	n/a	n/a	n/a	n/a	n/a
FishRiver Island 2	Good	n/a	Buggy	n/a	Good (May add Link to P&P result page)	Good	Good
HuronRiver	BLOCK	n/a	n/a	n/a	BLOCK	BLOCK	BLOCK
Jasperforest	BLOCK	n/a	BLOCK	Good	n/a	n/a	n/a
Blacksand	Buggy	n/a	n/a	n/a	n/a	n/a	n/a
eMenlow	Good	n/a	n/a	n/a	n/a	n/a	n/a

Note:

BLOCK	Critical bugs, more than 50% test cases are blocked
GOOD	Only Normal, Minor or Enhancement bugs, less than 10% test cases failed
BUGGY	Normal, Major and Critical bugs, more than 10% test cases failed

@@ Line 53: / Line 53: @@
 '''Detailed Test Result Summary'''
+{| {{table}}
+| align="center" style="background:#f0f0f0;"|'''Test Target'''
+| align="center" style="background:#f0f0f0;"|'''System & Core OS'''
+| align="center" style="background:#f0f0f0;"|'''SDK'''
+| align="center" style="background:#f0f0f0;"|'''Compliance'''
+| align="center" style="background:#f0f0f0;"|'''Stress'''
+| align="center" style="background:#f0f0f0;"|'''Power/Performance'''
+| align="center" style="background:#f0f0f0;"|'''Graphics'''
+| align="center" style="background:#f0f0f0;"|'''Mulitimedia'''
+|-
+| qemux86||Buggy||Good||n/a||n/a||n/a||n/a||n/a
+|-
+| qemux86-64||Buggy||Good||n/a||n/a||n/a||n/a||n/a
+|-
+| qemuarm||Buggy||Good||n/a||n/a||n/a||n/a||n/a
+|-
+| qemumips||Buggy||Good||n/a||n/a||n/a||n/a||n/a
+|-
+| qemuppc||Buggy||Good||n/a||n/a||n/a||n/a||n/a
+|-
+| SugarBay||Good||n/a||Buggy||Good||Good (May add Link to P&P result page)||Good||Good
+|-
+| CrownBay||Good||n/a||Good||n/a||n/a||Good||Good
+|-
+| CrownBay noemgd||n/a||n/a||n/a||n/a||n/a||n/a||n/a
+|-
+| FishRiver Island 2||Good||n/a||Buggy||n/a||Good (May add Link to P&P result page)||Good||Good
+|-
+| HuronRiver||BLOCK||n/a||n/a||n/a||BLOCK||BLOCK||BLOCK
+|-
+| Jasperforest||BLOCK||n/a||BLOCK||Good||n/a||n/a||n/a
+|-
+| Blacksand||Buggy||n/a||n/a||n/a||n/a||n/a||n/a
+|-
+| eMenlow||Good||n/a||n/a||n/a||n/a||n/a||n/a
+|-
+|
+|}
 '''Note:'''