Wiki Test Report Format: Difference between revisions
From Yocto Project
Jump to navigationJump to search
No edit summary |
No edit summary |
||
| Line 91: | Line 91: | ||
| '''Qemux86-64 Sato'''||28||0||<span style="color:green">28</span>||<span style="color:red">0</span>||<span style="color:red">0</span> | | '''Qemux86-64 Sato'''||28||0||<span style="color:green">28</span>||<span style="color:red">0</span>||<span style="color:red">0</span> | ||
|- | |- | ||
| '''Qemux86-64 Sato-SDK'''||31||0||<span style="color:green">31</span>||0||0 | | '''Qemux86-64 Sato-SDK'''||31||0||<span style="color:green">31</span>||<span style="color:red">0</span>||<span style="color:red">0</span> | ||
|- | |- | ||
| '''Qemux86 Sato'''||28||0||<span style="color:green">28</span>||0||0 | | '''Qemux86 Sato'''||28||0||<span style="color:green">28</span>||<span style="color:red">0</span>||<span style="color:red">0</span> | ||
|- | |- | ||
| '''Qemux86 Sato-SDK'''||31||0||<span style="color:green">31</span>||0||0 | | '''Qemux86 Sato-SDK'''||31||0||<span style="color:green">31</span>||<span style="color:red">0</span>||<span style="color:red">0</span> | ||
|- | |- | ||
| '''Qemumips Sato'''||26||0||<span style="color:green">26</span>||0||0 | | '''Qemumips Sato'''||26||0||<span style="color:green">26</span>||<span style="color:red">0</span>||<span style="color:red">0</span> | ||
|- | |- | ||
| '''Qemumips Sato-SDK'''||32||0||<span style="color:green">32</span>||0||0 | | '''Qemumips Sato-SDK'''||32||0||<span style="color:green">32</span>||<span style="color:red">0</span>||<span style="color:red">0</span> | ||
|- | |- | ||
| '''Qemuppc Sato'''||20||0||<span style="color:green">19</span>||<span style="color:red">0</span>||<span style="color:red">1 (bug 414)</span> | | '''Qemuppc Sato'''||20||0||<span style="color:green">19</span>||<span style="color:red">0</span>||<span style="color:red">1 (bug 414)</span> | ||
| Line 105: | Line 105: | ||
| '''Qemuppc Sato-SDK'''||26||0||<span style="color:green">25</span>||<span style="color:red">0</span>||<span style="color:red">1 (bug 414) </span> | | '''Qemuppc Sato-SDK'''||26||0||<span style="color:green">25</span>||<span style="color:red">0</span>||<span style="color:red">1 (bug 414) </span> | ||
|- | |- | ||
| '''Qemuarm Sato'''||26||0||<span style="color:green">26</span>||0||0 | | '''Qemuarm Sato'''||26||0||<span style="color:green">26</span>||<span style="color:red">0</span>||<span style="color:red">0</span> | ||
|- | |- | ||
| '''Qemuarm Sato-SDK'''||32||0||<span style="color:green">32</span>||0||0 | | '''Qemuarm Sato-SDK'''||32||0||<span style="color:green">32</span>||<span style="color:red">0</span>||<span style="color:red">0</span> | ||
|- | |- | ||
| '''Core Build System'''||8||0||<span style="color:green">7</span>||<span style="color:red">1 (bug 1133)</span>||<span style="color:red">0</span> | | '''Core Build System'''||8||0||<span style="color:green">7</span>||<span style="color:red">1 (bug 1133)</span>||<span style="color:red">0</span> | ||
| Line 113: | Line 113: | ||
| '''ADT'''||6||0||<span style="color:green">5</span>||<span style="color:red">1 (bug 414)</span>||<span style="color:red">0</span> | | '''ADT'''||6||0||<span style="color:green">5</span>||<span style="color:red">1 (bug 414)</span>||<span style="color:red">0</span> | ||
|- | |- | ||
| '''Power/Performance'''||4||0||<span style="color:green">4</span>||0||0 | | '''Power/Performance'''||4||0||<span style="color:green">4</span>||<span style="color:red">0</span>||<span style="color:red">0</span> | ||
|- | |- | ||
| '''Stress'''||2||0||<span style="color:green">2</span>||0||0 | | '''Stress'''||2||0||<span style="color:green">2</span>||<span style="color:red">0</span>||<span style="color:red">0</span> | ||
|- | |- | ||
| '''Total'''||637||0||<span style="color:green">600</span>||<span style="color:red">12</span>||<span style="color:red">21</span> | | '''Total'''||637||0||<span style="color:green">600</span>||<span style="color:red">12</span>||<span style="color:red">21</span> | ||
Revision as of 08:07, 7 June 2011
This page is to define Test Report Format for Yocto
Test Summary
Here is a test result summary for a build.
Test Result Matrix
A general Test Result Summary and a detailed Test Result Summary should be provided here.
Example
| Test Result Summary | ||||
|---|---|---|---|---|
| Component | Target | Status | Comments | |
| BSP | SugarBay | GOOD | Zypper install/remove has issue; 3D game running failed; | |
| CrownBay | BLOCK | emgd crownbay build failed; matchbox-panel segfault; S3 fail on B0 board | ||
| JasperForest | GOOD | The only issue is zypper install/remove bug | ||
| Blacksand | Buggy | mouse/keyboard do not work in X window; zypper install/remove not work well | ||
| Beagleboard | GOOD | Only one RTC issue exists | ||
| Routerstationpro | GOOD | Only zypper install has issue | ||
| Mpc8315e-rdb | GOOD | Only zypper install has issue | ||
| QEMU | qemux86 | Buggy | Only zypper install/remove has issue | |
| qemux86-64 | Buggy | Only zypper install/remove has issue | ||
| qemuarm | Buggy | zypper could not work well | ||
| qemuppc | Buggy | zypper could not work well | ||
| qemumips | Buggy | Only zypper install has issue | ||
| ADT | BLOCK | i686 toolchain not copied to sharing folder; unfs does not work with qemuppc; | ||
| Core Build System | BUGGY | sstate and non-gplv3 build do not work correctly | ||
| Stress | GOOD | Sugarbay pass 18 hours stress and Jasperforest pass 12 hours stress | ||
| Compliance | GOOD | Sugarbay pass LTP and POSIX test | ||
| Power/Performance | GOOD | |||
Note:
| BLOCK | Critical bugs, more than 50% test cases are blocked |
| GOOD | Only Normal, Minor or Enhancement bugs, less than 10% test cases failed |
| BUGGY | Normal, Major and Critical bugs, more than 10% test cases failed |
| Detailed Test Result for each component | |||||
|---|---|---|---|---|---|
| Target | Total TCs | Not Run | Passed | Failed | Not testable (Blocked) |
| Sugarbay Sato-SDK | 64 | 0 | 60 | 1 (bug 1101) | 3 (bug 1112, 883) |
| Crownbay Sato-SDK | 60 | 0 | 55 | 1 (bug 1099) | 4 |
| Jasperforest LSB-SDK | 34 | 0 | 34 | 0 | 0 |
| n450 Sato-SDK | 60 | 0 | 52 | 2 (bug 613, 1099) | 6 (bug 613) |
| eMenlow Sato-SDK | 60 | 0 | 48 | 2 (bug 1099, 503) | 6 (bug 503) |
| Routerstationpro Sato-SDK | 29 | 0 | 29 | 0 | 0 |
| Mpc8315e-rdb Sato-SDK | 30 | 0 | 26 | 4 (bug 766) | 0 |
| Qemux86-64 Sato | 28 | 0 | 28 | 0 | 0 |
| Qemux86-64 Sato-SDK | 31 | 0 | 31 | 0 | 0 |
| Qemux86 Sato | 28 | 0 | 28 | 0 | 0 |
| Qemux86 Sato-SDK | 31 | 0 | 31 | 0 | 0 |
| Qemumips Sato | 26 | 0 | 26 | 0 | 0 |
| Qemumips Sato-SDK | 32 | 0 | 32 | 0 | 0 |
| Qemuppc Sato | 20 | 0 | 19 | 0 | 1 (bug 414) |
| Qemuppc Sato-SDK | 26 | 0 | 25 | 0 | 1 (bug 414) |
| Qemuarm Sato | 26 | 0 | 26 | 0 | 0 |
| Qemuarm Sato-SDK | 32 | 0 | 32 | 0 | 0 |
| Core Build System | 8 | 0 | 7 | 1 (bug 1133) | 0 |
| ADT | 6 | 0 | 5 | 1 (bug 414) | 0 |
| Power/Performance | 4 | 0 | 4 | 0 | 0 |
| Stress | 2 | 0 | 2 | 0 | 0 |
| Total | 637 | 0 | 600 | 12 | 21 |
