LTP result: Difference between revisions
From Yocto Project
Jump to navigationJump to search
(Created page with '==Introduction== Linux Test Project (LTP) is an open source project with a goal to deliver test suites to the open source community that validate the reliability, robustness, and…') |
No edit summary |
||
| Line 7: | Line 7: | ||
* to be the reference if Yocto users want to get a base picture | * to be the reference if Yocto users want to get a base picture | ||
==Overall matrix== | |||
== | Overall matrix tends to track the change of LTP test results along with Yocto's progress. | ||
For each LTP failure, we define below status: | |||
* '''Not start''': not start investigation yet | |||
* '''Pending''': in investigation | |||
* '''Fixed''': fixed, which is expected to disappear in next milestone (bug link may be attached in comment) | |||
* '''Not A Bug''': if the failure is simply caused by missing some packages in current profile (the alternative profile may be listed in the comment if it works) | |||
In below table we list total LTP failures for each architectures in each milestone, with the number in parenthesis indicating "Fixed" number and "Not A Bug number". This way the owner of each machine is able to update the column independently. | |||
{|border="1" | |||
|| '''Milestone''' || '''qemux86''' || '''qemuppc''' || '''qemuarm''' || '''qemumips''' || '''atom-pc''' | |||
|- | |||
|| 1.0-M2 || 36(0/0) || n/a || n/a || n/a || n/a | |||
|} | |||
Below table includes which LTP categories are included in the measurement: | |||
{|border="1" | |||
|| '''Milestone''' || '''qemux86''' || '''qemuppc''' || '''qemuarm''' || '''qemumips''' || '''atom-pc''' | |||
|- | |||
|| 1.0-M2 | |||
|| | |||
syscalls | |||
mm | |||
ipc | |||
sched | |||
math | |||
nptl | |||
pty | |||
admin_tools | |||
timers | |||
command | |||
|| n/a || n/a || n/a || n/a | |||
|} | |||
==Current Milestone (1.0-M2)== | |||
'''LTP''': ltp-full-20100430.tgz | '''LTP''': ltp-full-20100430.tgz | ||
| Line 20: | Line 60: | ||
'''run command''': ./runltp -p -l M2-sdk-20101216.log 1>&M2-sdk-20101216-full.log | '''run command''': ./runltp -p -l M2-sdk-20101216.log 1>&M2-sdk-20101216-full.log | ||
{|border="1" | {|border="1" | ||
| Line 97: | Line 138: | ||
|- | |- | ||
|} | |} | ||
==Previous milestone== | |||
a placeholder. Once we reach a new milestone, the data under "current milestone" will be moved here. | |||
Revision as of 09:57, 17 December 2010
Introduction
Linux Test Project (LTP) is an open source project with a goal to deliver test suites to the open source community that validate the reliability, robustness, and stability of Linux. (http://ltp.sourceforge.net)
Unlike typical desktop/server distributions, embedded linux distribution is highly customized for a specific purpose. It's not unusual to observe more failures in Yocto than in a typical desktop Linux distro like Ubuntu, etc. So the major purposes of this work are:
- to understand the reasons for existing LTP failures. If the cause is from a specific package itself, go to fix it. Or else if the error simply come from the fact that a given package doesn't exist in current profile, add the explanation here
- to track LTP test results cross Yocto releases, and ensure a declining error number for same profile
- to be the reference if Yocto users want to get a base picture
Overall matrix
Overall matrix tends to track the change of LTP test results along with Yocto's progress.
For each LTP failure, we define below status:
- Not start: not start investigation yet
- Pending: in investigation
- Fixed: fixed, which is expected to disappear in next milestone (bug link may be attached in comment)
- Not A Bug: if the failure is simply caused by missing some packages in current profile (the alternative profile may be listed in the comment if it works)
In below table we list total LTP failures for each architectures in each milestone, with the number in parenthesis indicating "Fixed" number and "Not A Bug number". This way the owner of each machine is able to update the column independently.
| Milestone | qemux86 | qemuppc | qemuarm | qemumips | atom-pc |
| 1.0-M2 | 36(0/0) | n/a | n/a | n/a | n/a |
Below table includes which LTP categories are included in the measurement:
| Milestone | qemux86 | qemuppc | qemuarm | qemumips | atom-pc |
| 1.0-M2 |
syscalls mm ipc sched math nptl pty admin_tools timers command |
n/a | n/a | n/a | n/a |
Current Milestone (1.0-M2)
LTP: ltp-full-20100430.tgz
Profile: poky-image-sdk
run command: ./runltp -p -l M2-sdk-20101216.log 1>&M2-sdk-20101216-full.log
| Name | life cycle | Machine | Profile | Status | Owner | Comments / Bugzilla Links |
| chdir01A | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | symlink error |
| ioctl03 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| lchown02 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| lchown02_16 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| splice02 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| sysctl03 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| syslog01 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| syslog02 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| syslog03 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| syslog04 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| syslog05 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| syslog07 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| syslog08 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| syslog09 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| syslog10 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| utimensat01 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| mtest06_2 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| mmapstress06 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| su01 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| cron02 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| cron_deny01 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| cron_allow01 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| cron_dirs_checks01 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| at_deny01 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| at_allow01 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| clock_gettime03 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| timer_create04 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| ld | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| ldd | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| file | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| tar | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| cron | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| logrotate | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| cpio | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| unzip01 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian | |
| gzip01 | 1.0-M2 | Qemux86 | poky-image-sdk | Pending | Kevin Tian |
Previous milestone
a placeholder. Once we reach a new milestone, the data under "current milestone" will be moved here.
