LTP result: Difference between revisions

From Yocto Project
Jump to navigationJump to search
No edit summary
No edit summary
Line 14: Line 14:
* '''Not start''': not start investigation yet
* '''Not start''': not start investigation yet
* '''Pending''': in investigation
* '''Pending''': in investigation
* '''LTP''': fail on other distribution too
* '''Fixed''': fixed, which is expected to disappear in next milestone (bug link may be attached in comment)
* '''Fixed''': fixed, which is expected to disappear in next milestone (bug link may be attached in comment)
* '''Not A Bug''': if the failure is simply caused by missing some packages in current profile (the alternative profile may be listed in the comment if it works)
* '''Not A Bug''': if the failure is simply caused by missing some packages in current profile (the alternative profile may be listed in the comment if it works)


In below table we list total LTP failures for each architectures in each milestone, with the number in parenthesis indicating "Fixed" number and "Not A Bug number". This way the owner of each machine is able to update the column independently.
In below table we list total LTP failures for each architectures in each milestone, with the number in parenthesis indicating "Native"/"Fixed"/"Not A Bug number". All log files on a machine is compressed and linked here too. This way the owner of each machine is able to update the column independently.


{|border="1"
{|border="1"
|| '''Milestone''' || '''qemux86''' || '''qemuppc''' || '''qemuarm''' || '''qemumips''' || '''atom-pc'''
|| '''Milestone''' || '''qemux86''' || '''qemuppc''' || '''qemuarm''' || '''qemumips''' || '''atom-pc'''
|-
|-
|| 1.0-M2 || 36(0/0) || n/a || n/a || n/a || n/a
|| 1.0-M2  
|| 36(0/0/0)  
|| n/a || n/a || n/a || n/a
|}
|}


Line 55: Line 58:
==Current Milestone (1.0-M2)==
==Current Milestone (1.0-M2)==


'''LTP''': ltp-full-20100430.tgz
This show details info about each failure in a milestone, merged from the results from all machines:
 
'''LTP version''': ltp-full-20100430.tgz


'''Profile''': poky-image-sdk
'''Profile''': poky-image-sdk
Line 63: Line 68:


{|border="1"
{|border="1"
|| '''Name''' || '''life cycle''' || '''Machine''' || '''Profile''' || '''Status''' || '''Owner''' || '''Comments / Bugzilla Links'''  
|| '''Name''' || '''Machine''' || '''Profile''' || '''Status''' || '''Owner''' || '''Comments / Bugzilla Links'''  
|-
|-
||chdir01A || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian || symlink error
||chdir01A || Qemux86 || poky-image-sdk || Pending || Kevin Tian || symlink error
|-
|-
||ioctl03 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||
||ioctl03 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||
|-
|-
||lchown02 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||lchown02 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||lchown02_16 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||lchown02_16 ||Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||splice02 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||splice02 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||sysctl03 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||sysctl03 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||syslog01 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||syslog01 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||syslog02 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||syslog02 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||syslog03 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||syslog03 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||syslog04 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||syslog04 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||syslog05 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||syslog05 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||syslog07 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||syslog07 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||syslog08 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||syslog08 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||syslog09 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||syslog09 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||syslog10 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||syslog10 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||utimensat01 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||utimensat01 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||mtest06_2 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||mtest06_2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||mmapstress06 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||mmapstress06 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||su01 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||su01 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||cron02 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||cron02 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||cron_deny01 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||cron_deny01 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||cron_allow01 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||cron_allow01 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||cron_dirs_checks01 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||cron_dirs_checks01 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||at_deny01 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||at_deny01 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||at_allow01 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||at_allow01 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||clock_gettime03 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||clock_gettime03 || Qemux86 || poky-image-sdk || Pending || Kevin Tian || get unexpected error number
|-
|-
||timer_create04 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||timer_create04 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||ld || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||ld || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||ldd || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||ldd || Qemux86 || poky-image-sdk || Not A Bug || Kevin Tian || No 'file' and 'ldd' in sdk image
|-
|-
||file || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||file || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||tar || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||tar || Qemux86 || poky-image-sdk || Not A Bug || Kevin Tian || 'file' is not available in sdk image
|-
|-
||cron || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||cron || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||logrotate || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||logrotate || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||cpio || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||cpio || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||unzip01 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||unzip01 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
||gzip01 || 1.0-M2 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
||gzip01 || Qemux86 || poky-image-sdk || Pending || Kevin Tian ||  
|-
|-
|}
|}

Revision as of 10:17, 17 December 2010

Introduction

Linux Test Project (LTP) is an open source project with a goal to deliver test suites to the open source community that validate the reliability, robustness, and stability of Linux. (http://ltp.sourceforge.net)

Unlike typical desktop/server distributions, embedded linux distribution is highly customized for a specific purpose. It's not unusual to observe more failures in Yocto than in a typical desktop Linux distro like Ubuntu, etc. So the major purposes of this work are:

  • to understand the reasons for existing LTP failures. If the cause is from a specific package itself, go to fix it. Or else if the error simply come from the fact that a given package doesn't exist in current profile, add the explanation here
  • to track LTP test results cross Yocto releases, and ensure a declining error number for same profile
  • to be the reference if Yocto users want to get a base picture

Overall matrix

Overall matrix tends to track the change of LTP test results along with Yocto's progress.

For each LTP failure, we define below status:

  • Not start: not start investigation yet
  • Pending: in investigation
  • LTP: fail on other distribution too
  • Fixed: fixed, which is expected to disappear in next milestone (bug link may be attached in comment)
  • Not A Bug: if the failure is simply caused by missing some packages in current profile (the alternative profile may be listed in the comment if it works)

In below table we list total LTP failures for each architectures in each milestone, with the number in parenthesis indicating "Native"/"Fixed"/"Not A Bug number". All log files on a machine is compressed and linked here too. This way the owner of each machine is able to update the column independently.

Milestone qemux86 qemuppc qemuarm qemumips atom-pc
1.0-M2 36(0/0/0) n/a n/a n/a n/a

Below table includes which LTP categories are included in the measurement:

Milestone qemux86 qemuppc qemuarm qemumips atom-pc
1.0-M2

syscalls

mm

ipc

sched

math

nptl

pty

admin_tools

timers

command

n/a n/a n/a n/a

Current Milestone (1.0-M2)

This show details info about each failure in a milestone, merged from the results from all machines:

LTP version: ltp-full-20100430.tgz

Profile: poky-image-sdk

run command: ./runltp -p -l M2-sdk-20101216.log 1>&M2-sdk-20101216-full.log


Name Machine Profile Status Owner Comments / Bugzilla Links
chdir01A Qemux86 poky-image-sdk Pending Kevin Tian symlink error
ioctl03 Qemux86 poky-image-sdk Pending Kevin Tian
lchown02 Qemux86 poky-image-sdk Pending Kevin Tian
lchown02_16 Qemux86 poky-image-sdk Pending Kevin Tian
splice02 Qemux86 poky-image-sdk Pending Kevin Tian
sysctl03 Qemux86 poky-image-sdk Pending Kevin Tian
syslog01 Qemux86 poky-image-sdk Pending Kevin Tian
syslog02 Qemux86 poky-image-sdk Pending Kevin Tian
syslog03 Qemux86 poky-image-sdk Pending Kevin Tian
syslog04 Qemux86 poky-image-sdk Pending Kevin Tian
syslog05 Qemux86 poky-image-sdk Pending Kevin Tian
syslog07 Qemux86 poky-image-sdk Pending Kevin Tian
syslog08 Qemux86 poky-image-sdk Pending Kevin Tian
syslog09 Qemux86 poky-image-sdk Pending Kevin Tian
syslog10 Qemux86 poky-image-sdk Pending Kevin Tian
utimensat01 Qemux86 poky-image-sdk Pending Kevin Tian
mtest06_2 Qemux86 poky-image-sdk Pending Kevin Tian
mmapstress06 Qemux86 poky-image-sdk Pending Kevin Tian
su01 Qemux86 poky-image-sdk Pending Kevin Tian
cron02 Qemux86 poky-image-sdk Pending Kevin Tian
cron_deny01 Qemux86 poky-image-sdk Pending Kevin Tian
cron_allow01 Qemux86 poky-image-sdk Pending Kevin Tian
cron_dirs_checks01 Qemux86 poky-image-sdk Pending Kevin Tian
at_deny01 Qemux86 poky-image-sdk Pending Kevin Tian
at_allow01 Qemux86 poky-image-sdk Pending Kevin Tian
clock_gettime03 Qemux86 poky-image-sdk Pending Kevin Tian get unexpected error number
timer_create04 Qemux86 poky-image-sdk Pending Kevin Tian
ld Qemux86 poky-image-sdk Pending Kevin Tian
ldd Qemux86 poky-image-sdk Not A Bug Kevin Tian No 'file' and 'ldd' in sdk image
file Qemux86 poky-image-sdk Pending Kevin Tian
tar Qemux86 poky-image-sdk Not A Bug Kevin Tian 'file' is not available in sdk image
cron Qemux86 poky-image-sdk Pending Kevin Tian
logrotate Qemux86 poky-image-sdk Pending Kevin Tian
cpio Qemux86 poky-image-sdk Pending Kevin Tian
unzip01 Qemux86 poky-image-sdk Pending Kevin Tian
gzip01 Qemux86 poky-image-sdk Pending Kevin Tian

Previous milestone

a placeholder. Once we reach a new milestone, the data under "current milestone" will be moved here.

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=LTP_result&oldid=356"