POSIX-results: Difference between revisions

From Yocto Project
Jump to navigationJump to search
Line 58: Line 58:
'''POSIX''': that test case is in development, that causes the test fails
'''POSIX''': that test case is in development, that causes the test fails


===Current Milestone (1.8 M3 RC3 20150303 build)===
===Current Milestone (1.8 M4 RC2 20150403 build)===
'''Yocto 1.8 M3 RC3 20150303 build''' with POSIX test suite version '''20150119'''.
'''Yocto 1.8 M4 RC2 20150403 build''' with POSIX test suite version '''20150119'''.


====AIO====
====AIO====

Revision as of 08:18, 9 April 2015

Introduction

This page records the POSIX test results of Yocto for current version. History POSIX test results have been moved to https://wiki.yoctoproject.org/wiki/POSIX_History_Results. You can get more informations from POSIX test overall on Yocto.

The online POSIX specification could be viewed at IEEE Std 1003.1, 2004 Edition.

The POSIX test suite is from ltp and current version is 20140422. All POSIX test cases can be get at POSIX git repository.

Test POSIX on following platforms:

  • beagleboard
  • mpc8315rdb
  • routerstation
  • sugarbay
  • blacksand
  • huronriver

There are 1794 test cases in total, only failed items are list here.

Result Codes

These informations are from POSIX test suit Documentation/HOWTO_ResultCodes.

The PTS result codes are:

#define PTS_PASS        0
#define PTS_FAIL        1
#define PTS_UNRESOLVED  2
#define PTS_UNSUPPORTED 4
#define PTS_UNTESTED    5

A brief discussion of each of these follows:

PTP_PASS        Used when the test executes fully with no problems and passes.
PTP_FAIL        Used when the test executes fully, but fails.
PTP_UNRESOLVED  Used when the test was blocked from fully completing and
                the pass/failure results cannot be determined.
PTP_UNSUPPORTED Used if the test is for a conditional feature that is
                not implemented.
PTP_UNTESTED    Used when a feature does not have a test associated
                with it because:
                - The test is just a stub and doesn't do anything
                - The test is only partially complete and can't really
                  finish the test
                - The test is complete in some cases, but certain things
                  can happen that leave the test incomplete.  When these
                  happen, it's PTP_UNTESTED.

For every failed case which returns PTP_FAIL there is a item in https://bugzilla.yoctoproject.org to follow. For other cases which don't return PTP_PASS find the root cause.

Results and analysis

The results of groups SIG SEM THR TMR MSG TPS MEM are listed. Conclusion of some test cases have been given.

Values of clumn "status" descript the current status of the case. Some keywords include:

Bug: it is a bug and the test case needs to be fixed.

Ignore: the case is not appropriate for the hardware or linux

POSIX: that test case is in development, that causes the test fails

Current Milestone (1.8 M4 RC2 20150403 build)

Yocto 1.8 M4 RC2 20150403 build with POSIX test suite version 20150119.

AIO

POSIX cases status platforms error msg analysis bugzilla
aio_cancel_7-1 mpc8315e conformance/interfaces/aio_cancel/aio_cancel_7-1: execution: UNRESOLVED
aio_error/3-1 Not A Bug genericx86-64 mpc8315e edgerouter beaglebone conformance/interfaces/aio_error/3-1: execution: UNTESTED

This test tries to call aio_error with argument aiocbp which does not point at a control block for an asynchronous I/O request, and expect error EINVAL.

The POSIX spec says about aio_error: The aio_error( ) function may fail if: [EINVAL] The aiocbp argument does not refer to an asynchronous operation whose return status has not yet been retrieved.

Yocto 2666

aio_read/9-1 Not A Bug genericx86-64 mpc8315e edgerouter beaglebone conformance/interfaces/aio_read/9-1: execution: UNSUPPORTED

ltp commit 91d8c0c85c97366a7ec4ee08ef3b3e7a38661ac8 will check the limit of aio queue, if there is no limit, the case is not supported and return UNSUPPORTED. Test case try to overflow the aio queue, but not quit sure the critical value on different OS.

aio_return/2-1 Not A Bug genericx86-64 mpc8315e edgerouter beaglebone conformance/interfaces/aio_return/2-1: execution: UNTESTED

In specification, second call to aio_return() return -1 is a kind of possibility, depends on implementation. In eglibc the return value is same as first return.

Yocto 2665
aio_return/3-2 Not A Bug genericx86-64 mpc8315e edgerouter beaglebone conformance/interfaces/aio_return/3-2: execution: UNTESTED

Same with aio_return/2-1

Yocto 2665
aio_return/4-1 Not A Bug genericx86-64 mpc8315e edgerouter beaglebone conformance/interfaces/aio_return/4-1: execution: UNTESTED

call aio_return with with argument aiocbp which does not point at a control block for an asynchronous I/O request, and expect error EINVAL. But this is optional.

Yocto 2665

aio_suspend/5-1 POSIX genericx86-64 mpc8315e edgerouter beaglebone conformance/interfaces/aio_suspend/5-1: execution: UNSUPPORTED

Stub Case Test case is a stub

aio_write/7-1 Not A Bug genericx86-64 mpc8315e edgerouter beaglebone conformance/interfaces/aio_write/7-1: execution: UNSUPPORTED

ltp commit b56638a7a8ae9cd37e5698de3186dc3cb1434cd8 will check the limit of aio queue, if there is no limit, the case is not supported and return UNSUPPORTED. Test case try to overflow the aio queue, but const value 1024 is not enough

SIG

POSIX cases status platforms error msg analysis bugzilla
n/a

SEM

POSIX cases status platforms error msg analysis bugzilla
sem_init/7-1 Unsupported mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sem_init/sem_init_7-1: execution: UNTESTED
 [17:07:31]sysconf(_SC_SEM_NSEMS_MAX) = -1
 [17:07:31]File ../../../conformance/interfaces
 /sem_init/7-1.c cannot test: There is no 
 constraint on SEM_NSEMS_MAX

Then return UNTESTED

In eglibc, sysconf() is implemented in sysdeps/posix/sysconf.c. When pass _SC_SEM_NSEMS_MAX to sysconf(), it checks whether SEM_NSEMS_MAX is defined, then if defined return SEM_NSEMS_MAX, otherwise return -1.

No SEM_NSEMS_MAX defined in Yocto same as in Ubuntu and Fedora, so the case fails.

It return UNTESTED, we don't need care it.

THR

POSIX cases status platforms error msg analysis bugzilla
pthread_mutexattr_gettype_1-2 Not A Bug genericx86-64 mpc8315e edgerouter beaglebone conformance/interfaces/pthread_mutexattr_gettype/pthread_mutexattr_gettype_1-2: execution: FAILED

Not a bug. Test result depends on system implement. Test case try to initialize a pthread_mutexattr_t object with pthread_mutexattr_init() and call pthread_mutexattr_gettype() with an invalid 'attr'.

pthread_mutexattr_gettype_1-3 Not A Bug genericx86-64 mpc8315e edgerouter beaglebone conformance/interfaces/pthread_mutexattr_gettype/pthread_mutexattr_gettype_1-3: execution: FAILED

Not a bug. Test result depends on system implement. Test case try to initialize a pthread_mutexattr_t object with pthread_mutexattr_init() and call pthread_mutexattr_gettype() with an invalid 'attr'.

pthread_mutexattr_settype_1-1 Not A Bug genericx86-64 mpc8315e edgerouter beaglebone conformance/interfaces/pthread_mutexattr_settype/pthread_mutexattr_settype_1-1: execution: FAILED

Not a bug. Test result depends on system implement. Test case try to initialize a pthread_mutexattr_t object with pthread_mutexattr_init() and call pthread_mutexattr_gettype() with an invalid 'attr'.

pthread_cond_broadcast/1-2 Limitted to memory mpc8315e edgerouter conformance/interfaces/pthread_cond_broadcast/1-2: execution: UNTESTED

The case create 1000 process, and check the system memory. If system available memory size less than it requires, the case exit with UNTESTED. When run single case on routerstation, it passed. But fails on mpc8315e to run single case even just after reboot.

pthread_create/1-6 POSIX mpc8315e edgerouter conformance/interfaces/pthread_create/pthread_create_1-6: execution: HUNG

Testcase create NCPU=4 threads running with a high priority with the same sched policy policy and one with a low-priority. The low-priority thread should not run until the other threads stop running, unless the machine has more than NCPU processors. Redefine NCPU = sysconf(_SC_NPROCESSORS_CONF), testcase should pass。

pthread_rwlock_rdlock/2-1 Unsupport by eglibc mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/pthread_rwlock_rdlock/pthread_rwlock_rdlock_2-1: execution: FAILED

Main thread read lock 'rwlock' with high priority. Child thread1 write lock 'wrlock' with medium priority will block. child thread2 read lock 'rwlock' with low priority should block because a high priority write lock is waiting, but read lock doesn't block. This test case is to test option Thread Execution Scheduling of pthread rwlock. But eglibc/libc don't support this option. They provide a rwlock->__data.__flags option to identify perfer 'read lock' or write lock, and the default is 'read lock' first. So second reader can get the lock when a higher priority writer is waiting for the lock. A Non-portable api pthread_rwlockattr_setkind_np() is provided to set this value. The test case is not supported.

pthread_rwlock_rdlock/2-2 Unsupport by eglibc mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/pthread_rwlock_rdlock/pthread_rwlock_rdlock_2-2: execution: FAILED

Main thread read lock 'rwlock' with high priority. Child thread1 write lock 'wrlock' with low priority will block. child thread2 read lock 'rwlock' with same low priority should block because a same priority write lock is waiting, but read lock doesn't block. See pthread_rwlock_rdlock/2-1

pthread_rwlock_unlock_4-1 genericx86-64 cconformance/interfaces/pthread_rwlock_unlock/pthread_rwlock_unlock_4-1: execution: SIGNALED
pthread_rwlock_unlock_4-2 genericx86-64 cconformance/interfaces/pthread_rwlock_unlock/pthread_rwlock_unlock_4-2: execution: SIGNALED
pthread_rwlock_unlock_3-1 Unsupport by eglibc mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/pthread_rwlock_unlock/pthread_rwlock_unlock_3-1: execution: FAILED

Main thread write lock 'rwlock' with high priority. Child thread1 write lock 'wrlock' with medium priority will block. Child thread2 read lock 'wrlock' with same mediumpriority. Child thread3 write lock 'wrlock' with low priority. When main thread unlock 'rwlock', child thread1 get the 'wrlock'. When child thread1 unlock 'rwlock', child thread2 should get the read lock 'wrlock' because it has higher priority than child thread3. But it is the child thread3 who get the 'wrlock'. See pthread_rwlock_rdlock/2-1

TMR

POSIX cases status platforms error msg analysis bugzilla
clock_getcpuclockid/5-1 Implement related mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/clock_getcpuclockid/clock_getcpuclockid_5-1: execution: UNTESTED
 clock_getcpuclockid(1, ..) passed unexpectedly 
Case tests permission. When call clock_getcpuclockid() as a unpriviledge user to get process 'init' cpu time, it succeeds but expect fails with EPERM.

This is not supported by Linux. In man page, it addresses EPERM "Specified as an optional error in POSIX.1-2001; Does not occur on Linux unless the kernel does not support obtaining the per-process CPU-time clock of another process.".

MSG

POSIX cases status platforms error msg analysis bugzilla
n/a

TPS

POSIX cases status platforms error msg analysis bugzilla
sched_get_priority_max/1-3 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_get_priority_max/sched_get_priority_max_1-3: execution: UNSUPPORTED Schedule policy SCHED_SPORADIC is not supported by linux
sched_get_priority_min/1-3 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_get_priority_min/sched_get_priority_min_1-3: execution: UNSUPPORTED Schedule policy SCHED_SPORADIC is not supported by linux
sched_getparam/6-1 Not Supported mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_getparam/6-1: execution: UNTESTED

Testing with user deamon (euid, uid) = (1, 1) The function sched_getparam has successed.

This case try to call sched_getparam as non-root user to get scheduling parameters of process with pid 0. It is a system call and implemented in kernel source sched/core.c. In SYSCALL_DEFINE2(sched_getparam, ...) it calls security_task_getscheduler() to check the permission. For default security operations, security_task_getscheduler() is a stub function and just return 0. In the manual of sched_getparam about return value: EPERM The calling process does not have appropriate privileges (Linux: does not have the CAP_SYS_NICE capability).

sched_getscheduler/7-1 Not Supportted mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_getscheduler/7-1: execution: UNTESTED

Testing with user deamon (uid: 1) The function sched_getscheduler has successed.

Similiar with sched_getparam/6-1
sched_setparam/20-1 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setparam/sched_setparam_20-1: execution: UNSUPPORTED PTHREAD_SCOPE_PROCESS for pthread_attr_setscope() is not supported by linux
sched_setparam/21-1 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setparam/sched_setparam_21-1: execution: UNSUPPORTED PTHREAD_SCOPE_PROCESS for pthread_attr_setscope() is not supported by linux
sched_setparam/21-2 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setparam/sched_setparam_21-2: execution: UNSUPPORTED PTHREAD_SCOPE_PROCESS for pthread_attr_setscope() is not supported by linux
sched_setparam/23-2 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setparam/sched_setparam_23-2: execution: UNSUPPORTED PTHREAD_SCOPE_PROCESS for pthread_attr_setscope() is not supported by linux
sched_setparam/23-3 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setparam/sched_setparam_23-3: execution: UNSUPPORTED PTHREAD_SCOPE_PROCESS for pthread_attr_setscope() is not supported by linux
sched_setparam/23-4 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setparam/sched_setparam_23-4: execution: UNSUPPORTED PTHREAD_SCOPE_PROCESS for pthread_attr_setscope() is not supported by linux
sched_setparam/23-5 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setparam/sched_setparam_23-5: execution: UNSUPPORTED PTHREAD_SCOPE_PROCESS for pthread_attr_setscope() is not supported by linux
sched_setparam/25-2 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setparam/sched_setparam_25-2: execution: UNSUPPORTED PTHREAD_SCOPE_PROCESS for pthread_attr_setscope() is not supported by linux
sched_setparam/25-3 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setparam/sched_setparam_25-3: execution: UNSUPPORTED PTHREAD_SCOPE_PROCESS for pthread_attr_setscope() is not supported by linux
sched_setparam/25-4 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setparam/sched_setparam_25-4: execution: UNSUPPORTED PTHREAD_SCOPE_PROCESS for pthread_attr_setscope() is not supported by linux
sched_setscheduler/17-2 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setscheduler/sched_setscheduler_17-2: execution: UNSUPPORTED Schedule policy SCHED_SPORADIC is not supported by linux
sched_setscheduler/17-3 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setscheduler/sched_setscheduler_17-3: execution: UNSUPPORTED Schedule policy SCHED_SPORADIC is not supported by linux
sched_setscheduler/17-4 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setscheduler/sched_setscheduler_17-4: execution: UNSUPPORTED Schedule policy SCHED_SPORADIC is not supported by linux
sched_setscheduler/15-1 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setscheduler/sched_setscheduler_15-1: execution: UNSUPPORTED Schedule policy SCHED_SPORADIC is not supported by linux
sched_setscheduler/15-2 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setscheduler/sched_setscheduler_15-2: execution: UNSUPPORTED Schedule policy SCHED_SPORADIC is not supported by linux
sched_setscheduler/19-2 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setscheduler/sched_setscheduler_19-2: execution: UNSUPPORTED Schedule policy SCHED_SPORADIC is not supported by linux
sched_setscheduler/19-3 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setscheduler/sched_setscheduler_19-3: execution: UNSUPPORTED Schedule policy SCHED_SPORADIC is not supported by linux
sched_setscheduler/19-4 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setscheduler/sched_setscheduler_19-4: execution: UNSUPPORTED Schedule policy SCHED_SPORADIC is not supported by linux
sched_setscheduler/22-1 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setscheduler/sched_setscheduler_22-1: execution: UNSUPPORTED Schedule policy SCHED_SPORADIC is not supported by linux
sched_setscheduler/22-2 Ignore mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/sched_setscheduler/sched_setscheduler_22-2: execution: UNSUPPORTED Schedule policy SCHED_SPORADIC is not supported by linux

MEM

POSIX cases status platforms error msg analysis bugzilla
mmap/31-1 Not Supported genericx86-64 edgerouter conformance/interfaces/mmap/mmap_31-1: execution: UNSUPPORTED

This case is NOT supported by Linux. Test parameters length and offset for mmap() are too large. In function get_unmapped_area(), it checks that length should less than TASK_SIZE, otherwise it returns -ENOMEM. TASK_SIZE value differs on different platforms.

mmap/11-4 Not Supported mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/mmap/mmap_11-4: execution: FAILED
 pa: 0x400fb000 
 pa_2: 0x40115000 
 Modification of the partial page
 at the end of an object is written out
This case maps a file with length of 1/2 * page_size, then write char 'b' to next byte of 1/2 * page_size. Then try to re-mmap() the file and expect the byte next to of 1/2 * page_size should not be character 'b'.

Test on all platforms are failed. The root cause is that after first mmap(), the mapped page for the file is cached in page cache. When do the 2nd or 3rd mmap() still get the same mapped page. The mapped page is cached. This case is not supported by Linux. mmap() is a system call and implemented by mmap2(). Then the call sequence is mmap2() ==> sys_mmap_pgoff() ==> do_mmap_pgoff() ==> mmap_region() ==> file->f_op->mmap(). In ext3 file system, file->f_op->mmap() point to generic_file_mmap(). In generic_file_mmap(), assign the Page Fault Exception Handler to filemap_fault(). When access the Virtual Memory Area, page fault will occur and the run filemap_fault(). In filemap_fault(), kernel run find_get_page() to search the physical page in the page cache.

shm_open_39-1 LTP mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/shm_open/shm_open_39-1: execution: UNTESTED Glibc 2.19 restricts shm_open and shm_unlink to SHMDIR: http://sourceware.org/git/?p=glibc.git;a=commit;h=5d30d853295a5fe04cad22fdf649c5e0da6ded8c Then if the file name is too long, shm_open will return EINVAL, not ENAMETOOLONG
shm_open_39-2 LTP mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/shm_open/shm_open_39-2: execution: UNTESTED Glibc 2.19 restricts shm_open and shm_unlink to SHMDIR: http://sourceware.org/git/?p=glibc.git;a=commit;h=5d30d853295a5fe04cad22fdf649c5e0da6ded8c Then if the file name is too long, shm_open will return EINVAL, not ENAMETOOLONG
shm_unlink_10-1 LTP mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/shm_unlink/shm_unlink_10-1: execution: UNTESTED Glibc 2.19 restricts shm_open and shm_unlink to SHMDIR: http://sourceware.org/git/?p=glibc.git;a=commit;h=5d30d853295a5fe04cad22fdf649c5e0da6ded8c Then if the file name is too long, shm_unlink will return ENOENT, not ENAMETOOLONG
shm_unlink_10-2 LTP mpc8315e edgerouter genericx86-64 beaglebone conformance/interfaces/shm_unlink/shm_unlink_10-2: execution: UNTESTED Glibc 2.19 restricts shm_open and shm_unlink to SHMDIR: http://sourceware.org/git/?p=glibc.git;a=commit;h=5d30d853295a5fe04cad22fdf649c5e0da6ded8c Then if the file name is too long, shm_unlink will return ENOENT, not ENAMETOOLONG