Posix result: Difference between revisions

Revision as of 01:19, 1 November 2018

Introduction

The Open POSIX* Test Suite is an open source test suite with the goal of performing conformance, functional, stress, and performance testing of the functions described in the IEEE Std 1003.1-2001 System Interfaces specification. Eventual testing of the full specification is desired (http://posixtest.sourceforge.net/)

Similar to [LTP result], embedded linux distribution is highly customized for a specific purpose, and thus both the kernel and base environment may not satisfy the posix requirement. So the major purposes of this work are:

to understand the reasons for existing posix failures. If the cause is from a specific package itself, go to fix it. Or else if the error simply come from the fact that a given package doesn't exist in current profile, add the explanation here
to track poix test results cross Yocto releases, and ensure a declining error number for same profile
to be the reference if Yocto users want to get a base picture

There is one bugzilla entry related: http://bugzilla.pokylinux.org/show_bug.cgi?id=172

Overall trend

Overall trend tends to track the change of posix test results along with Yocto's progress.

In below table we list total POSIX failures for each architectures in each milestone: (now this wiki doesn't allow upload files. So for detail logs please refer to earlier bug link)

Milestone

qemux86

qemu-x86_64

qemuppc

qemuarm

qemumips

atom-pc

sugarbay

blacksand

beagleboard

routerstation

mpc8315e-rdb

huronriver

edgerouter

beaglebone

genericx86-64

2.6 RC1

n/a

48

n/a

50

47

2.6 M3 RC1

n/a

48

n/a

50

47

2.6 M2 RC1

n/a

48

n/a

51

48

2.6 M1 RC1

n/a

48

n/a

47

2.5 RC1

n/a

47

n/a

48

45

2.5 M3 RC1

n/a

45

n/a

47

44

2.5 M2 RC1

n/a

45

n/a

47

44

2.5 M1 RC3

n/a

45

n/a

47

45

47

2.5 M1 RC1

n/a

45

n/a

47

45

2.4 RC3

n/a

45

n/a

47

45

46

2.4 RC2

n/a

45

n/a

45

46

2.4 RC1

n/a

45

n/a

47

45

46

2.4 M2 RC3

n/a

45

n/a

46

45

46

2.4 M2 RC2

n/a

45

n/a

46

45

46

2.4 M1 RC1

n/a

45

n/a

46

45

2.3 RC2

n/a

46

n/a

46

45

2.3 M3 RC2

n/a

46

n/a

46

45

2.3 M2 RC3

n/a

47

n/a

47

46

2.3 M1 RC1

n/a

47

n/a

47

46

2.2 RC4

n/a

47

n/a

47

46

2.2 M3 RC1

n/a

47

n/a

47

46

2.2 M2 RC1

n/a

47

n/a

48

46

2.0 RC3

n/a

43

n/a

44

41

44

2.0 RC2

n/a

43

n/a

44

42

44

1.9 M3 RC1

n/a

43

n/a

44

41

44

1.9 M2 RC2

n/a

47

n/a

47

44

47

1.9 M1 RC2

n/a

46

n/a

47

44

47

1.8 M4 RC2 20150403

n/a

46

n/a

48

44

47

1.8 M3 RC3 20150303

n/a

47

n/a

47

44

47

1.7 M4 RC5 20141018

n/a

47

n/a

47

45

47

1.7 M4 RC4 20141011

n/a

47

n/a

49

45

47

1.7 M2 RC1 20140730

n/a

49

n/a

50

48

1.7 M1 RC1 20140625

n/a

50

n/a

52

48

1.5.2 RC3 20140428

n/a

41

44

43

n/a

41

1.6 M5 RC4 20140411

n/a

47

n/a

51

45

1.6 M5 RC2 20140402

n/a

47

n/a

51

45

1.6 M5 RC1 20140326

n/a

45

n/a

45

46

47

n/a

1.6 M4 RC1 20140312

n/a

45

n/a

46

47

n/a

1.6 M3 RC2 20140218

n/a

45

n/a

45

49

47

n/a

1.6 M3 RC1 20140213

n/a

41

43

42

n/a

1.6 M2 RC1 20140114

n/a

41

n/a

41

43

42

n/a

1.5.1 RC2 20131213

n/a

41

44

43

n/a

1.6 M1 RC1 20131203

n/a

41

n/a

42

n/a

1.5 M5 RC6

n/a

41

n/a

41

44

43

n/a

1.5 M4 RC3

n/a

41

n/a

45

43

n/a

1.5 M4 RC2

n/a

41

n/a

43

44

n/a

1.5 M2 RC1 20130703

n/a

31

n/a

41

n/a

1.4.1 RC1 20130612

n/a

122

n/a

124

126

124

n/a

1.5 M1 RC1 20130604

n/a

119

n/a

119

n/a

120

n/a

1.3.2 20130516

n/a

122

n/a

123

124

n/a

1.4 M6 RC2 20130418

n/a

122

n/a

123

126

124

n/a

1.4 M5 RC3 20130403

n/a

122

n/a

123

n/a

124

n/a

1.4 M5 RC2 20130327

n/a

122

n/a

123

124

n/a

1.4 M5 RC2 20130327

n/a

123

124

1.4 M4 RC1 20130220

n/a

124

126

124

122

1.4 M3 RC2 20130131

n/a

123

124

122

1.4 M3 RC1 20130122

n/a

123

124

122

1.2.2 20130105

n/a

128

130

127

1.2.2 20121206

n/a

127

129

126

128

1.4 M1 RC1 20121128

n/a

123

125

124

1.2.2 20121018

n/a

124

126

124

122

1.3 M5 RC4 20121010

n/a

123

124

122

1.3 M5 RC3 20121003

n/a

122

124

127

124

122

1.3 M5 RC2 20120926

n/a

122

125

124

122

1.3 M4 RC3 20120916

n/a

123

124

123

122

1.3 M3 RC2 20120814

n/a

124

122

123

1.3 M2 20120802

n/a

124

n/a

125

123

1.3 M2 RC1 20120712

n/a

126

n/a

126

128

n/a

127

1.1.2 20120630

n/a

128

n/a

162

167

126

n/a

1.2.1 20120629

n/a

128

125

129

126

130

1.2 denzil meta-intel

n/a

126

n/a

129

1.3 M1 RC1 20120613

n/a

125

126

129

126

127

1.3 20120606 build

n/a

126

125

126

n/a

Denzil 1.2 RC5

n/a

128

125

126

129

126

n/a

Denzil 1.2 RC4

n/a

128

126

128

127

n/a

1.2 M4 RC3

n/a

127

128

125

n/a

126

n/a

1.2 M4 RC2

n/a

127

125

127

130

126

n/a

1.2 M4 RC1

n/a

129

126

127

130

126

n/a

1.2 M3 RC1

n/a

130

127

129

132

126

n/a

1.1.1

n/a

10

9

31

33

32

n/a

1.2 M2 RC1(****)

n/a

130

128

131

132

128

n/a

1.2 M1 RC2(***)

n/a

90

92

n/a

98

91

92

89

n/a

1.0.2

n/a

35

32

33

n/a

30

71

34

n/a

1.1 M4 RC4

n/a

31

33

32

n/a

25

30

31

32

n/a

1.1 M4 RC3

n/a

32

35

33

n/a

24

23

37

33

31

n/a

1.1 M4 RC2

n/a

31

34

n/a

25

24

31

33

32

n/a

1.1 M3

n/a

32

34

n/a

23

24

30

32

31

n/a

1.1 M2

n/a

30

34

32

n/a

31

26

32

69

33

n/a

1.1 M1(**)

n/a

35

49

43

n/a

50

48

46

84

49

n/a

1.0-M2 (*)

1744

1745

1750

1753

n/a

1744

1755

1754

n/a

*: 1.0-M2 misses some link flags, and thus many unexpected link failures observed
**: 1.1 M1 has gcc upgrade from 4.5.1 to 4.6.1, which has two new options, -Wunused-but-set-variable and -Waddress enabled by default. They casused many cases failed. We need add -Wno-unused-but-set-variable and -Wno-address to fix the problems.
***: From 1.2, We using POSIX test suite in LTP test suite instead of POSIX test suite 1.5.2. 
****: From 1.2 M2, We also test AIO.

general tips:

use "-m 512" to launch the qemu targets
make sure below line is uncommented in LDFLAGS file:

#-D_XOPEN_SOURCE=600 -lpthread -lrt -lm

don't test AIO as the test cases themselves have compilation warnings which are treated as errors (use 'exit' while doesn't include stdlib.h)

Result analysis

Current results are base on Yocto 2.5 and POSIX test suite in LTP release 20170929. After analysis every test case except PASS will be list here, whether a patch will be gave to fix it or put the case to blacklist with a explanation.

POSIX test results: https://wiki.yoctoproject.org/wiki/POSIX-results

Test environment

OpenPOSIX version: posixtestsuite in LTP release 20180515

profile: core-image-sato-sdk

run command: cd /opt/ltp/testcases/open_posix_testsuite; make generate-makefiles; make conformance-all conformance-test; make tools-all; bin/run-posix-option-group-test.sh [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

Current Milestone (Yocto 2.6 RC1 build)

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures		47	50	48

Previous Milestone

Yocto 2.6 M3 RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures		47	50	48

Yocto 2.6 M2 RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures		48	51	48

Yocto 2.6 M1 RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures		47		48

Yocto 2.5 RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures		45	48	47

Yocto 2.5 M3 RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures		44	47	45

Yocto 2.5 M2 RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures		44	47	45

Yocto 2.5 M1 RC3 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	47	45	47	45

Yocto 2.5 M1 RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures		45	47	45

Yocto 2.4 RC3 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	46	45	47	45

Yocto 2.4 RC2 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	46	45		45

Yocto 2.4 RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	46	45	47	45

Yocto 2.4 M2 RC3 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	46	45	46	45

Yocto 2.4 M2 RC2 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	46	45	46	45

Yocto 2.4 M1 RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	45	45	46	45

Yocto 2.3 RC2 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	45	45	46	46

Yocto 2.3 M3 RC2 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	45	45	46	46

Yocto 2.3 M2 RC3 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	46	46	47	47

Yocto 2.3 M1 RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	46	46	47	47

Yocto 2.2 RC4 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	46	46	47	47

Yocto 2.2 M3 RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	46	46	47	47

Yocto 2.2 M2 RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	46	46	48	47

Yocto 2.0 RC2 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	44	41	44	43

Yocto 2.0 RC2 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	44	42	44	43

Yocto 1.9 M3 RC1 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	44	41	44	43

Yocto 1.9 M2 RC2 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	47	44	47	47

Yocto 1.9 M1 RC2 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	47	44	47	46

Yocto 1.8 M4 RC2 20150403 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	47	44	48	46

Yocto 1.8 M3 RC3 20150303 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	47	44	47	47

Yocto 1.7 M4 RC5 20141018 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	47	45	47	47

Yocto 1.7 M4 RC4 20141011 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	47	45	49	47

Yocto 1.7 M2 RC1 20140730 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	48	48	50	49

Yocto 1.7 M1 RC1 20140625 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	48	48	52	50

Yocto 1.6 M5 RC4 20140411 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	45		51	47

Yocto 1.6 M5 RC2 20140402 build

Status	genericx86-64	beaglebone	edgerouter	mpc8315e-rdb
failures	45		51	47

Yocto 1.6 M5 RC1 20140326 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	45	n/a	47	45	46	n/a

Yocto 1.6 M4 RC1 20140312 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	45	n/a	47	46	46	n/a

Yocto 1.6 M3 RC2 20140218 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	45	n/a	47	45	49	n/a

Yocto 1.6 M3 RC1 20140213 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures		n/a	42	41	43	n/a

Yocto 1.6 M2 RC1 20140114 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	41	n/a	42	41	43	n/a

Yocto 1.6 M1 RC1 20131203 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	41	n/a	42	n/a	42	n/a

Yocto 1.5.1 RC2 20131213 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures		n/a	43	41	44	n/a

Yocto 1.5 M5 RC6 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	41	n/a	43	41	44	n/a

Yocto 1.5 M4 RC3 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	41	n/a	43	n/a	45	n/a

Yocto 1.5 M4 RC2 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	41	n/a	44	n/a	43	n/a

Yocto 1.5 M2 RC1 20130703 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	31	n/a	41	n/a	n/a	n/a

Yocto 1.4.1 RC1 20130612 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	122	n/a	124	124	126	n/a

Yocto 1.5 M1 RC1 20130604 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	119	n/a	120	119	n/a	n/a

Yocto 1.3.2 20130516 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	122	n/a	124	123	124	n/a

Yocto 1.4 M6 RC2 20130418 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	122	n/a	124	123	126	n/a

Yocto 1.4 M5 RC3 20130403 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	122	n/a	124	123	n/a	n/a

Yocto 1.4 M5 RC2 20130327 build

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	n/a	n/a	124	123	124

Yocto 1.4 M4 RC1 20130220 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120903

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	n/a	n/a	124	124	126	122

Yocto 1.4 M3 RC2 20130131 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120903

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	n/a	n/a	124	123	124	122

Yocto 1.4 M3 RC1 20130122 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120903

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	n/a	n/a	124	123	124	122

Yocto 1.4 M1 RC1 20121128 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120903

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	n/a	n/a	124	123	125

Yocto 1.3 M5 RC4 20121010 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120903

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	n/a	123	123	124	124	122

Yocto 1.3 M5 RC3 20121003 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120903

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	n/a	122	124	127	124	122

Yocto 1.3 M5 RC2 20120926 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120614

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	n/a	122	124	125	125	122

Yocto 1.3 M4 RC3 20120916 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120614

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures		123	123	124	124	122

Yocto 1.3 M3 RC2 20120814 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120617

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures			123	124	122	123

Yocto 1.3 M2 20120802 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120617

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	124	n/a	123	125	123	123

Yocto 1.3 M2 RC1 20120712 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120401

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	126	n/a	n/a	126	128	127

Yocto 1.2.1 20120629 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120401

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	128	125	126	125	129	130

Yocto 1.2 Denzil meta-intel

environment OpenPOSIX version: posixtestsuite in LTP release 20120401

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

git info:

Tree/Branch: poky/denzil

Commit: 7552ccd06c6327b5f36ee0cceb432946f8e98d3f

Meta-intel branch: denzil

Commit: a79035a584cd618d7202ce006cfb2d714a0ce717

result

Status	sugarbay	blacksand	huronriver
failures	126	126	129

Yocto 1.3 M1 RC1 20120613 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120401

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation	huronriver
failures	125	125	126	126	129	127

Yocto 1.3 20120606 build

environment OpenPOSIX version: posixtestsuite in LTP release 20120401

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation
failures	126	125	126	125	126

Yocto 1.2 Denzil RC5

environment OpenPOSIX version: posixtestsuite in LTP release 20120401

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation
failures	128	125	126	126	129

Yocto 1.2 Denzil RC4

environment OpenPOSIX version: posixtestsuite in LTP release 20120401

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation
failures	128	126	127	126	128

Yocto 1.2 M4 RC3

environment OpenPOSIX version: posixtestsuite in LTP release 20120401

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation
failures	127	128	125	n/a	126

Yocto 1.2 M4 RC2

environment OpenPOSIX version: posixtestsuite in LTP release 20120401

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation
failures	127	125	126	127	130

Yocto 1.2 M4 RC1

environment OpenPOSIX version: posixtestsuite in LTP release 20120104

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation
failures	129	126	126	127	130

Yocto 1.2 M3 RC1

environment OpenPOSIX version: posixtestsuite in LTP release 20120104

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation
failures	130	127	126	129	132

Yocto 1.1.1

environment OpenPOSIX version: posixtestsuite-1.5.2.tar

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation
failures	10	9	32	31	33

Yocto 1.2 M2 RC1

environment OpenPOSIX version: posixtestsuite in LTP release 20110915

profile: core-image-sato-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	mpc8315e-rdb	beagleboard	routerstation
failures	128	130	128	131	132

Yocto 1.2 M1

environment OpenPOSIX version: posixtestsuite in LTP release 20110915

profile: core-image-sato-sdk

run command: ./run_tests [MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	qemuppc	qemuarm	qemumips	mpc8315e-rdb	beagleboard	routerstation
failures	98	98	90	92	92	89	91	92

Yocto 1.0.2

environment OpenPOSIX version: posixtestsuite-1.5.2.tar

profile: poky-image-sato-sdk

run command: ./run_tests [MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	qemuppc	qemuarm	qemumips	mpc8315e-rdb	beagleboard	routerstation
failures			35	32	33	34	30	71

Yocto 1.1 M4 RC4

environment OpenPOSIX version: posixtestsuite-1.5.2.tar

profile: core-image-sato-sdk

run command: ./run_tests [MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	qemuppc	qemuarm	qemumips	mpc8315e-rdb	beagleboard	routerstation
failures	25	25	31	33	32	32	30	31

Yocto 1.1 M4 RC3

environment OpenPOSIX version: posixtestsuite-1.5.2.tar

profile: core-image-sato-sdk

run command: ./run_tests [MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	qemuppc	qemuarm	qemumips	mpc8315e-rdb	beagleboard	routerstation
failures	24	23	32	35	33	31	37	33

Yocto 1.1 M4 RC2

environment OpenPOSIX version: posixtestsuite-1.5.2.tar

profile: core-image-sato-sdk

run command: ./run_tests [MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	qemuppc	qemuarm	qemumips	mpc8315e-rdb	beagleboard	routerstation
failures	25	24	31	34	34	32	31	33

Yocto 1.1 M3

environment OpenPOSIX version: posixtestsuite-1.5.2.tar

profile: core-image-sato-sdk

run command: ./run_tests [MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	qemuppc	qemuarm	qemumips	mpc8315e-rdb	beagleboard	routerstation
failures			32	34	34	31	30	32

Yocto 1.1 M2

environment OpenPOSIX version: posixtestsuite-1.5.2.tar

profile: core-image-sato-sdk

run command: ./run_tests [MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	qemuppc	qemuarm	qemumips	mpc8315e-rdb	beagleboard	routerstation
failures	31	26	30	34	32	33	32	69

Yocto 1.1 M1

environment OpenPOSIX version: posixtestsuite-1.5.2.tar

profile: core-image-sato-sdk

run command: ./run_tests [MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	sugarbay	blacksand	qemuppc	qemuarm	qemumips	mpc8315e-rdb	beagleboard	routerstation
failures	50	48	35	49	43	49	46	84

Yocto 1.0 M2

environment OpenPOSIX version: posixtestsuite-1.5.2.tar

profile: poky-image-sdk

run command: ./run_tests [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]

result

Status	qemux86	qemu-x86_64	qemuppc	qemuarm	qemumips	atom-pc	blacksand	beagleboard	routerstation
failures	1744	1744	1745	1750	1753	1753	1744	1755	1754

The major problem for high failure numbers here is due to one environmental issue. It's suggested to uncomment below line from LDFLAGS file:

#-D_XOPEN_SOURCE=600 -lpthread -lrt -lm

libpthread and librt are critical for posix interface test. Without doing this the majority of the failures are link failures being failed to find necessary libraries.

My quick test on qemux86 shows that the failure could be reduced a lot to <200 in a rough run

Based on above fact, no more analysis is done on 1.0-M2 posix results. The thorough analysis will be postponed to 1.0-M3 which will use the right test steps and newer 2.6.37 kernel

@@ Line 226: / Line 226: @@
 bin/run-posix-option-group-test.sh [AIO|MEM|MSG|SEM|SIG|THR|TMR|TPS]
-==Current Milestone (Yocto 2.6 M2 RC1 build)==
+==Current Milestone (Yocto 2.6 RC1 build)==
 {|border="1"
 || '''Status''' || '''genericx86-64''' || '''beaglebone''' || '''edgerouter''' || '''mpc8315e-rdb'''
 |-
-|| failures ||  || 48 || 51 || 48
+|| failures ||  || 47 || 50 || 48
 |}