LSB Result

From Yocto Project
Jump to navigationJump to search

Introduction

The Linux Standard Base(LSB) was created to lower the overall costs of supporting the Linux platform. By reducing the differences between individual Linux distributions, the LSB greatly reduces the costs involved with porting applications to different distributions, as well as lowers the cost and effort involved in after-market support of those applications. (http://www.linuxfoundation.org/collaborate/workgroups/lsb)

LSB Test Suite support the following architecture.

  • i486
  • ia64
  • ppc32
  • ppc64
  • s390
  • s390x
  • x86_64

So LSB don't support arm and mips at present time and it will add test suite to support arm and mips possibly in the future.

Test Environment

  • Image: core-image-lsb-qt3
  • Platform: x86-32(emenlow), x86-64(jasperforest), ppc(mpc8315e-rdb)

result

Below table shows the result of LSB Test on different platforms.

Test Version X86-32(emenlow) X86-64(jasperforest) ppc(mpc8315e-rdb)
Command Check v 4.1.0-1 Success Success
Library Check v 4.1.0-1 Warning Warning
Alsa-T2C Tests v 4.1.0-1 Success Success
Core Tests v 4.1.0-1 Failures: 12 Failures: 66
Core-T2C Tests v 4.1.0-1 Success Success
Cpp-T2C Tests v 4.1.0-1 Success Success
Desktop Tests v 4.1.0-2 Failures: 2 Failures: 24
Desktop-T2C Tests v 4.1.0-1 Failures: 2 Failures: 17
Libstdc++ Tests v 4.1.0-13 Success Failures: 2
OLVER Core Tests v 4.1.0-1 Failures: 10 Block
Perl Tests v 4.1.0-1 Failures: 1 Failures: 1
Printing Tests v 4.1.0-1 Failures: 1 Failures: 3
Python Runtime Tests v 4.1.1-2 Success Success
Qt3-Azov Tests v 4.1.0-1 Success Failures: 106
Qt4-Azov Tests v 4.1.0-1 Failures: 5 Failures: 46
Xml2-Azov Tests v 4.1.0-1 Success Success
Xts5 Tests v 5.1.5-38 Failures: 1 Failures: 2
Apache Tests v 2.2.14-3 Success
Expect Tests v 5.43.0-11 Success Success
Groff Tests v 1.20.1-5 Success Success
Raptor Tests v 1.4.19-3 Success Success
Rsync Tests v 3.0.6-3 Success Success
Samba Tests v 3.4.3-5 Success Success
Tcl Tests v 8.5.7-6 Success Success
Xpdf Tests v 1.01-10 Success Failures: 1

Analysis of result

Below table shows the failed test cases.

  • Total failures: The number of total failures after running test. It includes known problems and real failures which should be fixed.
  • Known problem: Known problem means it is a bug of LSB itself and we can find trace these bugs on http://bugs.linuxbase.org. On link page we give a summary of known problems.
  • Failures(need to fix) : Real failures we should fix them

x86-32 (emenlw)

We have 4 failures to fix.

  • Core tests: 2
  • Xts5 Tests:1
Test Total failures Known problem Failures(need to fix) Note
Core Tests 12 10 2
Desktop Tests 2 2
Desktop-T2C Tests 2 2
OLVER Core Tests 10 10
Perl Tests 1 1
Printing Tests 1 1
Qt4-Azov Tests 5 5
Xts5 Tests 1 1

ppc(mpc8315e)

We have the following failures(including "unresolved" and "failed") to fix.

Test Total failures Known problem Failures(need to fix) Note
Core Tests 66 10 47(unresolved), 8(failed), 1(unreported)
Desktop Tests 24 20 2(unresolved), 1(failed), 1 unreported
Desktop-T2C Tests 17 2 15(unresolved)
Libstdc++ Tests 2 2
OLVER Core Tests Blocked
Perl Tests 1 1
Printing Tests 3 1 1(unresolved), 1(failure), 1(unreported)
Qt3-Azov 106 106(unresolved)
Qt4-Azov 46 1 45(unresolved)
Xts5 Tests 2 2
Xpdf Tests 1 1
  • Core Tests:
    • /tset/ANSI.os/math/... : SIGILL cause 47 unresolved failures.
  It could be caused by the difference of compilation tool between LSB Test Suite and mpc8315e. 
  -Error don't happen when I use LSB Test Suite with version 4.0.
  -I extract test case source codes from lsb test suite and run this code after compiling with compilation tool of mpc8315e,error of illegal instruction don't happen.
  So I think LSB official compilation tool could have some instructions which don't exist on compilation tool of mpc8315e.
  • OLVER Core Tests:
    • OLVER Core Tests can't be finished on mpc8315e-rdb
    • We think the reason which make test blocked is the low memory(128M on mpc8315e). Because we re-run the test case which is blocked during OLVER core tests and it can pass successfully. That is to say, if we run the single test case, it will be passed. But we run all OLVER core tests and the whole tests will be blocked. We have ever run OLVER Core Tests for nearly 50 hours and it can't finish.
    • lsb-test-olver-core may require about 512Mb of system memory (with an X server and a contemporary desktop running), as well as 2Gb of free disk space to complete a test run. The reference linkage: http://ldn.linuxfoundation.org/lsb/preparing-lsb-distribution-testing
  • Qt3-Azov Tests:
    • Out of memory error cause all of the failures, the following information arise on screen when running Qt3 test suite
   Killed process 22453 (Xvfb) total-vm:84064kB, anon-rss:73720kB, file-rss:8kB
   run-parts invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0,
   oom_score_adj=0
   Call Trace:
   [c0cadc90] [c0009f3c] show_stack+0x108/0x1c4 (unreliable)
   [c0cadce0] [c0431098] dump_stack+0x2c/0x44
   [c0cadcf0] [c00a1880] dump_header.clone.14+0x70/0x184
   [c0cadd30] [c00a1b80] oom_kill_process.clone.16+0x94/0x294
   [c0cadd60] [c00a222c] out_of_memory+0x270/0x33c
   [c0caddb0] [c00a6628] __alloc_pages_nodemask+0x624/0x65c
   [c0cade50] [c00a6694] __get_free_pages+0x34/0x6c
   [c0cade60] [c00269f8] copy_process+0xe4/0xd84   
   [c0cadec0] [c0027764] do_fork+0x80/0x39c
   [c0cadf20] [c0009cb4] sys_vfork+0x68/0x80
   [c0cadf40] [c0011d0c] ret_from_syscall+0x0/0x38       
     --- Exception: c00 at 0xfecc358
     LR = 0x10084518
    • I think this should be limited memory problem
  • Xpdf Tests:
    • SIGILL cause Xpdf failed.

Failures tracker

We should fix the following problems. We also filed bugs for them and try to fix them.

Tests Sub Tests Test Number Arch Bug ID of yocto Proposal suggestion
Core Tests /tset/LSB.fhs/var/lib/lib-tc 1 x86-32, x86-64, ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=1092 http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=xiaofeng/sudo
Core Tests /tset/LSB.fhs/var/lib-misc/lib-misc-tc 1 x86-32, x86-64, ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=1092 http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=xiaofeng/sudo
Core Tests /tset/ANSI.os/maths/* 47 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=1114 illegal instruction error
Core Tests /tset/LSB.os/libm/fpeInvalid/T.fpeInvalid 2 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=1114 illegal instruction error
Core Tests /tset/LI18NUX2K.L1/base/iswcntrl/T.iswcntrl 1th 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=
Core Tests /tset/LI18NUX2K.L1/base/iswdigit/T.iswdigit 1th 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=
Core Tests /tset/LI18NUX2K.L1/base/iswlower/T.iswlower 1th 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=
Core Tests /tset/LI18NUX2K.L1/base/iswupper/T.iswupper 1th 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=
Core Tests /tset/LI18NUX2K.L1/base/iswxdigit/T.iswxdigit 1th 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=
Desktop /tests/functions/GHashTable/GHashTable 5th 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=
Desktop /tests/functions/Param_Value/Param_Value 20th (unresolved) 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=
Desktop /tests/functions/SpawnProc/SpawnProc 2 (unresolved) 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=
Desktop-t2c /glib-t2c/tests/glib_threads/glib_threads 36th 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id= signal 6 error
Desktop-t2c /glib-t2c/tests/glib_random/glib_random 9th-23th 14 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=1114 illegal instruction
Desktop-t2c /gobject-t2c/tests/gobject_gtype_instance/gobject_gtype_instance 39th 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=
Libstdc++ Tests /27_io/basic_stringbuf/overflow/wchar_t/1.cc 1th 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id= math function "abs"
Libstdc++ Tests /26_numerics/complex/13450.cc 1th 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=
Printing Tests /convenience/cupsConvenience 37th (unresolved) 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=
Xts5 Tests /tset/Xlib10/kllclnt/Test 2th 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=
Xts5 Tests /tset/Xopen/strtkysym/Test 7 x86-32, ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=1095 http://git.pokylinux.org/cgit.cgi/poky-contrib/commit/?h=jingdonglu/distro&id=c6dde065b9fafabeda6904c23118a11e99d23e08
Perl Tests ../ext/IPC/SysV/t/ipcsysv.t 294 x86-32, ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=1078 Maybe a LSB bug(we have filed it as a bug on LSB bugzilla
Xpdf Tests /opt/lsb/appbat/bin/pdftotext test1.pdf 1 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=1114 illegal instruction

Known problems tracker

See details on Known problems tracker

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=LSB_Result&oldid=1874"