LSB Result: Difference between revisions

From Yocto Project
Jump to navigationJump to search
Line 331: Line 331:
==Known problems tracker==
==Known problems tracker==
See details on [[LSB_Bug_tracker | Known problems tracker]]
See details on [[LSB_Bug_tracker | Known problems tracker]]
==LSB Bug tracker(update 2012-04-13)==
Below table is a summary of known problems during LSB tests. We can find these issues on http://bugs.linuxbase.org. Some bugs go beyond the capacity of LSB 4.1 because of the faster updating speed than LSB Test Suite for some packages.
{|border="1"
|| '''Test'''  || '''SubTest''' || '''TestCase''' || '''TestNumber''' || '''Arch''' || '''Need fix?''' || '''Notes''' || '''Upstream Bugtracker'''
|-
|| Core Tests ||  || /tset/POSIX.os/devclass/c_iflag/T.c_iflag || 6, 7, 8, 9 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3033
|-
|| Core Tests ||  || /tset/POSIX.os/devclass/tcgetattr || 1, 2 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3033
|-
|| Core Tests ||  || /tset/LSB.os/ioprim/readv_L/T.readv_L || 24 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=844 
|-
|| Core Tests ||  || /tset/LSB.os/ioprim/writev_L/T.writev_L || 30 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=844
|-
|| Core Tests ||  ||/tset/ANSI.os/maths/*  || 47 || ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3253
|-
|| Core Tests ||  ||/tset/LSB.os/libm/fpeInvalid/T.fpeInvalid  || 2 || ppc || N || Known Problem ||http://bugs.linuxbase.org/show_bug.cgi?id=3253
|-
|| Core Tests ||  ||/tset/POSIX.os/files/fpathconf/T.fpathconf  || 15,16 || x86,x86-64,ppc || N || Known Problem ||http://bugs.linuxbase.org/show_bug.cgi?id=3078
|-
|| Core Tests || ||/tset/LI18NUX2K.L1/base/iswcntrl/T.iswcntrl 1st  || 1 || ppc || N || Upstream ||Compilation tool of LSB cause this problem, Compile test suite's source codes with poky compilation tool and no this problem. It can pass test with private test suite from Linux Foundation which will be merged to new version. http://dev.linuxfoundation.org/~stewb/ppc32/lsb-test-core-4.1.9-1.1.ppc.rpm 
|-
|| Core Tests || ||/tset/LI18NUX2K.L1/base/iswdigit/T.iswdigit 1st  || 1 || ppc || N || Upstream ||the same as above
|-
|| Core Tests || ||/tset/LI18NUX2K.L1/base/iswlower/T.iswlower 1st  || 1 || ppc || N || Upstream ||the same as above
|-
|| Core Tests || ||/tset/LI18NUX2K.L1/base/iswupper/T.iswupper  1st  || 1 || ppc || N || Upstream ||the same as above
|-
|| Core Tests || ||/tset/LI18NUX2K.L1/base/iswxdigit/T.iswxdigit 1st  || 1 || ppc || N || Upstream ||the same as above
|-
|| Core test || ||/tset/POSIX.os/procprim/fork/T.fork ||5|| x86-64 || N||Known Problem ||http://bugs.linuxbase.org/show_bug.cgi?id=2270(fixed after 4.1)
|-
|| Core test || ||/tset/LSB.fhs/var/lib-hwclock/lib-hwclock-tc ||1|| x86,x86-64,ppc|| N||Known Problem ||https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3116(Upstream, Don't need fix)
|-
|| Core test || ||/tset/LSB.os/network/gethostbyaddr/T.gethostbyaddr ||2|| x86,x86-64,ppc|| N||Known Problem ||Absent startup program avahi-daemon(Fixed).
|-
|| Desktop Tests|| desktop-gtkvts || /tests/functions/Param_Value/Param_Value || 20 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3044
|-
|| Desktop Tests|| desktop-gtkvts || /tests/functions/Selection/Selections || 33, 34 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3011
|-
|| Desktop Tests|| desktop-gtkvts|| /tests/functions/Keyboard_Accelerator_Groups/Keyboard_Accelerator_Groups || 27-29 || x86,x86-64,ppc|| N ||Known Problem||Function "gtk_accelerator_name" from gtk/gtkaccelgroup.c has been changed since version 2.24.8. So this is incorrect test case now.(Upstream, Don't need fix)
|-
|| Desktop Tests|| desktop-gtkvts || /tests/functions/Themeable_Stock_Images/Themeable_Stock_Images || 195-210 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3011 
|-
|| Desktop Tests|| desktop-gtkvts||/tests/functions/GtkTreeSortable/GtkTreeSortable ||15-18 || x86,x86-64,ppc || N || Upstream ||https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3238
|-
|| Desktop Tests|| || /tests/functions/SpawnProc/SpawnProc 2nd (unresolved) || 2 || ppc || N ||Known Problem ||http://bugzilla.pokylinux.org/show_bug.cgi?id=1117
|-
|| Desktop Tests || || /tests/functions/GHashTable/GHashTable|| 5 ||x86-32,x86-64,ppc|| N || snapshot pas
|-
|| Desktop-T2c || ||/atk-t2c/tests/AtkObject_relation/AtkObject_relation || 2,5,6 || x86,x86-64,ppc || N || Upstream ||https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3189
|-
|| Desktop-T2c|| ||/gobject-t2c/tests/gobject_value_arrays/gobject_value_arrays || 24 || x86,x86-64,ppc || N ||Upstream ||codes of qsort_r has changed since 2.30.2 So this is upstream problem. The higher version of LSB could modify this problem.
|-
|| Desktop-T2c || ||/gobject-t2c/tests/gobject_gtype_instance/gobject_gtype_instance || 39 || x86.x86-64,ppc || N || Upstream  ||https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3184
|-
|| Desktop-T2C Tests ||  || /freetype-t2c/tests/truetype_tables/truetype_tables || 2 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3153
|-
|| Desktop-t2c ||  ||/glib-t2c/tests/glib_threads/glib_threads  || 36 || ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3042
|-
|| Desktop-T2C ||  || /gobject-t2c/tests/gobject_gtype_instance/gobject_gtype_instance || 39 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3184 
|-
|| Desktop-t2c ||  ||/glib-t2c/tests/glib_random/glib_random 9th-23th || 14 || ppc || N || Known Problem ||http://bugs.linuxbase.org/show_bug.cgi?id=3253
|-
|| OLVER Core Tests ||  || /olver/math/cexp/tests/cpow_spec || 5162 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=2912
|-
|| OLVER Core Tests || locale_locale_scenario || /olver/other/other/tests/others || 1 || x86-32, x86-64 || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=2871
|-
|| OLVER Core Tests || ncurses_string_insert_scenario || /olver/other/other/tests/others || 2 || x86-32, x86-64 || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=2871 
|-
|| OLVER Core Tests || ncurses_string_add_scenario || /olver/other/other/tests/others || 3 || x86-32, x86-64 || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=2871 
|-
|| OLVER Core Tests ||  || /olver/other/other/tests/others || 561 || x86-32, x86-64 || N || Known Problem || http://linuxtesting.org/results/report?num=S0561 
|-
|| OLVER Core Tests || ncurses_line_rip_scenario || /olver/other/other/tests/others || 5 || x86-32, x86-64 || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=2871
|-
|| OLVER Core Tests ||  || /olver/util/compress/tests/gzputc_spec || 1 || x86-32, x86-64 || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3057 
|-
|| OLVER Core Tests ||  || /olver/util/format/tests/getdata_spec || 803 || x86-32, x86-64 || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=2913
|-
|| OLVER Core Tests ||  || /olver/util/format/tests/strptime_spec || 810, 808 || x86-32, x86-64 || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=2914 
|-
|| Qt3-Azov Tests  || ||/Qt3_Object_Model-t2c/tests/QMetaObject/QMetaObject ||3|| x86-64|| N||Known Problem||https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3231(Upstream, Don't need fix) ||
|-
|| Qt3-Azov Tests || ||/Qt3_Object_Model-t2c/tests/QMetaObject/QMetaObject ||15|| x86-64||N||Known Problem ||https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3231(Upstream, Don't need fix) ||
|-
|| Qt4-Azov Tests|| || /Normal_Test-t2c/tests/QSessionManager/QSessionManager|| 1-20|| x86 ||N||Known Problem ||https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3454 (Upstream, Don't need fix) ||
|-
|| Qt4-Azov Tests ||  || /Qt4_Input_Output-t2c/tests/QFileInfo/QFileInfo || 45 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3034 
|-
|| Qt4-Azov Tests ||  || /Qt4_Menus-t2c/tests/QMenuBar/QMenuBar || 30 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3124 
|-
|| Qt4-Azov Tests ||  || /Qt5_Menus-t2c/tests/QMenuBar/QMenuBar || 49 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3067 
|-
|| Qt4-Azov Tests ||  || /Qt4_Miscellaneous-t2c/tests/Qt4_Miscellaneous/Qt4_Miscellaneous || 1 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3134 
|-
|| Qt4-Azov Tests ||  || /Qt4_SQL-t2c/tests/QSqlTableModel/QSqlTableModel || 46 || x86-32, x86-64, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=2782 
|-
|| Printing Tests ||  || /testgs/testgs || 7 || x86-32, ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=2994
|-
|| Libstdc++ Tests||  ||/26_numerics/complex/13450.cc|| 7 || ppc || N||Known Problem||http://bugs.linuxbase.org/show_bug.cgi?id=2447
|-
|| Xpdf Tests ||    ||/opt/lsb/appbat/bin/pdftotext test1.pdf || 1 || ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3253
|-
|| Perl Tests ||  ||../ext/IPC/SysV/t/ipcsysv.t || 294 || x86-32, x86-64,ppc || N || Known Problem || http://bugs.linuxbase.org/show_bug.cgi?id=3249
|-
|| Perl Tests ||  ||op/pat.t || 142  || x86-32, x86-64,ppc|| N || Known Problem ||http://bugs.linuxbase.org/show_bug.cgi?id=3194
|-
|| Perl test || ||all/tst_perlModPresent.pl ||1|| x86, x86-64,ppc || N ||Upstream ||LSB 4.1 hasn't capability to test perl-5.14.2 because there are too many changes for new version perl than the old. So this is upstream problem. We can resolve this problem until the newer version for lsb test suite.(Upstream, Don't need fix)
|-
|| Perl test || ||../lib/AutoSplit.t ||402|| x86, x86-64,ppc || N ||Upstream || as same as above
|-
|| Perl test || ||../lib/autouse.t ||403|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/Class/ISA/t/00_about_verbose.t ||440|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/Class/ISA/t/01_old_junk.t ||441|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/Dumpvalue.t ||458|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/English.t ||460|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/ExtUtils/t/basic.t ||466|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/ExtUtils/t/FIRST_MAKEFILE.t ||469|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/ExtUtils/t/Liblist.t ||473|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/ExtUtils/t/PL_FILES.t ||480|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/ExtUtils/t/prereq_print.t ||483|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/ExtUtils/t/recurs.t ||486|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/ExtUtils/t/writemakefile_args.t ||490|| x86,x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/File/CheckTree.t ||493|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/overload.t ||644|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/Pod/t/eol.t ||651|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/Pod/t/Usage.t ||659|| x86, x86-64 || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/strict.t ||668|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/Test/Simple/t/overload.t ||708|| x86,ppc || N ||Upstream ||as same as above
|-
|| Perl test || ||../lib/Unicode/UCD.t ||809|| x86, x86-64,ppc || N ||Upstream ||as same as above
|-
|| Libstdc++ Tests || ||/27_io/basic_filebuf/sync/char/1.cc || 1 || x86-32,x86-64,ppc|| N || Snapshot pass ||It beyonds LSB Test Suite's power, LSB version is 4.1, to test higher version library libstdc++. More high version of LSB Test Suite will fix it.
|-
|| Libstdc++ Tests || ||/27_io/basic_filebuf/sync/wchar_t/1.cc || 1 || x86-32,x86-64,ppc|| N || Snapshot pass ||the same as above
|-
|| Libstdc++ Tests || ||/27_io/basic_istream/tellg/char/8348.cc || 1 || x86-32,x86-64,ppc|| N || Snapshot pass ||the same as above
|-
|| Libstdc++ Tests || ||/27_io/basic_istream/tellg/wchar_t/8348.cc || 1 || x86-32,x86-64,ppc|| N || Snapshot pass ||the same as above
|-
|| Libstdc++ Tests || ||/demangle/abi_examples/14.cc || 1 || x86-32,x86-64,ppc||N || Snapshot pass ||the same as above
|-
|| Libstdc++ Tests || ||/demangle/regression/cw-13.cc || 1 || x86-32,x86-64,ppc||N || Snapshot pass ||the same as above
ses||
|-
|| OLVER Core Tests || || /Normal_Test-t2c/tests/QSessionManager/QSessionManager ||1 ||x86, x86-64,ppc||N||Known Problem||https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=343
|-
|| Xts5 Tests || ||/tset/Xopen/strtkysym/Test ||7|| x86 || N||Known Problem || host contamination,Add patch "makekeys_crosscompile.patch" to libx11-1.4.4(fixed)
|-
|| Xts5 Tests  || ||/tset/Xlib10/kllclnt/Test 2nd || 2 || ppc || N || snapshot passes||
|-
|-
|| Tcl test || ||exec-9.7|| ||x86 ||N||Known Problem ||The test is ok by testing this case lonely. (Don't need fix)
|-
|}

Revision as of 07:15, 13 April 2012

Introduction

The Linux Standard Base(LSB) was created to lower the overall costs of supporting the Linux platform. By reducing the differences between individual Linux distributions, the LSB greatly reduces the costs involved with porting applications to different distributions, as well as lowers the cost and effort involved in after-market support of those applications. (http://www.linuxfoundation.org/collaborate/workgroups/lsb)

LSB Test Suite support the following architecture.

  • i486
  • ia64
  • ppc32
  • ppc64
  • s390
  • s390x
  • x86_64

LSB doesn't support arm and mips at present and maybe support in the future. YOCTO will make lsb test before releasing a new version.

Fixing bugs reported by LSB Test Suite has completed on Mar 5th. All of bugs have been fixed at that point. But some new bugs arise during lsb test recently because of the updating for OE-Core since Mar 5th. These new bugs will be fixed in 1.3.

Test Environment

  • Image: core-image-lsb-qt3
  • Platform: x86-32(emenlow), x86-64(jasperforest), ppc(mpc8315e-rdb)
  • Version of LSB Test Suite: 4.1

Setup test environment

We setup LSB test environment on emenlow as an example.

  • Build the image

1. Download the poky source

$ git clone git://git.pokylinux.org/poky.git

2. Download the intel BSP meta source

$ cd poky
$ git clone git://git.pokylinux.org/meta-intel.git

3. Downloand the qt3 meta source

$ git clone git://git.pokylinux.org/meta-qt3.git

4. Prepare the build environment

$ source oe-init-build-env

5. Edit the conf/bblayers.conf and conf/local.conf

$ vi conf/bblayers.conf 
…...
BBLAYERS = " \ 
  ##COREBASE##/meta \ 
  ##COREBASE##/meta-yocto \ 
  ##COREBASE##/meta-intel \
  ##COREBASE##/meta-intel/meta-emenlow \ 
  ##COREBASE##/meta-qt3 \
“
$ vi conf/local.conf
…...
MACHINE ?= "emenlow"
…... 
DISTRO ?= "poky-lsb 
…...

6. Build

$ bitbake core-image-lsb-qt3

When build successfully, you can find the kernel, rootfs image, rootfs cpio tarball, and modules tarball in build/tmp/deploy/images direcotry.


  • Install system

We assume that there is already a default OS on target(You can install the system by following the README.hardware in poky source). In order to install the new system, you need to prepare a free disk partition. If not, you need to create it by running 'fdisk' command. We assume the free disk partition is /dev/sda5.

There are 2 ways to install the rootfs to the target: Using rootfs image or using rootfs cpio tarball.

Using rootfs image:

1. Boot up the emenlow hardware by default mode.

2. Copy core-image-lsb-qt3-emenlow.ext3 to emenlow from host:

$ scp core-image-lsb-qt3-emenlow.ext3 root@emenlow-ip:/

3. Install the core-image-lsb-qt3-emenlow.ext3 to a free disk partition.

emenlow$ mount /dev/sda5 /mnt 
emenlow$ dd if=core-image-lsb-qt3-emenlow.ext3 of=/dev/sda5 
emenlow$ umount /mnt 
emenlow$ e2fsck -f /dev/sda5 
emenlow$ resize2fs /dev/sda5 


Using rootfs cpio tarball:

1. Boot up the emenlow hardware by default mode.

2. Copy core-image-lsb-qt3-emenlow.cpio.gz to emenlow from host:

$ scp core-image-lsb-qt3-emenlow.cpio.gz root@emenlow-ip:/

3. Install the core-image-lsb-qt3-emenlow.cpio.gz to a free disk partition.

emenlow$ gzip -d core-image-lsb-qt3-emenlow.cpio.gz
emenlow$ mount /dev/sda5 /mnt 
emenlow$ cd /mnt; cpio -iudm < ../core-image-lsb-qt3-emenlow.cpio 
emenlow$ sync; umount /mnt  


Then install the kernel and modified the grub.

1. Copy kernel to emenlow from host. We assume your boot disk partition is /dev/sda1:

$ scp bzImage-emenlow.bin root@emenlow-ip:/
$ mount /dev/sda1 /mnt
$ cp bzImage-emenlow.bin /mnt/boot/bzImage

2. Modify grub menu.lst

$ vi /mnt/boot/grub/menu.lst

add new lines as below:

title LSB 
root  (hd0,0) 
kernel /boot/bzImage root=/dev/sda5 rw

You can also change the default boot sequence.

3. Restart the emenlow


  • Install LSB testsuite

You can follow the File:LSB Test.pdf to install and run LSB test.

Result

Below table shows the result of LSB Test on different platforms.

  • X86(emenlow) result is updated in 2012-04-13.
  • X86-64(sugarbay) result is updated in 2012-04-13.
  • Ppc32(mpc8315e-rdb) result is updated in 2012-04-13
Test Version X86-32(emenlow) X86-64(sugarbay) ppc(mpc8315e-rdb)
Command Check v 4.1.0-1 Success Success Success
Library Check v 4.1.0-1 Failures: 1 Failures: 1 Failures: 1
Alsa-T2C Tests v 4.1.0-1 Success Success Success
Core Tests v 4.1.0-1 Failures: 26 Failures: 27 Failures: 67
Core-T2C Tests v 4.1.0-1 Success Success Success
Cpp-T2C Tests v 4.1.0-1 Success Success Success
Desktop Tests v 4.1.0-2 Failures: 27 Failures: 27 Failures: 30
Desktop-T2C Tests v 4.1.0-1 Failures: 6 Failures: 24 Failures: 20
Libstdc++ Tests v 4.1.0-13 Failures: 6 Failures: 6 Failures: 8
OLVER Core Tests v 4.1.0-1 Failures: 10 Failures: 10 Block
Perl Tests v 4.1.0-1 Failures: 87 Failures: 87 Failures: 87
Printing Tests v 4.1.0-1 Success Success Failures: 3
Python Runtime Tests v 4.1.1-2 Success Success Success
Qt3-Azov Tests v 4.1.0-1 Success Failures: 2 Failures: 202
Qt4-Azov Tests v 4.1.0-1 Failures: 25 Failures: 25 Failures: 92
Xml2-Azov Tests v 4.1.0-1 Success Success Success
Xts5 Tests v 5.1.5-38 Success Success 627
Apache Tests v 2.2.14-3 Success Success Success
Expect Tests v 5.43.0-11 Success Success Success
Groff Tests v 1.20.1-5 Success Success Success
Raptor Tests v 1.4.19-3 Success Success Success
Rsync Tests v 3.0.6-3 Success Success Success
Samba Tests v 3.4.3-5 Success Success Success
Tcl Tests v 8.5.7-6 Failures:1 Success Sucess
Xpdf Tests v 1.01-10 Success Success Failures: 1

Analysis of result

Below table shows the failed test cases.

  • Total failures: The number of total failures after running test. It includes known problems and real failures which should be fixed.
  • Known problem: Known problem means it is a bug of LSB itself and we can find trace these bugs on http://bugs.linuxbase.org. On link page we give a summary of known problems.
  • Failures(need to fix) : Real failures we should fix them
  • Test suite error : Caused by two reasons: one is for absent library needed by test suite. one is that test suite could lose some function which should be included in it.

x86-32 (emenlow,update(2012-04-13))

  • New problem: It is a bug without arising on previous version. It could be caused by some patches updated every day.
Test Total failures Known problem Failures(need to fix) Note
Core Tests 26 26
Library Check 1 1 missing library "libgdk_pixbuf_xlib-2.0.so.0" due to the updating of gdk-pixbuf
Libstdc++ Tests 6 6 all of these bugs go beyond the capacity of LSB 4.1. Perhaps, LSB 5.0 have the capacity to test high version libstdc++.
Desktop Tests 27 27
Desktop-T2C Tests 6 6
OLVER Core Tests 10 10
Perl Tests 87 87 all of these bugs go beyond the capacity of LSB 4.1. Perhaps, LSB 5.0 have the capacity to test high version perl
Qt4-Azov Tests 25 25
Tcl Tests 1 1

x86-64 (sugarbay,update(2012-04-13))

  • New problem: It is a bug without arising on previous version. It could be caused by some patches updated every day.
Test Total failures Known problem Failures(need to fix) Note
Library Check 1 1 as the same result as x86
Core Tests 27 26 1 Need to investigate this in Yocto 1.3
Libstdc++ Tests 6 6 as the same result as x86
Desktop Tests 27 27
Desktop-T2C Tests 24 24
OLVER Core Tests 10 10
Perl Tests 87 87 as the same result as x86
Qt4-Azov Tests 25 25
Qt3 Tests 2 2

ppc(mpc8315e,update partly(2012-04-13))

We have the following failures(including "unresolved" and "failed") to fix.

Test Total failures Known problem Failures(need to fix) Note
Core Tests 67 67
Desktop Tests 30 30
Desktop-T2C Tests 20 20
Libstdc++ Tests 8 8
OLVER Core Tests Blocked
Perl Tests 87 87
Printing Tests 3 3
Qt3-Azov 202 No enough memory
Qt4-Azov 92 92 as same as Qt3
  • For new problem, it has others besides the problems emenlow has
    • Qt4-Azov Tests
      • "Child process timed out" caused many unresolved items
    • No locale and charmap in the latest lsb image, there were many failures because of the absent of locale resources.
  • Illegal instruction(Known problem http://bugs.linuxbase.org/show_bug.cgi?id=3253)
    • 65 SIGILL error
      • Core Tests
        • /tset/ANSI.os/math/... : SIGILL cause 47 unresolved failures.
        • /tset/LSB.os/libm/fpeInvalid/T.fpeInvalid (1st--2nd)
      • Desktop-t2c Tests
        • /glib-t2c/tests/glib_random/glib_random (9th--23rd)
      • Xpdf Tests
        • pdftotext
    • It was caused by the difference of compilation tool between LSB Test Suite and mpc8315e. In fact, instruction "fsqrt" doesn't exist on e300core serial and CPU of mpc8315e belongs to e300core type. So this problem can't be solved for mpc8315e
  • OLVER Core Tests:
    • OLVER Core Tests can't be finished on mpc8315e-rdb
    • We think the reason which make test blocked is the low memory(128M on mpc8315e). Because we re-run the test case which is blocked during OLVER core tests and it can pass successfully. That is to say, if we run the single test case, it will be passed. But we run all OLVER core tests and the whole tests will be blocked. We have ever run OLVER Core Tests for nearly 50 hours and it can't finish.
    • lsb-test-olver-core may require about 512Mb of system memory (with an X server and a contemporary desktop running), as well as 2Gb of free disk space to complete a test run. The reference linkage: http://ldn.linuxfoundation.org/lsb/preparing-lsb-distribution-testing
  • Qt3-Azov Tests, Printing Tests
    • Out of memory error cause all of the failures, the information like the following arise on screen when running Qt3 and Printing test suite
   Killed process 22453 (Xvfb) total-vm:84064kB, anon-rss:73720kB, file-rss:8kB
   run-parts invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0,
   oom_score_adj=0
   Call Trace:
   [c0cadc90] [c0009f3c] show_stack+0x108/0x1c4 (unreliable)
   [c0cadce0] [c0431098] dump_stack+0x2c/0x44
   [c0cadcf0] [c00a1880] dump_header.clone.14+0x70/0x184
   [c0cadd30] [c00a1b80] oom_kill_process.clone.16+0x94/0x294
   [c0cadd60] [c00a222c] out_of_memory+0x270/0x33c
   [c0caddb0] [c00a6628] __alloc_pages_nodemask+0x624/0x65c
   [c0cade50] [c00a6694] __get_free_pages+0x34/0x6c
   [c0cade60] [c00269f8] copy_process+0xe4/0xd84   
   [c0cadec0] [c0027764] do_fork+0x80/0x39c
   [c0cadf20] [c0009cb4] sys_vfork+0x68/0x80
   [c0cadf40] [c0011d0c] ret_from_syscall+0x0/0x38       
     --- Exception: c00 at 0xfecc358
     LR = 0x10084518
    • I think this should be limited memory problem
  • Libstdc++ Tests
    • /26_numerics/complex/13450.cc
      • This test case check whether data type "long double" can't be supported or not on ppc. Mp8315e deviates from what the test expects. It is Known problem.
    • /27_io/basic_stringbuf/overflow/wchar_

Failures tracker(update 2012-04-13)

We should fix the following problems. We also filed bugs for them and try to fix them.

Tests Sub Tests Test Number Arch Bug ID of yocto Proposal suggestion
Library Check Unable to find library libgdk_pixbuf_xlib-2.0.so.0 x86,x86-64,ppc This bug need to fix in 1.3
Core Tests /tset/LSB.pam/testcases/pam_fail_delay/pam_fail_delay 2 x86-64 This bug need to be investigated deeply

Known problems tracker

See details on Known problems tracker

LSB Bug tracker(update 2012-04-13)

Below table is a summary of known problems during LSB tests. We can find these issues on http://bugs.linuxbase.org. Some bugs go beyond the capacity of LSB 4.1 because of the faster updating speed than LSB Test Suite for some packages.

Test SubTest TestCase TestNumber Arch Need fix? Notes Upstream Bugtracker
Core Tests /tset/POSIX.os/devclass/c_iflag/T.c_iflag 6, 7, 8, 9 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3033
Core Tests /tset/POSIX.os/devclass/tcgetattr 1, 2 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3033
Core Tests /tset/LSB.os/ioprim/readv_L/T.readv_L 24 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=844
Core Tests /tset/LSB.os/ioprim/writev_L/T.writev_L 30 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=844
Core Tests /tset/ANSI.os/maths/* 47 ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3253
Core Tests /tset/LSB.os/libm/fpeInvalid/T.fpeInvalid 2 ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3253
Core Tests /tset/POSIX.os/files/fpathconf/T.fpathconf 15,16 x86,x86-64,ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3078
Core Tests /tset/LI18NUX2K.L1/base/iswcntrl/T.iswcntrl 1st 1 ppc N Upstream Compilation tool of LSB cause this problem, Compile test suite's source codes with poky compilation tool and no this problem. It can pass test with private test suite from Linux Foundation which will be merged to new version. http://dev.linuxfoundation.org/~stewb/ppc32/lsb-test-core-4.1.9-1.1.ppc.rpm
Core Tests /tset/LI18NUX2K.L1/base/iswdigit/T.iswdigit 1st 1 ppc N Upstream the same as above
Core Tests /tset/LI18NUX2K.L1/base/iswlower/T.iswlower 1st 1 ppc N Upstream the same as above
Core Tests /tset/LI18NUX2K.L1/base/iswupper/T.iswupper 1st 1 ppc N Upstream the same as above
Core Tests /tset/LI18NUX2K.L1/base/iswxdigit/T.iswxdigit 1st 1 ppc N Upstream the same as above
Core test /tset/POSIX.os/procprim/fork/T.fork 5 x86-64 N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=2270(fixed after 4.1)
Core test /tset/LSB.fhs/var/lib-hwclock/lib-hwclock-tc 1 x86,x86-64,ppc N Known Problem https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3116(Upstream, Don't need fix)
Core test /tset/LSB.os/network/gethostbyaddr/T.gethostbyaddr 2 x86,x86-64,ppc N Known Problem Absent startup program avahi-daemon(Fixed).
Desktop Tests desktop-gtkvts /tests/functions/Param_Value/Param_Value 20 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3044
Desktop Tests desktop-gtkvts /tests/functions/Selection/Selections 33, 34 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3011
Desktop Tests desktop-gtkvts /tests/functions/Keyboard_Accelerator_Groups/Keyboard_Accelerator_Groups 27-29 x86,x86-64,ppc N Known Problem Function "gtk_accelerator_name" from gtk/gtkaccelgroup.c has been changed since version 2.24.8. So this is incorrect test case now.(Upstream, Don't need fix)
Desktop Tests desktop-gtkvts /tests/functions/Themeable_Stock_Images/Themeable_Stock_Images 195-210 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3011
Desktop Tests desktop-gtkvts /tests/functions/GtkTreeSortable/GtkTreeSortable 15-18 x86,x86-64,ppc N Upstream https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3238
Desktop Tests /tests/functions/SpawnProc/SpawnProc 2nd (unresolved) 2 ppc N Known Problem http://bugzilla.pokylinux.org/show_bug.cgi?id=1117
Desktop Tests /tests/functions/GHashTable/GHashTable 5 x86-32,x86-64,ppc N snapshot pas
Desktop-T2c /atk-t2c/tests/AtkObject_relation/AtkObject_relation 2,5,6 x86,x86-64,ppc N Upstream https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3189
Desktop-T2c /gobject-t2c/tests/gobject_value_arrays/gobject_value_arrays 24 x86,x86-64,ppc N Upstream codes of qsort_r has changed since 2.30.2 So this is upstream problem. The higher version of LSB could modify this problem.
Desktop-T2c /gobject-t2c/tests/gobject_gtype_instance/gobject_gtype_instance 39 x86.x86-64,ppc N Upstream https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3184
Desktop-T2C Tests /freetype-t2c/tests/truetype_tables/truetype_tables 2 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3153
Desktop-t2c /glib-t2c/tests/glib_threads/glib_threads 36 ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3042
Desktop-T2C /gobject-t2c/tests/gobject_gtype_instance/gobject_gtype_instance 39 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3184
Desktop-t2c /glib-t2c/tests/glib_random/glib_random 9th-23th 14 ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3253
OLVER Core Tests /olver/math/cexp/tests/cpow_spec 5162 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=2912
OLVER Core Tests locale_locale_scenario /olver/other/other/tests/others 1 x86-32, x86-64 N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=2871
OLVER Core Tests ncurses_string_insert_scenario /olver/other/other/tests/others 2 x86-32, x86-64 N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=2871
OLVER Core Tests ncurses_string_add_scenario /olver/other/other/tests/others 3 x86-32, x86-64 N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=2871
OLVER Core Tests /olver/other/other/tests/others 561 x86-32, x86-64 N Known Problem http://linuxtesting.org/results/report?num=S0561
OLVER Core Tests ncurses_line_rip_scenario /olver/other/other/tests/others 5 x86-32, x86-64 N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=2871
OLVER Core Tests /olver/util/compress/tests/gzputc_spec 1 x86-32, x86-64 N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3057
OLVER Core Tests /olver/util/format/tests/getdata_spec 803 x86-32, x86-64 N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=2913
OLVER Core Tests /olver/util/format/tests/strptime_spec 810, 808 x86-32, x86-64 N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=2914
Qt3-Azov Tests /Qt3_Object_Model-t2c/tests/QMetaObject/QMetaObject 3 x86-64 N Known Problem https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3231(Upstream, Don't need fix)
Qt3-Azov Tests /Qt3_Object_Model-t2c/tests/QMetaObject/QMetaObject 15 x86-64 N Known Problem https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3231(Upstream, Don't need fix)
Qt4-Azov Tests /Normal_Test-t2c/tests/QSessionManager/QSessionManager 1-20 x86 N Known Problem https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3454 (Upstream, Don't need fix)
Qt4-Azov Tests /Qt4_Input_Output-t2c/tests/QFileInfo/QFileInfo 45 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3034
Qt4-Azov Tests /Qt4_Menus-t2c/tests/QMenuBar/QMenuBar 30 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3124
Qt4-Azov Tests /Qt5_Menus-t2c/tests/QMenuBar/QMenuBar 49 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3067
Qt4-Azov Tests /Qt4_Miscellaneous-t2c/tests/Qt4_Miscellaneous/Qt4_Miscellaneous 1 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3134
Qt4-Azov Tests /Qt4_SQL-t2c/tests/QSqlTableModel/QSqlTableModel 46 x86-32, x86-64, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=2782
Printing Tests /testgs/testgs 7 x86-32, ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=2994
Libstdc++ Tests /26_numerics/complex/13450.cc 7 ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=2447
Xpdf Tests /opt/lsb/appbat/bin/pdftotext test1.pdf 1 ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3253
Perl Tests ../ext/IPC/SysV/t/ipcsysv.t 294 x86-32, x86-64,ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3249
Perl Tests op/pat.t 142 x86-32, x86-64,ppc N Known Problem http://bugs.linuxbase.org/show_bug.cgi?id=3194
Perl test all/tst_perlModPresent.pl 1 x86, x86-64,ppc N Upstream LSB 4.1 hasn't capability to test perl-5.14.2 because there are too many changes for new version perl than the old. So this is upstream problem. We can resolve this problem until the newer version for lsb test suite.(Upstream, Don't need fix)
Perl test ../lib/AutoSplit.t 402 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/autouse.t 403 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/Class/ISA/t/00_about_verbose.t 440 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/Class/ISA/t/01_old_junk.t 441 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/Dumpvalue.t 458 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/English.t 460 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/ExtUtils/t/basic.t 466 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/ExtUtils/t/FIRST_MAKEFILE.t 469 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/ExtUtils/t/Liblist.t 473 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/ExtUtils/t/PL_FILES.t 480 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/ExtUtils/t/prereq_print.t 483 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/ExtUtils/t/recurs.t 486 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/ExtUtils/t/writemakefile_args.t 490 x86,x86-64,ppc N Upstream as same as above
Perl test ../lib/File/CheckTree.t 493 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/overload.t 644 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/Pod/t/eol.t 651 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/Pod/t/Usage.t 659 x86, x86-64 N Upstream as same as above
Perl test ../lib/strict.t 668 x86, x86-64,ppc N Upstream as same as above
Perl test ../lib/Test/Simple/t/overload.t 708 x86,ppc N Upstream as same as above
Perl test ../lib/Unicode/UCD.t 809 x86, x86-64,ppc N Upstream as same as above
Libstdc++ Tests /27_io/basic_filebuf/sync/char/1.cc 1 x86-32,x86-64,ppc N Snapshot pass It beyonds LSB Test Suite's power, LSB version is 4.1, to test higher version library libstdc++. More high version of LSB Test Suite will fix it.
Libstdc++ Tests /27_io/basic_filebuf/sync/wchar_t/1.cc 1 x86-32,x86-64,ppc N Snapshot pass the same as above
Libstdc++ Tests /27_io/basic_istream/tellg/char/8348.cc 1 x86-32,x86-64,ppc N Snapshot pass the same as above
Libstdc++ Tests /27_io/basic_istream/tellg/wchar_t/8348.cc 1 x86-32,x86-64,ppc N Snapshot pass the same as above
Libstdc++ Tests /demangle/abi_examples/14.cc 1 x86-32,x86-64,ppc N Snapshot pass the same as above
Libstdc++ Tests /demangle/regression/cw-13.cc 1 x86-32,x86-64,ppc N Snapshot pass the same as above

ses||

OLVER Core Tests /Normal_Test-t2c/tests/QSessionManager/QSessionManager 1 x86, x86-64,ppc N Known Problem https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=343
Xts5 Tests /tset/Xopen/strtkysym/Test 7 x86 N Known Problem host contamination,Add patch "makekeys_crosscompile.patch" to libx11-1.4.4(fixed)
Xts5 Tests /tset/Xlib10/kllclnt/Test 2nd 2 ppc N snapshot passes
Tcl test exec-9.7 x86 N Known Problem The test is ok by testing this case lonely. (Don't need fix)