LSB Result: Difference between revisions

From Yocto Project
Jump to navigationJump to search
 
(129 intermediate revisions by 4 users not shown)
Line 10: Line 10:
*s390x
*s390x
*x86_64
*x86_64
So LSB don't support arm and mips at present time and it will add test suite to support arm and mips possibly in the future.
LSB doesn't support arm and mips at present and maybe support in the future.
YOCTO will make lsb test before releasing a new version.
 
Fixing bugs reported by LSB Test Suite has completed  on Mar 5th 2012. All of bugs have been fixed at that point.  But some new bugs arise during lsb test recently because of the updating for OE-Core since Mar 5th 2012. These new bugs will be fixed in 1.3.


==Test Environment==
==Test Environment==
Line 17: Line 20:
* Version of LSB Test Suite: 4.1
* Version of LSB Test Suite: 4.1


==result==
==Setup test environment ==
 
We setup LSB test environment on emenlow as an example.
 
* Build the image
1. Download the poky source
 
$ git clone git://git.pokylinux.org/poky.git
 
2. Download the intel BSP meta source
 
$ cd poky
$ git clone git://git.pokylinux.org/meta-intel.git
 
3. Downloand the qt3 meta source
 
$ git clone git://git.pokylinux.org/meta-qt3.git
 
4. Prepare the build environment
 
$ source oe-init-build-env
 
5. Edit the conf/bblayers.conf and conf/local.conf(LSB 4.1 core test requires pam support, so open the distro feature “pam”）
 
$ vi conf/bblayers.conf
…...
BBLAYERS = " \
  ##COREBASE##/meta \
  ##COREBASE##/meta-yocto \
  ##COREBASE##/meta-intel \
  ##COREBASE##/meta-intel/meta-emenlow \
  ##COREBASE##/meta-qt3 \
“
$ vi conf/local.conf
…...
MACHINE ?= "emenlow"
…...
DISTRO ?= "poky-lsb
DISTRO_FEATURES_append += " pam"
…...
 
6. Build
 
$ bitbake core-image-lsb-qt3
 
When build successfully, you can find the kernel, rootfs image, rootfs cpio tarball, and modules tarball in build/tmp/deploy/images direcotry.
 
 
* Install system
 
We assume that there is already a default OS on target(You can install the system by following the README.hardware in poky source). In order to install the new system, you need to prepare a free disk partition. If not, you need to create it by running 'fdisk' command. We assume the free disk partition is /dev/sda5.
 
There are 2 ways to install the rootfs to the target: Using rootfs image or using rootfs cpio tarball.
Using rootfs image:
 
1. Boot up the emenlow hardware by default mode.
 
2. Copy core-image-lsb-qt3-emenlow.ext3 to emenlow from host:
 
$ scp core-image-lsb-qt3-emenlow.ext3 root@emenlow-ip:/
 
3. Install the core-image-lsb-qt3-emenlow.ext3 to a free disk partition.
 
emenlow$ mount /dev/sda5 /mnt
emenlow$ dd if=core-image-lsb-qt3-emenlow.ext3 of=/dev/sda5
emenlow$ umount /mnt
emenlow$ e2fsck -f /dev/sda5
emenlow$ resize2fs /dev/sda5
 
 
Using rootfs cpio tarball:
 
1. Boot up the emenlow hardware by default mode.
 
2. Copy core-image-lsb-qt3-emenlow.cpio.gz to emenlow from host:
 
$ scp core-image-lsb-qt3-emenlow.cpio.gz root@emenlow-ip:/
 
3. Install the core-image-lsb-qt3-emenlow.cpio.gz to a free disk partition.
 
emenlow$ gzip -d core-image-lsb-qt3-emenlow.cpio.gz
emenlow$ mount /dev/sda5 /mnt
emenlow$ cd /mnt; cpio -ivdm <../core-image-lsb-qt3-emenlow.cpio
emenlow$ sync; umount /mnt 
 
Then install the kernel and modified the grub.
 
1. Copy kernel to emenlow from host. We assume your boot disk partition is /dev/sda1:
 
$ scp bzImage-emenlow.bin root@emenlow-ip:/
$ mount /dev/sda1 /mnt
$ cp bzImage-emenlow.bin /mnt/boot/bzImage
 
2. Modify grub menu.lst
$ vi /mnt/boot/grub/menu.lst
 
add new lines as below:
 
title LSB
root  (hd0,0)
kernel /boot/bzImage root=/dev/sda5 rw
 
You can also change the default boot sequence.
 
3. Restart the emenlow
 
 
* Install LSB testsuite
 
You can follow the [[File:LSB_Test.pdf‎]] to install and run LSB test.
 
==Result==
Below table shows the result of LSB Test on different platforms.
Below table shows the result of LSB Test on different platforms.
X86 result is updated in 2011-06-16. Ppc32 result is update in 2011-06-16
* X86(emenlow) result is updated in 2013-03-12.  
* X86-64(sugarbay) result is updated in 2014-01-20.
* Ppc32(mpc8315e-rdb) result is updated in 2012-10-18
{|border="1"
{|border="1"
|| '''Test''' || '''Version''' || '''X86-32(emenlow)''' || '''X86-64(jasperforest)''' || '''ppc(mpc8315e-rdb)'''  
|| '''Test''' || '''Version''' || '''X86-32(emenlow)''' || '''X86-64(sugarbay)''' || '''ppc(mpc8315e-rdb)'''  
|-
|-
|| '''Command Check''' || v 4.1.0-1 || Failures: 1 || || Failures: 1
|| '''Command Check''' || v 4.1.4-5 || Success || Success || Success
|-
|-
|| '''Library Check''' || v 4.1.0-1 || Failures: 8 || || Failures: 10
|| '''Library Check''' || v 4.1.4-5 || Failures: 6 ||Warnings: 57 || Failures: 4
|-
|-
|| '''Alsa-T2C Tests''' || v 4.1.0-1 || Test suite error ||  || Test suite error
|| '''Alsa-T2C Tests''' || v 4.1.0-1 || Success ||  Success || Success
|-
|-
|| '''Core Tests''' || v 4.1.0-1 || Failures: 13 ||  || Failures: 66
|| '''Core Tests''' || v 4.1.15-1 || Failures: 32 ||  Failures: 18|| Failures: 589
|-
|-
|| '''Core-T2C Tests''' || v 4.1.0-1 || Success || || Success
|| '''Core-T2C Tests''' || v 4.1.2-3 || Success || Success || Failure: 1
|-
|-
|| '''Cpp-T2C Tests''' || v 4.1.0-1 || Success || || Success
|| '''Cpp-T2C Tests''' || v 4.1.0-1 || Success || Success || Failures: 150
|-
|-
|| '''Desktop Tests''' || v 4.1.0-2 || Test suite error ||   || Test suite error
|| '''Desktop Tests''' || v 4.1.9-1 ||Failures: 33 || Failures: 39 || Failures: 191
|-
|-
|| '''Desktop-T2C Tests''' || v 4.1.0-1 || Test suite error ||   || Test suite error
|| '''Desktop-T2C Tests''' || v 4.1.3-3 ||Failures: 18 || Failures: 45  || Failures: 137
|-
|-
|| '''Libstdc++ Tests''' || v 4.1.0-13 || Failures: 6 ||   || Failures: 2
|| '''Libstdc++ Tests''' || v 4.1.0-18 || Success || Success || There are problems
|-
|-
|| '''OLVER Core Tests''' || v 4.1.0-1 || Failures: 10 ||   || Block  
|| '''OLVER Core Tests''' || v 4.1.4-1 || Failures: 9 || Failures: 11 || Block  
|-
|-
|| '''Perl Tests''' || v 4.1.0-1 || Failures: 1 ||   || Failures: 1
|| '''Perl Tests''' || v 4.1.8-1 || Success ||  Failures: 1 || Success
|-
|-
|| '''Printing Tests''' || v 4.1.0-1 || Failures: 1 ||   || Failures: 3
|| '''Printing Tests''' || v 4.1.2-1 || Failures: 11 || Failures: 11  || Failures: 6
|-
|-
|| '''Python Runtime Tests''' || v 4.1.1-2 || Success ||   || Success  
|| '''Python Runtime Tests''' || v 4.1.5-1 || Failures: 3 || Success || Success  
|-
|-
|| '''Qt3-Azov Tests''' || v 4.1.0-1 || Success ||   || Failures: 106
|| '''Qt3-Azov Tests''' || v 4.1.1-3 || Warnings: 4 (Test suite issue) || Not Test  || Success
|-
|-
|| '''Qt4-Azov Tests''' || v 4.1.0-1 ||Failures: 5 ||   || Failures: 69
|| '''Qt4-Azov Tests''' || v 4.1.3-1 ||Failures: 6 || Failures: 1  || Failures: 173
|-
|-
|| '''Xml2-Azov Tests''' || v 4.1.0-1 ||Success ||   || Success
|| '''Xml2-Azov Tests''' || v 4.1.0-1 || Failures: 1 || Success || Failures: 3
|-
|-
|| '''Xts5 Tests''' || v 5.1.5-38 || Warnings: 8||   || Warnings: 8
|| '''Xts5 Tests''' || v 5.1.5-45 || Failure: 1 || Warnings: 8 || Failures: 10
|-
|-
|| '''Apache Tests''' || v 2.2.14-3 || Success || ||   
|| '''Apache Tests''' || v 2.2.14-8 || Success || Success ||  Success
|-
|-
|| '''Expect Tests''' || v 5.43.0-11 || Success || || Success  
|| '''Expect Tests''' || v 5.43.0-13 || Success || Success || Success  
|-
|-
|| '''Groff Tests''' || v 1.20.1-5 || Success ||   || Success  
|| '''Groff Tests''' || v 1.20.1-7 || Success || Success  || Success  
|-
|-
|| '''Raptor Tests''' || v 1.4.19-3 || Success ||   || Success  
|| '''Raptor Tests''' || v 1.4.19-5 || Success || There are problems || Success  
|-
|-
|| '''Rsync Tests''' || v 3.0.6-3 || Success ||   || Success
|| '''Rsync Tests''' || v 3.0.6-6 || Success || Success || Failures: 1
|-
|-
|| '''Samba Tests''' || v 3.4.3-5 || Success ||  || Success  
|| '''Samba Tests''' || v 3.4.3-9 || Success ||Success || Success  
|-
|-
|| '''Tcl Tests''' || v 8.5.7-6 || Success ||  || Success
|| '''Tcl Tests''' || v 8.5.7-8 || Failures:1 ||  There are problems|| Sucess
|-
|-
|| '''Xpdf Tests''' || v 1.01-10 || Success ||   || Failures: 1  
|| '''Xpdf Tests''' || v 1.01-14 || Success || Success  || Failures: 1  
|}
|}


Line 80: Line 198:
* '''Failures(need to fix)''' : Real failures we should fix them
* '''Failures(need to fix)''' : Real failures we should fix them
* '''Test suite error''' : Caused by two reasons: one is for absent library needed by test suite. one is that test suite could lose some function which should be included in it.
* '''Test suite error''' : Caused by two reasons: one is for absent library needed by test suite. one is that test suite could lose some function which should be included in it.
===x86-32 (emenlow,update(2011-06-16))===
===x86 (emenlow,update(2013-03-12))===
   
   
* '''New problem''': It is a bug without arising on previous version. It could be caused by some patches updated every day.
* '''New problem''': It is a bug without arising on previous version. It could be caused by some patches updated every day.
Line 86: Line 204:
|| '''Test''' || '''Total failures''' || '''Known problem''' || '''Failures(need to fix)''' || '''Note'''  
|| '''Test''' || '''Total failures''' || '''Known problem''' || '''Failures(need to fix)''' || '''Note'''  
|-
|-
|| '''Command Check''' || 1 || || 1 ||New problem,Absent command
|| '''Core Tests''' || 32 || 31 || 1 ||    
|-
|-
|| '''Alsa-T2C Tests''' || || || || New problem,Test suite error
|| '''Library Check''' || 6 || 0 || 6 ||  
|-
|-
|| '''Core Tests''' || 13 || 13 || ||    
|| '''Libstdc++ Tests''' || 0 || 0 || 0 ||  
|-
|-
|| '''Libstdc++ Tests''' || 6 || 6 ||  ||  
|| '''Desktop Tests''' || 33 || 32 || 1 ||  
|-
|-
|| ''' Library Check''' || 8 || || 8 ||  New problem,Absent libraries
|| '''Desktop-T2C Tests''' || 18 || 18 || 0 ||   
|-
|-
|| '''Desktop Tests''' || || ||   ||  New problem,Test suite error
|| '''OLVER Core Tests''' || 9 || 8 || 1  ||   
|-
|-
|| '''Desktop-T2C Tests''' || || || || New problem,Test suite error
|| '''Perl Tests''' || 0 || 0 || 0 ||  
|-
|-
|| '''OLVER Core Tests''' || 10 || 10 ||   ||
|| '''Python Tests''' || 3 || 0 || 3 ||
|-
|-
|| '''Perl Tests''' || 1 || 1 || ||  
|| '''Qt4-Azov Tests''' || 6 || 6 || 0 ||
|-
|-
|| '''Printing Tests''' || 1 || 1 || ||
|| '''Xts5 Tests''' || 1 || 1 || 0 ||
|-
|-
|| '''Qt4-Azov Tests''' || 5 || 5 || ||
|| '''Tcl Tests''' || 1 || 1 || 0 ||  
|-
|-
|}
|}


* Some new problems arose for latest lsb test
===x86-64 (sugarbay,update(2014-01-21))===
** Command Check
*** No command "strip" which existed in the previous lsb image.
* '''New problem''': It is a bug without arising on previous version. It could be caused by some patches updated every day.
** Library Check
*** The following libraries which existed in the previous lsb image are absent
**** libatk-1.0.so.0 ,libgdk-x11-2.0.so ,libcairo.so.2,library libpango-1.0.so,libpangoft2-1.0.so.0,libpangoxft-1.0.so.0 ,library libpangocairo-1.0.so 
** Desktop Tests,Desktop-T2C Tests(Test suite error)
*** It is caused by the above absent libraries because LSB Test Suite can't find the library it want to test.
*  Absent libQtopenGL
 
===ppc(mpc8315e,update(2011-06-16))===
We have the following failures(including "unresolved" and "failed") to fix.  
{|border="1"
{|border="1"
|| '''Test''' || '''Total failures''' || '''Known problem''' || '''Failures(need to fix)''' || '''Note'''  
|| '''Test''' || '''Total failures''' || '''Known problem''' || '''Failures(need to fix)''' || '''Note'''  
|-
|-
|| '''Command Check''' || 1 || || 1 ||New problem,Absent command
|| '''Library Check''' || 0 || 0 || 0 || 57 Warning
|-
|| '''Core Tests''' || 18 || 16 || 2 || /tset/LSB.fhs/root/sbin/sbin-tc 10; /tset/LSB.os/time/clock/T.clock 1
|-
|| '''Desktop Tests''' || 39 || 39 || 0 || 27 failures in /tests/functions/GtkTextIter/GtkTextIter, it's not a bug:https://bugzilla.redhat.com/show_bug.cgi?id=984537
|-
|| '''Desktop-T2C Tests''' || 45 || 33 || 12  || LSB #3740; LSB #3844; LSB #3799
|-
|-
|| '''Alsa-T2C Tests''' || || || || New problem,Test suite error
|| '''Libstdc++ Tests''' || 0 || 0 || 0 ||  
|-
|-
|| '''Core Tests''' || 66 || 10 || 47(unresolved), 8(failed), 1(unreported) ||  
|| '''OLVER Core Tests''' || 11 || 9 || 2  || LSB #3441; LSB #3762
|-
|| '''Perl Tests''' || 1 || 1 || 0 || LSB #3814
|-
|| ''' Printing Tests''' ||  12  || 5 || 7 || LSB #3603
|-
 
|| '''Qt4-Azov Tests''' || 1 || 1 || 0 || 
|-
|-
|| '''Desktop Tests''' || || || || New problem, test suite error
 
|| '''Xml2-Azov Tests''' || 0 || 0 || 0 ||
|-
|-
|| '''Desktop-T2C Tests''' || || || || New problem,  test suite error
 
|| '''Xts5 Tests''' || 0 || 0 || 0 || Warnings: 8
|-
|-
|| '''Library Check''' || 10 || 10 || || New problem, Absent libraries
|| '''Raptor Tests''' || 1 || 0 || 1 || Failed to start the test
|-
|-
|| '''Libstdc++ Tests''' || 2 ||   ||  ||
|| '''Tcl Tests''' || 1 || 0 || 1 || Failed to start the test
|-
 
|}
 
===ppc(mpc8315e,update (2012-04-13))===
We have the following failures(including "unresolved" and "failed") to fix.
{|border="1"
|| '''Test''' || '''Total failures''' || '''Known problem''' || '''Failures(need to fix)''' || '''Note'''
|-
|| '''Core Tests''' || 67 || 67 ||  ||   
|-
|| '''Desktop Tests''' || 30 || 30||  ||
|-
|| '''Desktop-T2C Tests''' || 20 || 20 ||  ||
|-
|| '''Libstdc++ Tests''' || 8 || 8  ||  ||  
|-
|-
|| '''OLVER Core Tests''' ||  ||  ||  || Blocked  
|| '''OLVER Core Tests''' ||  ||  ||  || Blocked  
|-
|-
|| '''Perl Tests''' || 1 || 1 ||  ||  
|| '''Perl Tests''' || 87 || 87 ||  ||
|-
|-
|| '''Printing Tests''' || 3 || 1 || 1(unresolved), 1(failure), 1(unreported)  ||   
|| '''Printing Tests''' || 3 || 3 ||   ||   
|-
|-
|| '''Qt3-Azov''' || 106 ||  ||  ||  No enough memory
|| '''Qt3-Azov''' || 202 ||  ||  ||  No enough memory
|-
|-
|| '''Qt4-Azov''' || 69 || 5 || 64(unresolved) || New problem
|| '''Qt4-Azov''' || 92 || 92 || || as same as Qt3
|-
|-
|| '''Xpdf Tests''' || 1 ||  || 1 || 
|}
|}
*For new problem, it has others besides the problems emenlow has
*For new problem, it has others besides the problems emenlow has
** Qt4-Azov Tests
** Qt4-Azov Tests
*** "Child process timed out" caused 64 unresolved itmes 
*** "Child process timed out" caused many unresolved items
*Illegal instruction
** No locale and charmap in the latest lsb image, there were many failures because of the absent of locale resources.
*Illegal instruction(Known problem http://bugs.linuxbase.org/show_bug.cgi?id=3253)
** 65 SIGILL error  
** 65 SIGILL error  
*** Core Tests
*** Core Tests
Line 193: Line 333:
** /26_numerics/complex/13450.cc
** /26_numerics/complex/13450.cc
*** This test case check whether data type "long double" can't be supported or not on ppc. Mp8315e deviates from what the test expects. It is Known problem.
*** This test case check whether data type "long double" can't be supported or not on ppc. Mp8315e deviates from what the test expects. It is Known problem.
** /27_io/basic_stringbuf/overflow/wchar_t/1.cc
** /27_io/basic_stringbuf/overflow/wchar_
*** No enough memory for this test case. It can pass on qemuppc with 256M and fail with 128M.


==Failures tracker(update 2011-06-16)==
==Failures tracker(update 2012-04-13)==
We should fix the following problems. We also filed bugs for them and try to fix them.
We should fix the following problems. We also filed bugs for them and try to fix them.
{|border="1"
{|border="1"
||  '''Tests''' || '''Sub Tests''' || '''Test Number''' || '''Arch''' || '''Bug ID of yocto''' || '''Proposal suggestion'''  
||  '''Tests''' || '''Sub Tests''' || '''Test Number''' || '''Arch''' || '''Bug ID of yocto''' || '''Proposal suggestion'''  
|-
|-
|| '''Core Tests''' || /tset/LI18NUX2K.L1/base/iswcntrl/T.iswcntrl 1st  || 1 || ppc || http://bugzilla.pokylinux.org/show_bug.cgi?id=1115 ||
|| '''Library Check''' || || Unable to find library libgdk_pixbuf_xlib-2.0.so.0|| x86,x86-64,ppc||  ||This bug need to fix in 1.3
|-
|| '''Core Tests''' || /tset/LI18NUX2K.L1/base/iswdigit/T.iswdigit 1st  || 1 || ppc || http://bugzilla.pokylinux.org/show_bug.cgi?id=1115 ||
|-
|| '''Core Tests''' || /tset/LI18NUX2K.L1/base/iswlower/T.iswlower 1st  || 1 || ppc || http://bugzilla.pokylinux.org/show_bug.cgi?id=1115 ||
|-
|| '''Core Tests''' || /tset/LI18NUX2K.L1/base/iswupper/T.iswupper  1st  || 1 || ppc || http://bugzilla.pokylinux.org/show_bug.cgi?id=1115 ||
|-
|| '''Core Tests''' || /tset/LI18NUX2K.L1/base/iswxdigit/T.iswxdigit 1st  || 1 || ppc || http://bugzilla.pokylinux.org/show_bug.cgi?id=1115 ||
|-
|| '''Desktop Tests''' || /tests/functions/SpawnProc/SpawnProc 2nd (unresolved) || 2 || ppc || http://bugzilla.pokylinux.org/show_bug.cgi?id=1117 ||
 
|-
|| '''Printing Tests''' || /convenience/cupsConvenience 37th (unresolved) || 37 || ppc || http://bugzilla.pokylinux.org/show_bug.cgi?id=1123 ||
|-
|| '''Library Check''' || No library libQtOpenGL||  || x86,x86-64,ppc|| ||http://git.pokylinux.org/cgit.cgi/poky-contrib/commit/?h=xiaofeng/opengl&id=1f612db123f367778de2a7a6fafd31b79538f075
http://git.pokylinux.org/cgit.cgi/poky-contrib/commit/?h=xiaofeng/opengl&id=f654a8321554458d47d1cab6ae1201c1f60467e3
|-
|| '''Command Check''' ||No command strip ||  || x86,x86-64,ppc ||  || upstream recipes should plus command "strip"  linking to command ${TARGET}-poky-linux-strip
|-
|| '''Library Check''' || No libraries:libatk-1.0.so.0 ,libgdk-x11-2.0.so ,libcairo.so.2,library libpango-1.0.so,libpangoft2-1.0.so.0,libpangoxft-1.0.so.0 ,library libpangocairo-1.0.so||  || x86,x86-64,ppc ||  ||Add packages including these libraries to task-core-lsb.bb or upstream recipes or task list
|-
|| '''Desktop Tests''' || Test suite error||  || x86,x86-64,ppc ||  ||After plussing the above packages, this test item can be test.
|-
|| '''Desktop-T2c Tests''' ||Test suite error ||  || x86,x86-64,ppc ||  ||After plussing the above packages, this test item can be test.
|-
|-
|| '''Alsa-T2C Tests''' ||Test suite error || ||x86,x86-64 ppc || ||Add package including library "libasound.so.2.0.0 "to task-core-lsb.bb or upstream recipes or task list
||'''Core Tests'''|| ||/tset/LSB.pam/testcases/pam_fail_delay/pam_fail_delay 2||x86-64|| ||This bug need to be investigated deeply
|-
|-
|| '''Qt4-Azov Tests''' ||Child process timed out ||  || ppc ||  ||
|-
|}
|}


==Known problems tracker==
==Known problems tracker==
See details on [[LSB_Bug_tracker | Known problems tracker]]
See details on [[LSB_Bug_tracker | Known problems tracker]]

Latest revision as of 02:58, 22 January 2014

Introduction

The Linux Standard Base(LSB) was created to lower the overall costs of supporting the Linux platform. By reducing the differences between individual Linux distributions, the LSB greatly reduces the costs involved with porting applications to different distributions, as well as lowers the cost and effort involved in after-market support of those applications. (http://www.linuxfoundation.org/collaborate/workgroups/lsb)

LSB Test Suite support the following architecture.

  • i486
  • ia64
  • ppc32
  • ppc64
  • s390
  • s390x
  • x86_64

LSB doesn't support arm and mips at present and maybe support in the future. YOCTO will make lsb test before releasing a new version.

Fixing bugs reported by LSB Test Suite has completed on Mar 5th 2012. All of bugs have been fixed at that point. But some new bugs arise during lsb test recently because of the updating for OE-Core since Mar 5th 2012. These new bugs will be fixed in 1.3.

Test Environment

  • Image: core-image-lsb-qt3
  • Platform: x86-32(emenlow), x86-64(jasperforest), ppc(mpc8315e-rdb)
  • Version of LSB Test Suite: 4.1

Setup test environment

We setup LSB test environment on emenlow as an example.

  • Build the image

1. Download the poky source 

$ git clone git://git.pokylinux.org/poky.git

2. Download the intel BSP meta source 

$ cd poky
$ git clone git://git.pokylinux.org/meta-intel.git

3. Downloand the qt3 meta source 

$ git clone git://git.pokylinux.org/meta-qt3.git

4. Prepare the build environment 

$ source oe-init-build-env

5. Edit the conf/bblayers.conf and conf/local.conf(LSB 4.1 core test requires pam support, so open the distro feature “pam”）

$ vi conf/bblayers.conf 

…...
BBLAYERS = " \ 
  ##COREBASE##/meta \ 
  ##COREBASE##/meta-yocto \ 
  ##COREBASE##/meta-intel \
  ##COREBASE##/meta-intel/meta-emenlow \ 
  ##COREBASE##/meta-qt3 \
“

$ vi conf/local.conf 

…...
MACHINE ?= "emenlow"
…... 
DISTRO ?= "poky-lsb 
DISTRO_FEATURES_append += " pam"
…...

6. Build 

$ bitbake core-image-lsb-qt3

When build successfully, you can find the kernel, rootfs image, rootfs cpio tarball, and modules tarball in build/tmp/deploy/images direcotry.


  • Install system

We assume that there is already a default OS on target(You can install the system by following the README.hardware in poky source). In order to install the new system, you need to prepare a free disk partition. If not, you need to create it by running 'fdisk' command. We assume the free disk partition is /dev/sda5.

There are 2 ways to install the rootfs to the target: Using rootfs image or using rootfs cpio tarball.

Using rootfs image:

1. Boot up the emenlow hardware by default mode.

2. Copy core-image-lsb-qt3-emenlow.ext3 to emenlow from host: 

$ scp core-image-lsb-qt3-emenlow.ext3 root@emenlow-ip:/

3. Install the core-image-lsb-qt3-emenlow.ext3 to a free disk partition. 

emenlow$ mount /dev/sda5 /mnt 
emenlow$ dd if=core-image-lsb-qt3-emenlow.ext3 of=/dev/sda5 
emenlow$ umount /mnt 
emenlow$ e2fsck -f /dev/sda5 
emenlow$ resize2fs /dev/sda5


Using rootfs cpio tarball:

1. Boot up the emenlow hardware by default mode.

2. Copy core-image-lsb-qt3-emenlow.cpio.gz to emenlow from host: 

$ scp core-image-lsb-qt3-emenlow.cpio.gz root@emenlow-ip:/

3. Install the core-image-lsb-qt3-emenlow.cpio.gz to a free disk partition. 

emenlow$ gzip -d core-image-lsb-qt3-emenlow.cpio.gz
emenlow$ mount /dev/sda5 /mnt 
emenlow$ cd /mnt; cpio -ivdm <../core-image-lsb-qt3-emenlow.cpio 
emenlow$ sync; umount /mnt


Then install the kernel and modified the grub.

1. Copy kernel to emenlow from host. We assume your boot disk partition is /dev/sda1: 

$ scp bzImage-emenlow.bin root@emenlow-ip:/
$ mount /dev/sda1 /mnt
$ cp bzImage-emenlow.bin /mnt/boot/bzImage

2. Modify grub menu.lst 

$ vi /mnt/boot/grub/menu.lst

add new lines as below: 

title LSB 
root  (hd0,0) 
kernel /boot/bzImage root=/dev/sda5 rw

You can also change the default boot sequence.

3. Restart the emenlow


  • Install LSB testsuite

You can follow the File:LSB Test.pdf to install and run LSB test.

Result

Below table shows the result of LSB Test on different platforms.

  • X86(emenlow) result is updated in 2013-03-12.
  • X86-64(sugarbay) result is updated in 2014-01-20.
  • Ppc32(mpc8315e-rdb) result is updated in 2012-10-18
Test Version X86-32(emenlow) X86-64(sugarbay) ppc(mpc8315e-rdb)
Command Check v 4.1.4-5 Success Success Success
Library Check v 4.1.4-5 Failures: 6 Warnings: 57 Failures: 4
Alsa-T2C Tests v 4.1.0-1 Success Success Success
Core Tests v 4.1.15-1 Failures: 32 Failures: 18 Failures: 589
Core-T2C Tests v 4.1.2-3 Success Success Failure: 1
Cpp-T2C Tests v 4.1.0-1 Success Success Failures: 150
Desktop Tests v 4.1.9-1 Failures: 33 Failures: 39 Failures: 191
Desktop-T2C Tests v 4.1.3-3 Failures: 18 Failures: 45 Failures: 137
Libstdc++ Tests v 4.1.0-18 Success Success There are problems
OLVER Core Tests v 4.1.4-1 Failures: 9 Failures: 11 Block
Perl Tests v 4.1.8-1 Success Failures: 1 Success
Printing Tests v 4.1.2-1 Failures: 11 Failures: 11 Failures: 6
Python Runtime Tests v 4.1.5-1 Failures: 3 Success Success
Qt3-Azov Tests v 4.1.1-3 Warnings: 4 (Test suite issue) Not Test Success
Qt4-Azov Tests v 4.1.3-1 Failures: 6 Failures: 1 Failures: 173
Xml2-Azov Tests v 4.1.0-1 Failures: 1 Success Failures: 3
Xts5 Tests v 5.1.5-45 Failure: 1 Warnings: 8 Failures: 10
Apache Tests v 2.2.14-8 Success Success Success
Expect Tests v 5.43.0-13 Success Success Success
Groff Tests v 1.20.1-7 Success Success Success
Raptor Tests v 1.4.19-5 Success There are problems Success
Rsync Tests v 3.0.6-6 Success Success Failures: 1
Samba Tests v 3.4.3-9 Success Success Success
Tcl Tests v 8.5.7-8 Failures:1 There are problems Sucess
Xpdf Tests v 1.01-14 Success Success Failures: 1

Analysis of result

Below table shows the failed test cases.

  • Total failures: The number of total failures after running test. It includes known problems and real failures which should be fixed.
  • Known problem: Known problem means it is a bug of LSB itself and we can find trace these bugs on http://bugs.linuxbase.org. On link page we give a summary of known problems.
  • Failures(need to fix) : Real failures we should fix them
  • Test suite error : Caused by two reasons: one is for absent library needed by test suite. one is that test suite could lose some function which should be included in it.

x86 (emenlow,update(2013-03-12))

  • New problem: It is a bug without arising on previous version. It could be caused by some patches updated every day.
Test Total failures Known problem Failures(need to fix) Note
Core Tests 32 31 1
Library Check 6 0 6
Libstdc++ Tests 0 0 0
Desktop Tests 33 32 1
Desktop-T2C Tests 18 18 0
OLVER Core Tests 9 8 1
Perl Tests 0 0 0
Python Tests 3 0 3
Qt4-Azov Tests 6 6 0
Xts5 Tests 1 1 0
Tcl Tests 1 1 0

x86-64 (sugarbay,update(2014-01-21))

  • New problem: It is a bug without arising on previous version. It could be caused by some patches updated every day.
Test Total failures Known problem Failures(need to fix) Note
Library Check 0 0 0 57 Warning
Core Tests 18 16 2 /tset/LSB.fhs/root/sbin/sbin-tc 10; /tset/LSB.os/time/clock/T.clock 1
Desktop Tests 39 39 0 27 failures in /tests/functions/GtkTextIter/GtkTextIter, it's not a bug:https://bugzilla.redhat.com/show_bug.cgi?id=984537
Desktop-T2C Tests 45 33 12 LSB #3740; LSB #3844; LSB #3799
Libstdc++ Tests 0 0 0
OLVER Core Tests 11 9 2 LSB #3441; LSB #3762
Perl Tests 1 1 0 LSB #3814
Printing Tests 12 5 7 LSB #3603
Qt4-Azov Tests 1 1 0
Xml2-Azov Tests 0 0 0
Xts5 Tests 0 0 0 Warnings: 8
Raptor Tests 1 0 1 Failed to start the test
Tcl Tests 1 0 1 Failed to start the test

ppc(mpc8315e,update (2012-04-13))

We have the following failures(including "unresolved" and "failed") to fix.

Test Total failures Known problem Failures(need to fix) Note
Core Tests 67 67
Desktop Tests 30 30
Desktop-T2C Tests 20 20
Libstdc++ Tests 8 8
OLVER Core Tests Blocked
Perl Tests 87 87
Printing Tests 3 3
Qt3-Azov 202 No enough memory
Qt4-Azov 92 92 as same as Qt3
  • For new problem, it has others besides the problems emenlow has
    • Qt4-Azov Tests
      • "Child process timed out" caused many unresolved items
    • No locale and charmap in the latest lsb image, there were many failures because of the absent of locale resources.
  • Illegal instruction(Known problem http://bugs.linuxbase.org/show_bug.cgi?id=3253)
    • 65 SIGILL error
      • Core Tests
        • /tset/ANSI.os/math/... : SIGILL cause 47 unresolved failures.
        • /tset/LSB.os/libm/fpeInvalid/T.fpeInvalid (1st--2nd)
      • Desktop-t2c Tests
        • /glib-t2c/tests/glib_random/glib_random (9th--23rd)
      • Xpdf Tests
        • pdftotext
    • It was caused by the difference of compilation tool between LSB Test Suite and mpc8315e. In fact, instruction "fsqrt" doesn't exist on e300core serial and CPU of mpc8315e belongs to e300core type. So this problem can't be solved for mpc8315e
  • OLVER Core Tests:
    • OLVER Core Tests can't be finished on mpc8315e-rdb
    • We think the reason which make test blocked is the low memory(128M on mpc8315e). Because we re-run the test case which is blocked during OLVER core tests and it can pass successfully. That is to say, if we run the single test case, it will be passed. But we run all OLVER core tests and the whole tests will be blocked. We have ever run OLVER Core Tests for nearly 50 hours and it can't finish.
    • lsb-test-olver-core may require about 512Mb of system memory (with an X server and a contemporary desktop running), as well as 2Gb of free disk space to complete a test run. The reference linkage: http://ldn.linuxfoundation.org/lsb/preparing-lsb-distribution-testing
  • Qt3-Azov Tests, Printing Tests
    • Out of memory error cause all of the failures, the information like the following arise on screen when running Qt3 and Printing test suite
   Killed process 22453 (Xvfb) total-vm:84064kB, anon-rss:73720kB, file-rss:8kB
   run-parts invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0,
   oom_score_adj=0
   Call Trace:
   [c0cadc90] [c0009f3c] show_stack+0x108/0x1c4 (unreliable)
   [c0cadce0] [c0431098] dump_stack+0x2c/0x44
   [c0cadcf0] [c00a1880] dump_header.clone.14+0x70/0x184
   [c0cadd30] [c00a1b80] oom_kill_process.clone.16+0x94/0x294
   [c0cadd60] [c00a222c] out_of_memory+0x270/0x33c
   [c0caddb0] [c00a6628] __alloc_pages_nodemask+0x624/0x65c
   [c0cade50] [c00a6694] __get_free_pages+0x34/0x6c
   [c0cade60] [c00269f8] copy_process+0xe4/0xd84   
   [c0cadec0] [c0027764] do_fork+0x80/0x39c
   [c0cadf20] [c0009cb4] sys_vfork+0x68/0x80
   [c0cadf40] [c0011d0c] ret_from_syscall+0x0/0x38       
     --- Exception: c00 at 0xfecc358
     LR = 0x10084518
    • I think this should be limited memory problem
  • Libstdc++ Tests
    • /26_numerics/complex/13450.cc
      • This test case check whether data type "long double" can't be supported or not on ppc. Mp8315e deviates from what the test expects. It is Known problem.
    • /27_io/basic_stringbuf/overflow/wchar_

Failures tracker(update 2012-04-13)

We should fix the following problems. We also filed bugs for them and try to fix them.

Tests Sub Tests Test Number Arch Bug ID of yocto Proposal suggestion
Library Check Unable to find library libgdk_pixbuf_xlib-2.0.so.0 x86,x86-64,ppc This bug need to fix in 1.3
Core Tests /tset/LSB.pam/testcases/pam_fail_delay/pam_fail_delay 2 x86-64 This bug need to be investigated deeply

Known problems tracker

See details on Known problems tracker

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=LSB_Result&oldid=11893"