LSB Result: Difference between revisions

From Yocto Project
Jump to navigationJump to search
Line 23: Line 23:
1. Download the poky source
1. Download the poky source


$ git clone git://git.pokylinux.org/poky.git
$ git clone git://git.pokylinux.org/poky.git


2. Download the intel BSP meta source
2. Download the intel BSP meta source


$ cd poky
$ cd poky


$ ls
$ ls


bitbake  documentation  LICENSE  meta  meta-demoapps  meta-rt  meta-yocto  oe-init-build-env  pyshtables.py  README  README.hardware scripts
bitbake  documentation  LICENSE  meta  meta-demoapps  meta-rt  meta-yocto  oe-init-build-env  pyshtables.py  README  README.hardware scripts


$ git clone git://git.pokylinux.org/meta-intel.git
$ git clone git://git.pokylinux.org/meta-intel.git


$ ls
$ ls


bitbake  documentation  LICENSE  meta  meta-demoapps  meta-intel meta-rt  meta-yocto  oe-init-build-env  pyshtables.py  README README.hardware  scripts
bitbake  documentation  LICENSE  meta  meta-demoapps  meta-intel meta-rt  meta-yocto  oe-init-build-env  pyshtables.py  README README.hardware  scripts


3. Downloand the qt3 meta source
3. Downloand the qt3 meta source


$ git clone git://git.pokylinux.org/meta-qt3.git
$ git clone git://git.pokylinux.org/meta-qt3.git


$ ls
$ ls


bitbake  documentation  LICENSE  meta  meta-demoapps  meta-intel meta-qt3  meta-rt  meta-yocto  oe-init-build-env  pyshtables.py  README README.hardware  scripts
bitbake  documentation  LICENSE  meta  meta-demoapps  meta-intel meta-qt3  meta-rt  meta-yocto  oe-init-build-env  pyshtables.py  README README.hardware  scripts


4. Prepare the build environment  
4. Prepare the build environment  


$ source oe-init-build-env
$ source oe-init-build-env


5. edit the conf/bblayers.conf and conf/local.conf
5. edit the conf/bblayers.conf and conf/local.conf

Revision as of 09:10, 25 July 2011

Introduction

The Linux Standard Base(LSB) was created to lower the overall costs of supporting the Linux platform. By reducing the differences between individual Linux distributions, the LSB greatly reduces the costs involved with porting applications to different distributions, as well as lowers the cost and effort involved in after-market support of those applications. (http://www.linuxfoundation.org/collaborate/workgroups/lsb)

LSB Test Suite support the following architecture.

  • i486
  • ia64
  • ppc32
  • ppc64
  • s390
  • s390x
  • x86_64

So LSB don't support arm and mips at present time and it will add test suite to support arm and mips possibly in the future.

Test Environment

  • Image: core-image-lsb-qt3
  • Platform: x86-32(emenlow), x86-64(jasperforest), ppc(mpc8315e-rdb)
  • Version of LSB Test Suite: 4.1

How to test LSB

We take emenlow for example to make lsb test.

  • Build

1. Download the poky source

$ git clone git://git.pokylinux.org/poky.git

2. Download the intel BSP meta source

$ cd poky
$ ls
bitbake  documentation  LICENSE  meta  meta-demoapps  meta-rt  meta-yocto  oe-init-build-env  pyshtables.py  README  README.hardware scripts
$ git clone git://git.pokylinux.org/meta-intel.git
$ ls
bitbake  documentation  LICENSE  meta  meta-demoapps  meta-intel meta-rt  meta-yocto  oe-init-build-env  pyshtables.py  README README.hardware  scripts

3. Downloand the qt3 meta source

$ git clone git://git.pokylinux.org/meta-qt3.git
$ ls
bitbake  documentation  LICENSE  meta  meta-demoapps  meta-intel meta-qt3  meta-rt  meta-yocto  oe-init-build-env  pyshtables.py  README README.hardware  scripts

4. Prepare the build environment

$ source oe-init-build-env

5. edit the conf/bblayers.conf and conf/local.conf

$ vi conf/bblayers.conf 
…...
BBLAYERS = " \ 
 <POKY_BASE>/meta \ 
 <POKY_BASE>/meta-yocto \ 
 <POKY_BASE>/meta-intel/meta-emenlow \ 
 <POKY_BASE>/meta-qt3 \
“
$ vi conf/local.conf
…...
MACHINE ?= "emenlow"
…... 
DISTRO ?= "poky-lsb 
…...

6. Prepare the download source directory

7. Build the image

$ bitbake core-image-lsb-qt3

8. When build successfully, you can find the kernel, rootfs image, rootfs cpio tarball, and modules tarball in build/tmp/deploy/images direcotry.

Installation You can install the rootfs to the hardware by using rootfs image or rootfs cpio tarball. Using rootfs image:

1. Boot up the emenlow hardware by default mode.

2. Copy core-image-lsb-qt3-emenlow.ext3 to emenlow from host:

$ scp core-image-lsb-qt3-emenlow.ext3 root@emenlow-ip:/

3. Install the core-image-lsb-qt3-emenlow.ext3

emenlow$ mount /dev/sda5 /mnt 
emenlow$ dd if=core-image-lsb-qt3-emenlow.ext3 of=/dev/sda5 
emenlow$ umount /mnt 
emenlow$ e2fsck -f /dev/sda5 
emenlow$ resize2fs /dev/sda5 
	

Using rootfs cpio tarball: 1. Boot up the emenlow hardware by default mode. 2. Copy core-image-lsb-qt3-emenlow.cpio.gz to emenlow from host:

$ scp core-image-lsb-qt3-emenlow.cpio.gz root@emenlow-ip:/

3. Install the core-image-lsb-qt3-emenlow.cpio.gz

emenlow$ gzip -d core-image-lsb-qt3-emenlow.cpio.gz
emenlow$ mount /dev/sda5 /mnt 
emenlow$ cd /mnt; cpio -iudm < ../ core-image-lsb-qt3-emenlow.cpio 
emenlow$ sync; umount /mnt  

Then install the kernel and modified the grub

1. Copy kernel to emenlow from host: $ scp bzImage-emenlow.bin root@emenlow-ip:/ 2. mount /dev/sda1 /mnt 3. $ cp bzImage-emenlow.bin /mnt/boot/bzImage 4. Modify grub menu.lst $ vi /mnt/boot/grub/menu.lst add new lines as below:

title LSB root (hd0,0) kernel /boot/bzImage root=/dev/sda5 rw

modified the first line, change '0' to '1'

default 1

5. Restart the emenlow

Then install the LSB testsuite to the target: emenlow$ mount /dev/sda2 /mnt emenlow$ cp -rf /mnt/lsb-* . emenlow$ umount /mnt emenlow$ sh /usr/bin/LSB_setup.sh

You can refer to LSB_Test document for more information.

result

Below table shows the result of LSB Test on different platforms. X86 result is updated in 2011-06-23. Ppc32 result is updated in 2011-06-30

Test Version X86-32(emenlow) X86-64(jasperforest) ppc(mpc8315e-rdb)
Command Check v 4.1.0-1 Success Success
Library Check v 4.1.0-1 Warnings: 49 Warnings: 49
Alsa-T2C Tests v 4.1.0-1 Success Success
Core Tests v 4.1.0-1 Failures: 13 Failures: 68
Core-T2C Tests v 4.1.0-1 Success Success
Cpp-T2C Tests v 4.1.0-1 Success Success
Desktop Tests v 4.1.0-2 Failures: 18 23
Desktop-T2C Tests v 4.1.0-1 Failures: 3 16
Libstdc++ Tests v 4.1.0-13 Failures: 6 Failures: 8
OLVER Core Tests v 4.1.0-1 Failures: 10 Block
Perl Tests v 4.1.0-1 Failures: 1 Failures: 1
Printing Tests v 4.1.0-1 Failures: 1 Failures: 3
Python Runtime Tests v 4.1.1-2 Success Success
Qt3-Azov Tests v 4.1.0-1 Success Failures: 106
Qt4-Azov Tests v 4.1.0-1 Failures: 5 Failures: 69
Xml2-Azov Tests v 4.1.0-1 Success Success
Xts5 Tests v 5.1.5-38 Warnings: 8 Warnings: 8
Apache Tests v 2.2.14-3 Success Success
Expect Tests v 5.43.0-11 Success Success
Groff Tests v 1.20.1-5 Success Success
Raptor Tests v 1.4.19-3 Success Success
Rsync Tests v 3.0.6-3 Success Success
Samba Tests v 3.4.3-5 Success Success
Tcl Tests v 8.5.7-6 Success Success
Xpdf Tests v 1.01-10 Success Failures: 1

Analysis of result

Below table shows the failed test cases.

  • Total failures: The number of total failures after running test. It includes known problems and real failures which should be fixed.
  • Known problem: Known problem means it is a bug of LSB itself and we can find trace these bugs on http://bugs.linuxbase.org. On link page we give a summary of known problems.
  • Failures(need to fix) : Real failures we should fix them
  • Test suite error : Caused by two reasons: one is for absent library needed by test suite. one is that test suite could lose some function which should be included in it.

x86-32 (emenlow,update(2011-06-23))

  • New problem: It is a bug without arising on previous version. It could be caused by some patches updated every day.
Test Total failures Known problem Failures(need to fix) Note
Core Tests 13 13
Libstdc++ Tests 6 6
Desktop Tests 18 18
Desktop-T2C Tests 3 3
OLVER Core Tests 10 10
Perl Tests 1 1
Printing Tests 1 1
Qt4-Azov Tests 5 5

ppc(mpc8315e,update partly(2011-06-30))

We have the following failures(including "unresolved" and "failed") to fix.

Test Total failures Known problem Failures(need to fix) Note
Core Tests 68 68
Desktop Tests 23 23
Desktop-T2C Tests 16 16
Libstdc++ Tests 8 8
OLVER Core Tests Blocked
Perl Tests 1 1
Printing Tests 3 1 1(unresolved), 1(failure), 1(unreported)
Qt3-Azov 106 No enough memory
Qt4-Azov 1024 5 child process timed out
  • For new problem, it has others besides the problems emenlow has
    • Qt4-Azov Tests
      • "Child process timed out" caused many unresolved items
  • Illegal instruction(Known problem http://bugs.linuxbase.org/show_bug.cgi?id=3253)
    • 65 SIGILL error
      • Core Tests
        • /tset/ANSI.os/math/... : SIGILL cause 47 unresolved failures.
        • /tset/LSB.os/libm/fpeInvalid/T.fpeInvalid (1st--2nd)
      • Desktop-t2c Tests
        • /glib-t2c/tests/glib_random/glib_random (9th--23rd)
      • Xpdf Tests
        • pdftotext
    • It was caused by the difference of compilation tool between LSB Test Suite and mpc8315e. In fact, instruction "fsqrt" doesn't exist on e300core serial and CPU of mpc8315e belongs to e300core type. So this problem can't be solved for mpc8315e
  • OLVER Core Tests:
    • OLVER Core Tests can't be finished on mpc8315e-rdb
    • We think the reason which make test blocked is the low memory(128M on mpc8315e). Because we re-run the test case which is blocked during OLVER core tests and it can pass successfully. That is to say, if we run the single test case, it will be passed. But we run all OLVER core tests and the whole tests will be blocked. We have ever run OLVER Core Tests for nearly 50 hours and it can't finish.
    • lsb-test-olver-core may require about 512Mb of system memory (with an X server and a contemporary desktop running), as well as 2Gb of free disk space to complete a test run. The reference linkage: http://ldn.linuxfoundation.org/lsb/preparing-lsb-distribution-testing
  • Qt3-Azov Tests, Printing Tests
    • Out of memory error cause all of the failures, the information like the following arise on screen when running Qt3 and Printing test suite
   Killed process 22453 (Xvfb) total-vm:84064kB, anon-rss:73720kB, file-rss:8kB
   run-parts invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0,
   oom_score_adj=0
   Call Trace:
   [c0cadc90] [c0009f3c] show_stack+0x108/0x1c4 (unreliable)
   [c0cadce0] [c0431098] dump_stack+0x2c/0x44
   [c0cadcf0] [c00a1880] dump_header.clone.14+0x70/0x184
   [c0cadd30] [c00a1b80] oom_kill_process.clone.16+0x94/0x294
   [c0cadd60] [c00a222c] out_of_memory+0x270/0x33c
   [c0caddb0] [c00a6628] __alloc_pages_nodemask+0x624/0x65c
   [c0cade50] [c00a6694] __get_free_pages+0x34/0x6c
   [c0cade60] [c00269f8] copy_process+0xe4/0xd84   
   [c0cadec0] [c0027764] do_fork+0x80/0x39c
   [c0cadf20] [c0009cb4] sys_vfork+0x68/0x80
   [c0cadf40] [c0011d0c] ret_from_syscall+0x0/0x38       
     --- Exception: c00 at 0xfecc358
     LR = 0x10084518
    • I think this should be limited memory problem
  • Libstdc++ Tests
    • /26_numerics/complex/13450.cc
      • This test case check whether data type "long double" can't be supported or not on ppc. Mp8315e deviates from what the test expects. It is Known problem.
    • /27_io/basic_stringbuf/overflow/wchar_

Failures tracker(update 2011-06-23)

We should fix the following problems. We also filed bugs for them and try to fix them.

Tests Sub Tests Test Number Arch Bug ID of yocto Proposal suggestion
Desktop Tests /tests/functions/SpawnProc/SpawnProc 2nd (unresolved) 2 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=1117
Printing Tests /convenience/cupsConvenience 37th (unresolved) 37 ppc http://bugzilla.pokylinux.org/show_bug.cgi?id=1123
Qt4-Azov Tests Child process timed out ppc

Known problems tracker

See details on Known problems tracker