Enabling Automation Test in Poky

Enable Automation Test in Poky

Currently we have implemented a handful automated sanity test cases in Poky (boot/ssh/dmesg/shutdown/zypper_help/zypper_search/rpm_query/compiler/connman). A user can follow the below steps to enable the runtime automation test:

• Make sure your working tree has latest test code from master. This feature was not part of the 0.9 release (but will be included in 1.0)
• Make sure your machine has the "expect" package installed. The package is necessary for testing. We will add a recipe for it later. Currently, you need install it by yourself. On Ubuntu host, you can use apt-get to install expect:

$ apt-get install expect

• Edit /etc/sudoers to allow the user run sudo without prompted for a password. For example, you can do like following for user "tester":

$ cat /etc/sudoers
# /etc/sudoers
#
# This file MUST be edited with the 'visudo' command as root.
#
# See the man page for details on how to write a sudoers file.
#
....
....
....

%tester ALL=(ALL) NOPASSWD: NOPASSWD: ALL

• Instead of requiring sudo access, you can run the poky-gen-tapdevs script from a build of meta-ide-support or a Poky toolchain install. This is recommended for more security-conscious environments. The script will create and set up networking for one or more tap devices that will be used automatically by the poky-qemu script if detected, and it will no longer require root access.
• Typically, you need a X environment to start QEMU. You can use your desktop's DISPLAY. Or, alternatively, you can install vncserver, and startup a X environment in localhost:x.0. (x can be 1, 2, 3.....n)
• For the VNC setup, the following xstartup script is known to work for an openSUSE 11.2 host:

#!/bin/sh

xrdb $HOME/.Xresources
xsetroot -solid grey
xhost +
xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
twm &

• For the VNC setup, you will also want to put the following in ~/.twmrc, which allows windows to be placed automatically.

RandomPlacement

• You need set "IMAGETEST" to "qemu" in build/conf/local.conf

$ cat local.conf
.....
# Set IMAGETEST to qemu if you want to build testcases and start
# testing in qemu after do_rootfs.
IMAGETEST = "qemu"
.....

• Select the test cases you want to run, by default all cases in sanity scenario(check the files under scripts/qemuimage-tests/sanity) will be run. If you want to choose only one or not all cases for testing, you can uncomment and edit TEST_SCEN in local.conf:

# By default test cases in sanity suite will be ran. If you want to run other
# test suite or specific test case(e.g. bat or boot test case under sanity suite),
# list them like following.
TEST_SCEN = "sanity:boot sanity:ssh sanity:dmesg"

• Then you can run "bitbake poky-image-xxx" (xxx can be minimal, sato, sdk, lsb). After the kernel/image are generated, you need run "bitbake poky-image-xxx -c qemuimagetest" or "bitbake poky-image-xxx -c qemuimagetest_standalone". It will trigger automation test and you will get the result after a while. qemuimagetest will re-generate rootfs image and qemuimagetest_standalone will use existing rootfs images under $DEPLOY_DIR/images.