Image tests

About the the testimage class

The build system has the ability to run a series of automated tests for qemu images.

All the tests are actually commands run on the target system over ssh.

The tests themselves are written in Python, making use of the unittest module.

The class that enables this is testimage.bbclass (which handles loading the tests and starting the qemu image)

Enabling and running the tests

Requirements

You should be aware of the following:

• runqemu script needs sudo access for setting up the tap interface, so you need to make sure it can do that non-interactively. That means you need to one of the following:
  • add NOPASSWD for your user either for ALL commands, either just for runqemu-ifup (but you need to provide the full path and that can change if you have multiple poky clones) or for all commands
  • manually configure a tap interface for your system
  • run as root the script in scripts/runqemu-gen-tapdev which should generate a list of tap devices (that's usually done in AutoBuilder-like setups)
• the DISPLAY variable needs to be set so that means you need to have an X server available (AutoBuilder starts a vncserver)

To use it add "testimage" to global inherit and call your target image with -c testimage, like this:

• for example build a qemu core-image-sato: bitbake core-image-sato
• add INHERIT += "testimage" in local.conf
• then call "bitbake core-image-sato -c testimage". That will run a standard suite of tests.

The tests themselves are the modules found meta/lib/oeqa/runtime.

You can change the tests run by appending or overrding the TEST_SUITES variable in local.conf. Each name in TEST_SUITES represents a required test for the image. That means that no skipping is allowed (even if the test isn't suitable for the image, e.g running the rpm tests on a images with no rpm). Appending "auto" to TEST_SUITES means that it will try to run all tests that are suitable for the image (each test decides that on it's own).

Note that the order in TEST_SUITES is important (it's the order modules run) and it influences tests dependencies. That means that tests that depend on other tests (e.g ssh depends on the ping test) should be added last. Each module can have multiple classes with multiple test methods (and Python unittest rules apply here).