Ptest

Introduction

Ptest (package test) is a concept for building, installing and running the test suites that are included in many packages, and producing a consistent output format.

Adding ptest to your build

Ptest is enabled in your build by adding "ptest" to the DISTRO_FEATURES variable. This will cause ptest-enabled packages to build and install the test suite in /usr/lib/<package>/ptest.

Running ptest

The "ptest-runner" package installs a "ptest-runner" shell script which loops through all installed ptest test suites and runs them in sequence.

Adding ptest support to a package

What constitutes a ptest?

A ptest must at minimum contain two things: run-ptest and the actual test.

run-ptest is a minimal shell script that starts the test suite. Note: It must not contain the test suite, only start it!

The test can be anything, from a simple shell script running a binary and checking its output to an elaborate system of test binaries and data files.

One major point of ptest is to consolidate the output format of all tests into a single common format. The format selected is the automake "simple test" format:

  result: testname

Where "result" is one of PASS, FAIL or SKIP and "testname" can be any identifying string. (The the same format used by Automake: http://www.gnu.org/software/automake/manual/automake.html#Simple-Tests)

General recipe preparations

First, add "ptest" to the "inherit" line in the package recipe.

If a test adds build-time or run-time dependencies to the package which are not there normally (such as requiring "make" to run the test suite), add those with a -ptest suffix, like this:

  RDEPENDS_${PN}-ptest += "make"

Building the test suite

Few packages support cross-compiling their test suites, so this is something we typically have to add.

Many automake-based packages compile and run the test suite in a single command: "make check". This doesn't work when cross-compiling (since we need to build on host and run on target), so we need to split that into two targets: One for building the test and one for running it. Our build of automake comes with a patch which does this, so packages using the plain "make check" arrangement from automake get this automatically.

Now add a do_compile_append function to build the test suite:

  do_compile_append() {
     if [ "${PN}" = "${BPN}" -a ${PTEST_ENABLED} = "1" ]; then
        oe_runmake buildtest-TESTS
     fi
  }

Installing the test suite

The ptest.bbclass contains a ptest_do_install function which copies the required "run-ptest" file and runs "make install-ptest" if there is such a target in the top-level Makefile. This provides a standardized install method to use in the package do_install_append function:

  do_compile_append() {
     ptest_do_install
  }