AutoBuilder/Adding Automated Tests
Preparation
Adding automated tests to the AutoBuilder assumes that the testing is done in either of the following formats:
- OpenEmbedded Selftest format
- Bitbake Runtime tests format
The AutoBuilder interfaces with the test tool via a calling command. Such test calling command is used to start executing a given set of tests. The test set description can come in the way of a configuration (file, variable, argument, etc) and it ultimately explicitly lists which tests are to be run. It is an expectation of the test tool that it procures the execution of the list of tests that has been specified at the moment of calling the starting command.
Sample Test for Instructions
For the purpose of this guide, the following test calling command will be used:
$ oe-selftest -r bbtests.BitbakeTests.test_warnings_errors
The test tool used is in the OpenEmbedded Selftest format and the calling line selects a single test from the BBTests suite. The reason for this choice is merely simplicity for writing this guide.
You can see the code of this particular test here (referencing the Krogoth release):
There should be about 5 lines of code there, the test can be simply explained as: execute a bitbake build from a file that doesn't exist and verify that it shows an error. It's a negative test.
Sample Test Output
Getting familiar with the output of the test is a good idea. Preparing the environment for running such test can be seen at the Oe-selftest#How_to_use section.
Once the test is run at the command line, the following terminal output is presented:
$ oe-selftest -r bbtests.BitbakeTests.test_warnings_errors 2016-06-15 13:55:30,964 - selftest - INFO - Running bitbake -e to get BBPATH 2016-06-15 13:55:32,027 - selftest - INFO - Checking that everything is in order before running the tests 2016-06-15 13:55:33,073 - selftest - INFO - Running bitbake -p 2016-06-15 13:55:36,228 - selftest - INFO - Loading tests from: oeqa.selftest.bbtests.BitbakeTests.test_warnings_errors 2016-06-15 13:55:36,230 - selftest - INFO - Adding: "include selftest.inc" in local.conf 2016-06-15 13:55:36,230 - selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf 2016-06-15 13:55:36 - test_warnings_errors (oeqa.selftest.bbtests.BitbakeTests) ... ok ---------------------------------------------------------------------- Ran 1 test in 0.782s OK 2016-06-15 13:55:37,012 - selftest - INFO - Finished 2016-06-15 13:55:37,012 - selftest - INFO - Removing the include from local.conf 2016-06-15 13:55:37,012 - selftest - INFO - Removing the include from bblayers.conf
an AutoBuilder to Run the Test
An AutoBuilder will be needed to run the testing, for letting the AutoBuilder know what you want to test, a config-set is required. Once you create the config-set, an AutoBuilder is needed to test and run it when finished.
