AutoBuilder/Adding Automated Tests - Revision history

Benjamin at 19:55, 15 June 2016

2016-06-15T19:55:02Z

← Older revision		Revision as of 19:55, 15 June 2016
Line 51:		Line 51:

	An AutoBuilder will be needed to run the testing, for letting the AutoBuilder know what you want to test, a config-set is required. Once you create the config-set, an AutoBuilder is needed to test and run it when finished.		An AutoBuilder will be needed to run the testing, for letting the AutoBuilder know what you want to test, a config-set is required. Once you create the config-set, an AutoBuilder is needed to test and run it when finished.

			==== Basic AutoBuilder Knowledge====

			====Setup an AutoBuilder Instance for Testing====

			==Test Enabling Steps==

			===Write the Build Set===

			====Include the Test in a Build Step====

			===Test the Build Set===

Benjamin: /* Preparation */

2016-06-15T19:25:08Z

Preparation

← Older revision		Revision as of 19:25, 15 June 2016
Line 47:		Line 47:
	2016-06-15 13:55:37,012 - selftest - INFO - Removing the include from bblayers.conf		2016-06-15 13:55:37,012 - selftest - INFO - Removing the include from bblayers.conf
	</pre>		</pre>

			=== an AutoBuilder to Run the Test ===

			An AutoBuilder will be needed to run the testing, for letting the AutoBuilder know what you want to test, a config-set is required. Once you create the config-set, an AutoBuilder is needed to test and run it when finished.

Benjamin: /* Preparation */

2016-06-15T18:59:16Z

Preparation

← Older revision		Revision as of 18:59, 15 June 2016
Line 22:		Line 22:
	There should be about 5 lines of code there, the test can be simply explained as: execute a bitbake build from a file that doesn't exist and verify that it shows an error. It's a negative test.		There should be about 5 lines of code there, the test can be simply explained as: execute a bitbake build from a file that doesn't exist and verify that it shows an error. It's a negative test.

	===Sample Test Output===
			====Sample Test Output====

	Getting familiar with the output of the test is a good idea. Preparing the environment for running such test can be seen at the [[Oe-selftest#How_to_use]] section.		Getting familiar with the output of the test is a good idea. Preparing the environment for running such test can be seen at the [[Oe-selftest#How_to_use]] section.

Benjamin: /* Preparation */

2016-06-15T18:58:35Z

Preparation

← Older revision		Revision as of 18:58, 15 June 2016
Line 22:		Line 22:
	There should be about 5 lines of code there, the test can be simply explained as: execute a bitbake build from a file that doesn't exist and verify that it shows an error. It's a negative test.		There should be about 5 lines of code there, the test can be simply explained as: execute a bitbake build from a file that doesn't exist and verify that it shows an error. It's a negative test.

	Getting familiar with the output of the test is a good idea. Preparing the environment for running such test can be seen at the [[Oe-selftest#How_to_use]] section.		===Sample Test Output===
			Getting familiar with the output of the test is a good idea. Preparing the environment for running such test can be seen at the [[Oe-selftest#How_to_use]] section.

			Once the test is run at the command line, the following terminal output is presented:

			<pre>
			$ oe-selftest -r bbtests.BitbakeTests.test_warnings_errors
			2016-06-15 13:55:30,964 - selftest - INFO - Running bitbake -e to get BBPATH
			2016-06-15 13:55:32,027 - selftest - INFO - Checking that everything is in order before running the tests
			2016-06-15 13:55:33,073 - selftest - INFO - Running bitbake -p
			2016-06-15 13:55:36,228 - selftest - INFO - Loading tests from: oeqa.selftest.bbtests.BitbakeTests.test_warnings_errors
			2016-06-15 13:55:36,230 - selftest - INFO - Adding: "include selftest.inc" in local.conf
			2016-06-15 13:55:36,230 - selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
			2016-06-15 13:55:36 - test_warnings_errors (oeqa.selftest.bbtests.BitbakeTests) ... ok

			----------------------------------------------------------------------
			Ran 1 test in 0.782s

			OK
			2016-06-15 13:55:37,012 - selftest - INFO - Finished
			2016-06-15 13:55:37,012 - selftest - INFO - Removing the include from local.conf
			2016-06-15 13:55:37,012 - selftest - INFO - Removing the include from bblayers.conf
			</pre>

Benjamin: /* AutoBuilder Adding Automated Tests Guide */

2016-06-15T18:54:29Z

AutoBuilder Adding Automated Tests Guide

New page

==Preparation==

Adding automated tests to the AutoBuilder assumes that the testing is done in either of the following formats:

*[[Oe-selftest|OpenEmbedded Selftest]] format
*[[Image tests|Bitbake Runtime tests]] format

The AutoBuilder interfaces with the test tool via a calling command. Such test calling command is used to start executing a given set of tests. The test set description can come in the way of a configuration (file, variable, argument, etc) and it ultimately explicitly lists which tests are to be run. It is an expectation of the test tool that it procures the execution of the list of tests that has been specified at the moment of calling the starting command.

===Sample Test for Instructions===

For the purpose of this guide, the following test calling command will be used:

<pre>$ oe-selftest -r bbtests.BitbakeTests.test_warnings_errors</pre>

The test tool used is in the OpenEmbedded Selftest format and the calling line selects a single test from the BBTests suite. The reason for this choice is merely simplicity for writing this guide.

You can see the code of this particular test here (referencing the Krogoth release):

http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/selftest/bbtests.py?id=krogoth-15.0.0#n58

There should be about 5 lines of code there, the test can be simply explained as: execute a bitbake build from a file that doesn't exist and verify that it shows an error. It's a negative test.

Getting familiar with the output of the test is a good idea. Preparing the environment for running such test can be seen at the [[Oe-selftest#How_to_use]] section.