Testing Toaster - Revision history

Alexandru.c.georgescu at 11:52, 15 March 2016

2016-03-15T11:52:37Z

← Older revision		Revision as of 11:52, 15 March 2016
Line 1:		Line 1:
			<span style="color: red"> This page is outdated. Please refer to [[Toaster testing plan]] page for testing Toaster approach.</span>


	[[Category:Toaster]]		[[Category:Toaster]]

Bavery: /* The selenium automated UI tests */

2015-09-18T06:16:41Z

The selenium automated UI tests

← Older revision		Revision as of 06:16, 18 September 2015
Line 71:		Line 71:
	./bitbake/lib/toaster/contrib/tts/toasteruitest/run_toastertests.py		./bitbake/lib/toaster/contrib/tts/toasteruitest/run_toastertests.py

	The above command takes the list of the tests to run from the toaster_test.cfg file. The tests themselves are located in toaster_automation_test.py.		The above command takes the list of the tests to run from the toaster_test.cfg file. The tests themselves are located in toaster_automation_test.py. The toaster_test.cfg file has a couple of important variables in it separated by which os you are running the tests on. These varables control where the toaster instance is located (defaults to http://127.0.0.1:8000) what browser to use (firefox is safest for selenium though there is a chrome driver available as well), what tests to run and what log level to save data at.

	In order for these tests to run properly, the following steps need to be taken before running, on a clean toaster setup:		In order for these tests to run properly, the following steps need to be taken before running, on a clean toaster setup:

Bavery: /* The selenium automated UI tests */

2015-09-18T06:06:33Z

The selenium automated UI tests

← Older revision		Revision as of 06:06, 18 September 2015
Line 51:		Line 51:
	The automated UI tests are targeted at validating and identifying regressions in the UI pages. These tests are based on the Selenium framework to drive interactions in the interface.		The automated UI tests are targeted at validating and identifying regressions in the UI pages. These tests are based on the Selenium framework to drive interactions in the interface.

	To ~~run~~ the selenium tests~~, please use~~:		The selenium tests require a couple of components:
			*scrot (a screen capture utility) and pip - a utility for installing python modules. The following is how you install it for ubuntu:
			** sudo apt-get install scrot python-pip python-virtualenv
			** virtualenv t-selenium
			** source ./t-selenium/bin/activate
			** pip install selenium
			* Since this is a web application, the UI can also be tested from a mac. To install the requirements there do:
			** sudo easy_install pip
			** sudo pip install virtualenv
			** virtualenv t-selenium
			** source ./t-selenium/bin/activate
			* also install the latest filefox extension from selenium located at: http://release.seleniumhq.org/selenium-ide/ and install all the extensions.

			The selenium tests are located in:

	./ bitbake/lib/toaster/contrib/tts/toasteruitest		./ bitbake/lib/toaster/contrib/tts/toasteruitest

			You run them with:
	./bitbake/lib/toaster/contrib/tts/toasteruitest/run_toastertests.py		./bitbake/lib/toaster/contrib/tts/toasteruitest/run_toastertests.py

			The above command takes the list of the tests to run from the toaster_test.cfg file. The tests themselves are located in toaster_automation_test.py.

	In order for these tests to run properly, the following steps need to be taken before running, on a clean toaster setup:		In order for these tests to run properly, the following steps need to be taken before running, on a clean toaster setup:

Mihail Stanciu: /* The oe-selftest toaster tests */

2015-08-21T11:57:27Z

The oe-selftest toaster tests

← Older revision		Revision as of 11:57, 21 August 2015
Line 39:		Line 39:
	source oe-init-build-env		source oe-init-build-env

	then modify the bblayers.conf file to add the meta-selftest layer and then run the command:		then modify the bblayers.conf file to add the meta-selftest layer, create a symbolic link to the toaster.sqlite file:

			ln -s ../toaster.sqlite toaster.sqlite

			and then run the command:

	oe-selftest --run-tests _toaster		oe-selftest --run-tests _toaster

Mihail Stanciu: /* The oe-selftest toaster tests */

2015-08-21T09:58:47Z

The oe-selftest toaster tests

← Older revision		Revision as of 09:58, 21 August 2015
Line 39:		Line 39:
	source oe-init-build-env		source oe-init-build-env

	and then run the command:		then modify the bblayers.conf file to add the meta-selftest layer and then run the command:

	oe-selftest --run-tests _toaster		oe-selftest --run-tests _toaster

Mihail Stanciu: /* The selenium automated UI tests */

2015-08-20T15:24:20Z

The selenium automated UI tests

← Older revision		Revision as of 15:24, 20 August 2015
Line 52:		Line 52:

	./bitbake/lib/toaster/contrib/tts/toasteruitest/run_toastertests.py		./bitbake/lib/toaster/contrib/tts/toasteruitest/run_toastertests.py


			In order for these tests to run properly, the following steps need to be taken before running, on a clean toaster setup:
			*Create new project named "selenium-project" using local poky release
			*Create 1 core-image-minimal image
			*Break a recipe file (by modifying a link, for example)
			*Create a new image using broken recipe to check error detection is working
			*Fix the recipe broken in previous step
			*Create 1 core-image-sato image

			In order to run all current tests, you need to modify the configuration file "toaster_test.cfg" with the following line:

			test_cases = [901, 902, 903, 904, 906, 910, 911, 912, 913, 914, 915, 916, 923, 924, 940, 941, 942, 943, 944, 945, 946, 947, 948, 949, 950, 951, 955, 956]

Mihail Stanciu: /* The oe-selftest toaster tests */

2015-08-20T15:11:53Z

The oe-selftest toaster tests

← Older revision		Revision as of 15:11, 20 August 2015
Line 35:		Line 35:
	* Create a new image using broken recipe to check error detection is working		* Create a new image using broken recipe to check error detection is working

	To run these tests you need to source an environment and then run the command "oe-selftest --run-tests _toaster"		To run these tests you need to source an environment by issuing:

			source oe-init-build-env

			and then run the command:

			oe-selftest --run-tests _toaster

	== The selenium automated UI tests ==		== The selenium automated UI tests ==

Mihail Stanciu: /* The oe-selftest toaster tests */

2015-08-20T15:09:29Z

The oe-selftest toaster tests

← Older revision		Revision as of 15:09, 20 August 2015
Line 28:		Line 28:
	These tests are verifying the correctness of the data gathered by the build process - it tests the bitbake/lib/bb/ui/toasterui.py and bitbake/lib/bb/ui/buildinfohelper.py files and the interface to the bitbake server process.		These tests are verifying the correctness of the data gathered by the build process - it tests the bitbake/lib/bb/ui/toasterui.py and bitbake/lib/bb/ui/buildinfohelper.py files and the interface to the bitbake server process.

			In order for these tests to run properly, the following steps need to be taken before running, on a clean toaster setup:
			* Create new project using local poky release
			* Create 1 core-image-minimal image
			* Create 1 core-image-sato image to check building from cache
			* Break a recipe file (by modifying a link, for example)
			* Create a new image using broken recipe to check error detection is working

			To run these tests you need to source an environment and then run the command "oe-selftest --run-tests _toaster"

	== The selenium automated UI tests ==		== The selenium automated UI tests ==

Ddalex: Created page with "Category:Toaster Toaster has 4 testing suites, targeted at verifying different parts of the system. This document briefly describes each testing suite. == Django Unit T..."

2015-08-19T09:52:56Z

Created page with "Category:Toaster Toaster has 4 testing suites, targeted at verifying different parts of the system. This document briefly describes each testing suite. == Django Unit T..."

New page

[[Category:Toaster]]

Toaster has 4 testing suites, targeted at verifying different parts of the system. This document briefly describes each testing suite.

== Django Unit Tests ==

Toaster is primarely a Django application. As such, it makes use of the Django test system that runs unit tests on mock data to ensure pieces of functionality work as expected, and prevent regressions.

To run the django unit tests, invoke them as with any Django test suite

./bitbake/lib/toaster/manage.py test

To add unit tests, simply add needed tests to the _tests.py_ file in the module you're editing.

== Toaster Test System (TTS) ==

The TTS is a suite aimed at running integration and acceptance tests. Currently, it verifying the correct syntax and form of the Python code, tries to start Toaster in both modes, and runs HTML5 validation on HTML-returning pages.

The tests are designed to be triggered through a continuous integration system.

To start manually it, run the tts test runner:

./bitbake/lib/toaster/contrib/tts/runner.py

== The oe-selftest toaster tests ==

These tests are verifying the correctness of the data gathered by the build process - it tests the bitbake/lib/bb/ui/toasterui.py and bitbake/lib/bb/ui/buildinfohelper.py files and the interface to the bitbake server process.

== The selenium automated UI tests ==

The automated UI tests are targeted at validating and identifying regressions in the UI pages. These tests are based on the Selenium framework to drive interactions in the interface.

To run the selenium tests, please use:

./ bitbake/lib/toaster/contrib/tts/toasteruitest

./bitbake/lib/toaster/contrib/tts/toasteruitest/run_toastertests.py