TipsAndTricks/TestingToasterWithContainers - Revision history

Bavery at 18:40, 4 January 2017

2017-01-04T18:40:55Z

← Older revision		Revision as of 18:40, 4 January 2017
Line 72:		Line 72:
	</code>		</code>
	and logs can be found in the referenced /tmp directory.		and logs can be found in the referenced /tmp directory.
			=== Using the CROPS toaster container test to do a git bisect ===

			* Pending Review.

Bavery at 05:44, 9 November 2016

2016-11-09T05:44:28Z

← Older revision		Revision as of 05:44, 9 November 2016
Line 11:		Line 11:
	** This container can be left running for multiple runs of the Django tests. Once you Ctrl-C out of it, it will be removed.		** This container can be left running for multiple runs of the Django tests. Once you Ctrl-C out of it, it will be removed.
	* Run the tests		* Run the tests
	** <code>TOASTER_TESTS_BROWSER=http://127.0.0.1:4444/wd/hub TOASTER_TESTS_URL=http://172.17.0.1:8000 ./bitbake/lib/toaster/manage.py test --liveserver=172.17.0.1:8000 tests.browser</code>
	** <code> TOASTER_TESTS_BROWSER=remote TOASTER_REMOTE_HUB=http://127.0.0.1:4444/wd/hub ./bitbake/lib/toaster/manage.py test --liveserver=172.17.0.1:8001 tests.browser </code>		** <code> TOASTER_TESTS_BROWSER=remote TOASTER_REMOTE_HUB=http://127.0.0.1:4444/wd/hub ./bitbake/lib/toaster/manage.py test --liveserver=172.17.0.1:8001 tests.browser </code>
	*** TOASTER_TESTS_BROWSER - remote is now an option , like firefox or chrome		*** TOASTER_TESTS_BROWSER - remote is now an option , like firefox or chrome

Bavery at 05:44, 9 November 2016

2016-11-09T05:44:16Z

← Older revision		Revision as of 05:44, 9 November 2016
Line 11:		Line 11:
	** This container can be left running for multiple runs of the Django tests. Once you Ctrl-C out of it, it will be removed.		** This container can be left running for multiple runs of the Django tests. Once you Ctrl-C out of it, it will be removed.
	* Run the tests		* Run the tests
	** <code>TOASTER_TESTS_BROWSER=http://127.0.0.1:4444/wd/hub TOASTER_TESTS_URL=http://172.17.0.1:8000 ./bitbake/lib/toaster/manage.py test --liveserver=172.17.0.1:8000 tests.browser</code>		** <code>TOASTER_TESTS_BROWSER=http://127.0.0.1:4444/wd/hub TOASTER_TESTS_URL=http://172.17.0.1:8000 ./bitbake/lib/toaster/manage.py test --liveserver=172.17.0.1:8000 tests.browser</code>
	*** TOASTER_TESTS_BROWSER -This provides the host side address for the Django tests to connect to. If you needed to change the first 4444 in the container step. Make sure you match it here. do not forget the wd/hub or you will get a 403 forbidden error.		** <code> TOASTER_TESTS_BROWSER=remote TOASTER_REMOTE_HUB=http://127.0.0.1:4444/wd/hub ./bitbake/lib/toaster/manage.py test --liveserver=172.17.0.1:8001 tests.browser </code>
	*** TOASTER_TESTS_URL - This is the web server the selenium instance inside the continer connects to to perform the tests. The 172.17.0.1 is explained below.		*** TOASTER_TESTS_BROWSER - remote is now an option , like firefox or chrome
	*** --liveserver=xxx tells Django to run the test server on a particular interface and port. ~~the~~ default is to run it on localhost:8081 which is not reachable by the container so we need to specify. For more information on liveserver see https://docs.djangoproject.com/en/1.10/topics/testing/tools/		*** TOASTER_REMOTE_HUB - This provides the host side address for the Django tests to connect to. If you needed to change the first 4444 in the container step. Make sure you match it here. do not forget the wd/hub or you will get a 403 forbidden error.
	**** ~~These 2addresses MUST match and they must be reachable from inside and outside the container.~~ The magic 172.17.0.1 is the default docker0 bridge on linux. Find it with		*** --liveserver=xxx tells Django to run the test server on a particular interface and port. The default is to run it on localhost:8081 which is not reachable by the container so we need to specify an address that is reachable by both the host and the container. For more information on liveserver see https://docs.djangoproject.com/en/1.10/topics/testing/tools/
			**** The magic 172.17.0.1 is the default docker0 bridge on linux. Find it with
	<code> ip -4 addr show dev docker0		<code> ip -4 addr show dev docker0
	4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default		4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default

Bavery at 00:01, 8 November 2016

2016-11-08T00:01:12Z

← Older revision		Revision as of 00:01, 8 November 2016
Line 71:		Line 71:
	FAILURE: See logs in /tmp/tmp.QzRu10hIfutoastertest		FAILURE: See logs in /tmp/tmp.QzRu10hIfutoastertest
	</code>		</code>
	and logs can be found in the ~~refenced~~ /tmp directory.		and logs can be found in the referenced /tmp directory.

Bavery at 00:00, 8 November 2016

2016-11-08T00:00:49Z

← Older revision		Revision as of 00:00, 8 November 2016
Line 1:		Line 1:
	=This is a ~~placeholder~~ for a ~~Tips&Tricks Article~~ to ~~Address Using~~ the Toaster ~~Container~~ to ~~Run Tests~~=		= How to test Toaster with Containers, Two Ways =
			It had been awhile since I ran the Selenium browser tests on one of my build systems and, once again, I found that something (?) I had upgraded was interfering with the tests. So, this page documents how to use the existing containers to run the tests .
			== Running the toaster built-in Selenium tests ==
			Note: Currently this only supports the firefox selenium container.

			Steps
			* Run the Selenium container:
			** <code>docker run -it --rm=true -p 5900:5900 -p 4444:4444 --name=selenium selenium/standalone-firefox-debug:2.53.0</code>
			*** 5900 is the default vnc port. If you are runing a vnc server on your machine map a different port e.g. -p 6900:5900 . This first number is the port mapping for your host and can be anything you want that isn't in use. The second 5900 is the vnc server port <strong>inside</strong> the container and cannot be changed.
			*** 4444 is the default selenium sever port. This provides a port for the Django tests to connect to on the host.
			** This container can be left running for multiple runs of the Django tests. Once you Ctrl-C out of it, it will be removed.
			* Run the tests
			** <code>TOASTER_TESTS_BROWSER=http://127.0.0.1:4444/wd/hub TOASTER_TESTS_URL=http://172.17.0.1:8000 ./bitbake/lib/toaster/manage.py test --liveserver=172.17.0.1:8000 tests.browser</code>
			*** TOASTER_TESTS_BROWSER -This provides the host side address for the Django tests to connect to. If you needed to change the first 4444 in the container step. Make sure you match it here. do not forget the wd/hub or you will get a 403 forbidden error.
			*** TOASTER_TESTS_URL - This is the web server the selenium instance inside the continer connects to to perform the tests. The 172.17.0.1 is explained below.
			*** --liveserver=xxx tells Django to run the test server on a particular interface and port. the default is to run it on localhost:8081 which is not reachable by the container so we need to specify. For more information on liveserver see https://docs.djangoproject.com/en/1.10/topics/testing/tools/
			**** These 2addresses MUST match and they must be reachable from inside and outside the container. The magic 172.17.0.1 is the default docker0 bridge on linux. Find it with
			<code> ip -4 addr show dev docker0
			4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
			inet 172.17.0.1/16 scope global docker0
			valid_lft forever preferred_lft forever
			</code>
			* if you would like to see the tests run, you can connect to the vnc server as so:
			** xtightvncviewer 127.0.0.1:5900
			** you need to wait for the test container to come up before this can connect.
			** if you changed the host side vnc port make sure you sue the same number here.
			** <strong> The password is "secret"</strong>. Chosen by the selenium folks...
			== Running the CROPS toaster container tests ==
			The Toaster container located at https://github.com/crops/toaster-container has a smoke test it runs as part of building a new container. This smoketest is run on travis and new docker images get pushed to Dockerhub iff the test passes. This test is slightly different from the above Django test framework tests in that
			* creates a bitbake environment
			* starts up toaster in that environment
			* pulls in information from the layer index
			* creates a new project
			* builds quilt-native

			It is a more limited test but it is end to end, so to speak. You get access to the test by cloning the toaster container repo.
			<code>
			git clone https://github.com/crops/toaster-container.git
			</code>
			Here's how you would run it
			<code>
			IMAGE=crops/toaster-master POKYDIR=/big/src/intel/myBugs/poky POKYBRANCH=COW VNCPORT=5900 ./tests/runtests.sh
			</code>
			The variables:
			* IMAGE=crops/toaster-master - This is the toaster container to use to run the toaster instance being tested. Use one that matches the poky you are testing e.g. toaster-master for master, toaster-krogoth for krogoth, etc.
			* POKYDIR=/home/me/poky - This is where the checkout of poky that you want to test lives on your host system. It will be bind mounted into the container.
			* POKYBRANCH=local - The type of project that gets created to test. on master, it would be a one of master,morty, or local
			* VNCPORT=5900 - If you specify this, you can watch the test run by connecting to the host on port 5900 with a vnc viewer. e.g. <code>xtightvncviewer 127.0.0.1:5900</code>
			** VNCPORT=5900 - This binds the vnc server to 0.0.0.0 and it can be accessed from any machine on the network.
			** VNCPORT=127.0.0.1:5900 - This binds it to localhost and it can be accessed from the machine running the tests only.
			** <strong> The password is "secret"</strong>. Chosen by the selenium folks...
			* runtests.sh - the script that handles the containers and runs the smoketest.py Located in the toaster container repo.

			A successful test looks like:
			<code>
			Running tests...
			smoketests PASSED!
			checkartifacts PASSED!
			ALL TESTS PASSED!
			</code>

			An unsuccessful test looks like
			<code>
			Running tests...
			Traceback (most recent call last):
			File "/big/src/intel/myBugs/toaster-container/tests/smoketests.py", line 76, in <module>
			raise Exception(msg)
			Exception: Error: Branch COW not in options: "local yocto project", "yocto project 2.2 "morty""

			Attempting to save screenshot to /big/src/intel/myBugs/toaster-container/screenshot.png
			FAILURE: See logs in /tmp/tmp.QzRu10hIfutoastertest
			</code>
			and logs can be found in the refenced /tmp directory.

Bavery: Created page with "=This is a placeholder for a Tips&Tricks Article to Address Using the Toaster Container to Run Tests="

2016-08-24T16:42:32Z

Created page with "=This is a placeholder for a Tips&Tricks Article to Address Using the Toaster Container to Run Tests="

New page

=This is a placeholder for a Tips&Tricks Article to Address Using the Toaster Container to Run Tests=