Running Toaster Tests with Containers

How to test Toaster with Containers, Two Ways

It had been awhile since I ran the Selenium browser tests on one of my build systems and, once again, I found that something (?) I had upgraded was interfering with the tests. So, this page documents how to use the existing containers to run the tests .

Running the toaster built-in Selenium tests

Note: Currently this only supports the firefox selenium container.

Steps

• Run the Selenium container:

** docker run -it --rm=true -p 5900:5900 -p 4444:4444 --name=selenium selenium/standalone-firefox-debug:2.53.0                                                                                          
*** 5900 is the default vnc port. If you are runing a vnc server on your machine map a different port e.g. -p 6900:5900 . This first number is the port mapping for your host and can be anything you want that isn't in use. The second 5900 is the vnc server port inside the container and cannot be changed. 
*** 4444 is the default selenium sever port. This provides a port for the Django tests to connect to on the host.
** This container can be left running for multiple runs of the Django tests. Once you Ctrl-C out of it, it will be removed.

• Run the tests

** TOASTER_TESTS_BROWSER=http://127.0.0.1:4444/wd/hub TOASTER_TESTS_URL=http://172.17.0.1:8000 ./bitbake/lib/toaster/manage.py test --liveserver=172.17.0.1:8000 tests.browser                            
*** TOASTER_TESTS_BROWSER -This provides the host side address for the Django tests to connect to. If you needed to change the first 4444 in the container step. Make sure you match it here. do not forget the wd/hub or you will get a 403 forbidden error.
*** TOASTER_TESTS_URL - This is the web server the selenium instance inside the continer connects to to perform the tests. The 172.17.0.1 is explained below.
***  --liveserver=xxx tells Django to run the test server on a particular interface and port.  the default is to run it on localhost:8081 which is not reachable by the container so we need to specify. For more information on liveserver see  https://docs.djangoproject.com/en/1.10/topics/testing/tools/
****  These 2addresses  MUST match and they must be reachable from inside and outside the container.  The magic 172.17.0.1 is the default docker0 bridge on linux.  Find it with 
 ip -4 addr show dev docker0
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
  inet 172.17.0.1/16 scope global docker0
  valid_lft forever preferred_lft forever

* if you would like to see the tests run, you can connect to the vnc server as so:
  ** xtightvncviewer 127.0.0.1:5900                                                                                                                                                                      
  ** you need to wait for the test container to come up before this can connect.
  ** if you changed the host side vnc port make sure you sue the same number here.