Setting up a production instance of Toaster

From Yocto Project
Jump to navigationJump to search

This page explains how to set up Toaster separately from the hardware running your Yocto Project builds. Do it this way if you want to share a single instance of Toaster across multiple users and build servers.

Setting up a Toaster Instance on a Remote Host

Under normal circumstances, starting Toaster causes three things happen:

  • A BitBake server starts
  • The Toaster UI starts, which connects to the BitBake server on one side and to the SQL database on the other side
  • The web server starts, which reads the database and displays the web user interface

Situations exist, however, where you might want to have multiple instances of Toaster running on various remote machines. You can create this situation by basically modifying how Toaster starts and where the common SQL database resides. You are able to do this because it is not required that Toaster starts the above set of components in order to run. Minimally, an instance of Toaster requires just one of the components to run. Consequently, you are free to manually start as many or few of the components as you need rather than using the Toaster script to cause all three things to happen.

The concepts for setting up multiple instances of Toaster revolve around maintaining a common SQL database and Web server that show data from that common database and then setting up separate instances of BitBake servers and Toaster user interfaces for each separate BitBake build directory. Note that the common SQL database and the Web server show data from all the various BitBake builds. Setting the SQL database outside of any BitBake build directories maintains a separation layer between the various builds.

The database is persistent because the logging database is set up external to the database server (e.g. MySQL). It is not even necessary to run the BitBake servers, the SQL server, and the Web server on the same machine. Each component can be run on its own machine.

Steps to get set up

1. Set up the SQL Logging Server and the Web Server

You can use any SQL server out of the box (e.g. apt-get install mysgl-server works for an Ubuntu system). If you are concerned about performance, you might want to hand-tune the server. You must set up proper username and password access for the server. You need administration rights for the mysql root account. Realize that this is not the same thing as root access on the machine.

Clone a separate, local Git repository of the Toaster master branch to use for running the Web server. You do not perform builds on this tree. You need to create this local repository away from any build areas.

In the separately cloned tree for the Web server, edit the bitbake/lib/toaster/toastermain/settings.py file so that the DATABASES value points to the previously created database server. Use the username and password you established earlier.

Run the database sync scripts to create the needed tables as follows: 

    $ python bitbake/lib/toaster/manage.py syncdb
    $ python bitbake/lib/toaster/manage.py migrate orm

Start the Web server using the following command: 

    $ python bitbake/lib/toaster/manage.py runserver

... or in the background: 

    $ nohup python bitbake/lib/toaster/manage.py runserver 2>toaster_web.log >toaster_web.log &

2. Enable build logging to the common SQL server for each build directory you are using

Edit _build local_ bitbake/lib/toaster/toastermain/settings.py to alter the DATABASES value to point to the common SQL logging server.

Create the required conf/toaster.conf file as per Bitbake extra options on "https://wiki.yoctoproject.org/wiki/Setting_up_a_local_instance_of_Toaster".

Start the BitBake server using the following command: 

    $ bitbake --postread conf/toaster.conf --server-only -t xmlrpc -B localhost:0 && export BBSERVER=localhost:-1

Start the logging user interface using the following command: 

    $ nohup bitbake --observe-only -u toasterui &

NOTE: No hard-coded ports are used as there is enough code to run autodiscovery for ports to prevent collisions.

At this point, you are ready to run your builds using commands such as: 

    $ bitbake core-image-minimal

When you are finished, you need to kill the BitBake server for that particular build area: 

    $ bitbake -m

3. Verify that it all works

You should examine the logs and be sure that the logging worked, that data is persistent, and that data from multiple builds from different areas was supported.

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Setting_up_a_production_instance_of_Toaster&oldid=12916"