Toaster architecture design

Introduction

Toaster project is located in bitbake/lib/toaster.

Toaster is built on the Django framework, if you're not familiar with Django framework, it's very well documented

The basic layout of the framework is that for each Django application A URL defined in urls.py, maps to a view defined in views.py, the urls and views have a 1:1 mapping to a function or View class in the views.py. These functions/Views runs the server side logic and return the data either to render a Django template which can be found in the templates directory or alternatively they can return whatever http response is needed (e.g. a JSON document).

Toaster stores a lot of information about the configuration of projects, layers and recipes, this is stored in Toaster database which is fully abstracted via Django's object relation mapping (orm). This means that we don't have to get our hands dirty with making database connections, caring which kind of database is chosen or mapping data in and out of the database. Toaster's database schema is defined in the models.py of each of the Django applications that makes up Toaster.

├── bldcollector
│   ├── admin.py
│   ├── __init__.py
│   ├── urls.py
│   └── views.py
├── bldcontrol
│   ├── admin.py
│   ├── bbcontroller.py
│   ├── __init__.py
│   ├── localhostbecontroller.py
│   ├── management
│   ├── migrations
│   ├── models.py
│   ├── tests.py
│   └── views.py
├── __init__.py
├── manage.py
├── orm
│   ├── __init__.py
│   ├── management
│   ├── migrations
│   ├── models.py
│   └── tests.py
├── toastergui
│   ├── fixtures
│   ├── __init__.py
│   ├── static
│   ├── tablefilter.py
│   ├── tables.py
│   ├── templates
│   ├── templatetags
│   ├── tests.py
│   ├── typeaheads.py
│   ├── urls.py
│   ├── views.py
│   └── widgets.py
└── toastermain
    ├── __init__.py
    ├── management
    ├── settings.py
    ├── urls.py
    └── wsgi.py

Toaster's Django applications

toastergui

This is where Toaster's UI related processing happens, the urls.py ,views.py are where the bulk of this happens. All the web pages you see in toaster are processed via this application. As Toaster utilises tables to display information we have a special view class that deals directly with this, cunningly called ToasterTables, this and a few others of our common 'widgets' can be found in widgets.py.

orm

This is the main database structure which holds the configuration data and the data that is collected from running a build. It also includes some utility and convenience functions on data objects returned. It is just used to define and share the ORM between the other Django applications and doesn't serve any web pages.

bldcontrol

This is another Django application that is used just to utilise the Django framework rather than actually serve web pages. This module is concerned with the task of polling to see if there is a build scheduled via the runbuilds.py command and launching the build process, which includes commanding bitbake and doing any Git cloning / fetching layers if needed via the localhostbecontroller (we used to have multiple controller implementations), there are a number of additional tables defined to support build creation and build control specific tasks such as a BuildRequest and BuildEnvironment.

bldcollector

This provides an interface for the build data to be manually uploaded to the orm, it is currently not in development.

toastermain

This is the overarching/main Django application which loads the 4 above. It is where the default settings for Toaster are.

Other

manage.py

Django framework script to manage the framework, includes commands to run unit tests, do database setup, migrations as well as running the development server process.

bitbake/bin/toaster

Shell script to run if running Toaster in standalone mode on your local system this sets up Toaster's environment, loads the default config, runs the database migrations if needed and grabs the data from the layerindex to bootstrap Toaster, it starts the webserver and the runbuilds poller process.

toastermain/wsgi.py

Django framework script that is the entry point to running the webserver as a wsgi application or as we call it production setup

Integration with bitbake

bitbake/lib/bb/ui/toasterui.py and buildinfohelper.py

These run in the bitbake observer process and are responsible for converting the events generated by bitbake by a build into objects in Toaster's database.

openembedded-core meta/classes/toaster.bbclass

The class that extends bitbake in order to generate build events that toaster needs (loaded via INHERIT += "toaster" )