|
|
| (59 intermediate revisions by 7 users not shown) |
| Line 1: |
Line 1: |
| [[Category:Toaster]] | | [[Category:Toaster]] |
| [[Toaster]] is a Web-based interface to the Bitbake build system and the Poky distribution inside the Yocto Project. | | [[Toaster]] is a web-based interface to OpenEmbedded and BitBake. |
| This project was formely known as Web Hob / Webhob / webhob, and you may still find references to the old name in the documentation.
| | [[File:Screenshot toaster.png|thumb|Screenshot of Toaster 2.1]] |
| | |
| General discussion about '''Toaster''' happens on a dedicated mailing list: [https://lists.yoctoproject.org/listinfo/toaster https://lists.yoctoproject.org/listinfo/toaster] | | General discussion about '''Toaster''' happens on a dedicated mailing list: [https://lists.yoctoproject.org/listinfo/toaster https://lists.yoctoproject.org/listinfo/toaster] |
|
| |
|
| Project planning is available below, and the design documents we follow.
| | === Using Toaster === |
| | |
| == User interface ==
| |
| The design of the user interface takes place in iterations, and most recent designs supersede older ones.
| |
| | |
| The latest iteration is always available at: http://www.yoctoproject.org/toaster
| |
| | |
| The document [[File:Toaster_1.6_content_structure.pdf]] shows the content structure of Toaster in the 1.6 release.
| |
| | |
| === Other design documentation ===
| |
| * [[Visualisations]] - an inventory of data visualisations we aim to include in the first release of Toaster
| |
| * [[File:WH_roadmap.pdf]] - the roadmap for Toaster development produced by the London-based agency Tobias&Tobias
| |
| * [[File:Multiuser_support_in_Web_Hob.pdf]] - different design approaches to handle multi-user scenarios
| |
| * [[File:Web_Hob_content_structure.pdf]] - Toaster information architecture, as it came out from the design project with Tobias&Tobias
| |
| | |
| === Old design documents ===
| |
| * [[Web_Hob_design_project_with_T%26T]] - The outcome of the design project with the external agency Tobias&Tobias
| |
| * [[Yocto Web Hob Design 0.0 — Archived|A preliminary design project]] by [http://www.jimkosem.com/ Jim Kosem]
| |
| | |
| == Architecture and component design ==
| |
| | |
| | |
| We are planning the capabilities of [[Toaster]] based on evolutive stages of implementation, using community feedback on each stage to plan, design and implement a set of capabilities for the next stage.
| |
| In the first stage, synchronized with the planning of [https://www.yoctoproject.org/download/toaster-web-interface Yocto Project 1.5 release], we implemented the back end of a build analysis / image inspection module.
| |
| As part of the Yocto Project 1.6 release, we are building a web-based graphical user interface for the build analysis / image inspection module.
| |
| | |
| [[Toaster]] is designed as a collection of components that will run independently performing isolated functions. The interfaces between components are documented on this wiki as to ease interoperability with newer components. From design phase, we've taken care to account for further expansion needs, and account for scalability problems.
| |
| | |
| * [[ Data store investigation results and choices ]]
| |
| | |
| * [[ Django models ]]
| |
| | |
| * [[ REST API Contracts ]]
| |
| | |
| * [[Bugzilla feature list]]
| |
| | |
| == Installation and Running ==
| |
| | |
| Before you start, make sure you install the [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#intro-requirements packages required by the Yocto Project]
| |
| | |
| === Using the Yocto Project 1.6 "Daisy" release or the master branch === | |
| | |
| ==== You need this stuff ready ====
| |
| | |
| * Django 1.5. The easiest way is to install it system wide [http://pip.readthedocs.org/en/latest/installing.html with pip]
| |
| | |
| $ sudo pip install django==1.5
| |
| | |
| Alternatively, you can use
| |
| | |
| $ pip install --user django==1.5
| |
| | |
| rather than <code>sudo pip</code>, to install it to your user site-packages directory under <code>~/.local</code>, which won't impact the rest of the system.
| |
| | |
| * South 0.8.4
| |
| | |
| $ sudo pip install South==0.8.4
| |
| | |
| * Make sure that port 8000 and 8200 are free, i.e. no servers on them
| |
| | |
| ==== To get it up and running ====
| |
| | |
| * clone the master tree: http://git.yoctoproject.org/git/poky
| |
| * set up a build as normal: <code> source poky/oe-init-build-env </code>
| |
| * at this point edit <code>local.conf</code>, or layers, etc.
| |
| * start Toaster system: <code> source toaster start </code>
| |
| * run builds normally: <code> bitbake core-image-minimal </code>
| |
| * to see the web interface: <code> xdg-open http://localhost:8000/ </code>
| |
| * to stop Toaster: <code> source toaster stop </code>
| |
| | |
| === Using the Dora + Toaster release (part of Yocto Project 1.5) ===
| |
| | |
| ==== You need this stuff ready ====
| |
| | |
| * Django 1.4.5. The easiest way is to install it system wide [http://pip.readthedocs.org/en/latest/installing.html with pip]
| |
| | |
| $ sudo pip install django==1.4.5
| |
| | |
| * Make sure that port 8000 and 8200 are free, i.e. no servers on them
| |
| | |
| ==== To get it up and running ====
| |
| | |
| * checkout the Dora + Toaster release tree: http://git.yoctoproject.org/cgit/cgit.cgi/poky/log/?h=dora-toaster
| |
| * set up a build as normal: <code> source poky/oe-init-build-env </code>
| |
| * at this point edit <code>local.conf</code>, or layers, etc.
| |
| * start Toaster system: <code> source toaster start </code>
| |
| * run builds normally: <code> bitbake core-image-minimal </code>
| |
| * to see the web interface: <code> xdg-open http://localhost:8000/ </code>
| |
| * to stop Toaster: <code> source toaster stop </code>
| |
| | |
| ===Bitbake extra options ===
| |
| | |
| This is an overview of extra options needed for Bitbake. These should be automatically added by Toaster
| |
| in a conf/toaster.conf file that is enabled when the system starts through the toaster command.
| |
| | |
| The gritty details are here:
| |
| | |
| [[Toaster]] needs the '''toaster''' bbclass enabled in Bitbake in order to record target image package information.
| |
| <code> INHERIT += "toaster" </code>
| |
| | |
| This enables the '''toaster''' bbclass in Bitbake, needed to record target image package information.
| |
| | |
| <code> INHERIT += "buildhistory" </code>
| |
| <br />
| |
| <code> BUILDHISTORY_COMMIT = "1" </code>
| |
| | |
| This enables '''buildhistory''' in Bitbake, needed to record target image package information.
| |
| This last command is not strictly needed, yet without it '''buildhistory''' will not work right in it's own right.
| |
| | |
| ==== A bunch of files are created and used under the build/ directory: ====
| |
| | |
| * toaster.sqlite - the database file
| |
| * toaster_web.log - the log file of the web server
| |
| * toaster_ui.log - the log file of the ui component
| |
| * .toastermain.pid - contains the pid of the web server
| |
| * .toasterui.pid - contains the pid of the DSI data bridge
| |
| * bitbake-cookerdaemon.log - the log file of the bitbake server
| |
| | |
| === Deleting a Build from the Toaster Database ===
| |
| | |
| By default, Toaster stores all build data in a <code>toaster.sqlite</code> database file located in the [http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#build-directory Build Directory].
| |
| | |
| You can delete builds from the Toaster database using two bespoke commands (<code>buildslist</code> and <code>builddelete</code>) added to
| |
| | |
| <code>/poky/bitbake/lib/toaster/manage.py</code>
| |
| | |
| ==== Follow these steps to delete a build: ====
| |
| | |
| <strong>1. Identify the build:</strong> Run the following command from your Build Directory to display a user-readable, tabbed list of build IDs and identification information:
| |
| | |
| $ python ../bitbake/lib/toaster/manage.py buildslist
| |
| | |
| <strong>2. Delete a specific build:</strong> Using the build ID displayed in the previous step, run the following command to delete a specific build:
| |
| | |
| $ python ../bitbake/lib/toaster/manage.py builddelete <build_id>
| |
| | |
| The build and all related information is deleted from the Toaster database. The deletion operation traverses the ForeignKey relationships recursively.
| |
| | |
| When using the <code>buildslist</code> and <code>builddelete</code> commands, you need to consider the following:
| |
| | |
| * For default configurations, your working directory must be the Build Directory. Using these command from the Build Directory allows Toaster to find the <code>toaster.sqlite</code> file, which is located in the Build Directory.
| |
| | |
| * For other database configurations, it is possible that you can use these commands from a directory other than the Build directory. To do so, the <code>toastermain/settings.py</code> file must be configured to point to the correct database backend.
| |
| | |
| It is recommended to always call <code>manage.py</code> from the Build Directory.
| |
| | |
| === Setting up a Toaster Instance on a Remote Host ===
| |
| | |
| Under normal circumstances, starting Toaster causes three things happen:
| |
| | |
| * A BitBake server starts
| |
| | |
| * The Toaster UI starts, which connects to the BitBake server on one side and to the SQL database on the other side
| |
| | |
| * The web server starts, which reads the database and displays the web user interface
| |
| | |
| Situations exist, however, where you might want to have multiple instances of Toaster running on various remote machines. You can create this situation by basically modifying how Toaster starts and where the common SQL database resides. You are able to do this because it is not required that Toaster starts the above set of components in order to run. Minimally, an instance of Toaster requires just one of the components to run. Consequently, you are free to manually start as many or few of the components as you need rather than using the Toaster script to cause all three things to happen.
| |
| | |
| The concepts for setting up multiple instances of Toaster revolve around maintaining a common SQL database and Web server that show data from that common database and then setting up separate instances of BitBake servers and Toaster user interfaces for each separate BitBake build directory. Note that the common SQL database and the Web server show data from all the various BitBake builds. Setting the SQL database outside of any BitBake build directories maintains a
| |
| separation layer between the various builds.
| |
| | |
| The database is persistent because the logging database is set up external to the database server (e.g. MySQL). It is not even necessary to run the BitBake servers, the SQL server, and the Web server on the same machine. Each component can be run on its own machine.
| |
| | |
| ==== Here are the steps to get set up: ====
| |
| | |
| ===== 1. Set up the SQL Logging Server and the Web Server =====
| |
| | |
| You can use any SQL server out of the box (e.g. <code>apt-get install mysgl-server</code> works for an Ubuntu system). If you are concerned about performance, you might want to hand-tune the server. You must set up proper username and password access for the server. You need administration rights for the mysql root account. Realize that this is not the same thing as root access on the machine.
| |
| | |
| Clone a separate, local Git repository of the Toaster master branch to use for running the Web server. You do not perform builds on this tree. You need to create this local repository away from any build areas.
| |
| | |
| In the separately cloned tree for the Web server, edit the <code>bitbake/lib/toaster/toastermain/settings.py</code> file so that the <code>DATABASES</code> value points to the previously created database server. Use the username and password you established earlier.
| |
| | |
| Run the database sync scripts to create the needed tables as follows:
| |
| | |
| $ python bitbake/lib/toaster/manage.py syncdb
| |
| $ python bitbake/lib/toaster/manage.py migrate orm
| |
| | |
| Start the Web server using the following command:
| |
| | |
| $ python bitbake/lib/toaster/manage.py runserver
| |
| | |
| ===== 2. Enable build logging to the common SQL server for each build directory you are using =====
| |
| | |
| Edit <code>_build local_ bitbake/lib/toaster/toastermain/settings.py</code> to alter the <code>DATABASES</code> value to point to the common SQL logging server.
| |
| | |
| Start the BitBake server using the following command:
| |
| | |
| $ bitbake --postread conf/toaster.conf --server-only -t xmlrpc -B localhost:0 && export BBSERVER=localhost:-1
| |
|
| |
|
| Start the logging user interface using the following command:
| | Toaster can run in various modes and setups. |
|
| |
|
| $ nohup bitbake --observer-only -u toasterui &
| | * '''Local mode''' - in this mode Toaster is setup for use as a local development tool. It can be used to configure builds or just as a receiver for builds done on the command line with bitbake. You can launch it like this: |
|
| |
|
| <strong>NOTE:</strong> No hard-coded ports are used as there is enough code to run autodiscovery for ports to prevent collisions.
| | $ source oe-init-build-env |
| | $ source toaster start |
|
| |
|
| At this point, you are ready to run your builds using commands such as:
| | You then navigate to the link in your browser (e.g. http://localhost:8000) and configure a project. Or start building in the normal way with bitbake via the command line. Toaster will automatically pick up the builds and you will be able to see them on the build dashboard in your browser. |
|
| |
|
| $ bitbake core-image-minimal
| | * '''Production mode''' - All the same functionality as the local mode but with the web server setup as a shared service for multiple developers to use, this sets up Toaster as a wsgi application and [[Setting up a production instance of Toaster|requires additional configuration]]. |
|
| |
|
| When you are finished, you need to kill the BitBake server for that particular build area:
| | === Toaster How-to's === |
|
| |
|
| $ bitbake -m
| | Specific pages with Toaster how-tos are available below. |
| | |
| ===== 3. Verify that it all works =====
| |
| | |
| You should examine the logs and be sure that the logging worked, that data is persistent, and that data from multiple builds from different areas was supported.
| |
| | |
| == Testing ==
| |
| Documentation and results related to [[Toaster]] Quality Assurance.
| |
| | |
| * [[ Toaster testing plan ]]
| |
| | |
| == Contributing to Toaster ==
| |
| Toaster is a community effort and welcomes your contribution.
| |
|
| |
|
| * [[Contribute to Toaster]] | | * [[Contribute to Toaster]] |
| | * [[Testing Toaster]] |
| | * [[Setting up a local instance of Toaster]] |
| | * [[Setting up a production instance of Toaster]] - documentation for Interactive mode |
| | * [https://docs.yoctoproject.org/dunfell/toaster-manual/toaster-manual-setup-and-use.html#using-the-toaster-web-interface How to use the Toaster web interface] |
| | * [[How to delete information from the Toaster database]] |
| | * [[How to support permission management in Build Mode for Toaster]] |
| | * [[ToasterTable]]: Toaster's internal API for adding sortable, filterable, paged tables |
|
| |
|
| == Old page content == | | === About Toaster === |
| | | [[File:Screenshot toaster analyis.png|thumb|Analysis of builds using Toaster]] |
| This page is about the Web Hob project. Web Hob is a web-based interface to the Yocto Project.
| |
| | |
| General discussion about '''Web Hob''' happens on a dedicated mailing list: [https://lists.yoctoproject.org/listinfo/webhob https://lists.yoctoproject.org/listinfo/webhob]
| |
| | |
| There have been 2 main pieces of work related to Web Hob so far:
| |
| | |
| * [[Web Hob design project with T&T|A design project with the London-based agency Tobias & Tobias]]
| |
| * [[Yocto Web Hob Design 0.0 — Archived|A preliminary design project]] by [http://www.jimkosem.com/ Jim Kosem]
| |
| | |
| === Web Hob information architecture === | |
| | |
| This document represents the content structure of the Web Hob application.
| |
| | |
| [[File:Web_Hob_content_structure.pdf]] | |
| | |
| === Different approaches to multi-user workflows ===
| |
|
| |
|
| This document outlines the different approaches we have uncovered so far to facilitate multi-user and team work with Web Hob.
| | * [[File:Working_with_design.pdf]] |
| | * [https://bugzilla.yoctoproject.org/buglist.cgi?list_id=213820&columnlist=status_whiteboard%2Cassigned_to%2Ctarget_milestone%2Cbug_status%2Cshort_desc%2Cbug_severity%2Cpriority&classification=Build%20System%20%26%20Metadata&query_based_on=Toaster-Opens&query_format=advanced&bug_status=NEW&bug_status=ACCEPTED&bug_status=IN%20PROGRESS%20DESIGN&bug_status=IN%20PROGRESS%20DESIGN%20COMPLETE&bug_status=IN%20PROGRESS%20IMPLEMENTATION&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=WaitForUpstream&component=toaster&product=Toaster&known_name=Toaster-Opens Bug list] |
| | * [[Toaster architecture design]] |
| | * [[Toaster and bitbake communications]] |
| | * [[Toaster database]] |
| | * [[Toaster testing plan]] |
|
| |
|
| * [[File:Multiuser_support_in_Web_Hob.pdf]]
| | === In progress documentation === |
|
| |
|
| === Visualisations index ===
| | We are currently preparing the documentation for the Toaster build functionality. The content here is just a brain dump of what we need to cover (in no particular order). Feel free to add and create content as you see fit: |
|
| |
|
| The build analysis functionality in Web Hob will include several graphical presentations of build data. The first step to design them is [[visualisations |listing them all]].
| | *[[Using virtualenv]] |
| | *[[Setting up a production instance of Toaster]] |
| | *[[manage.py commands]] - this should include an explanation of lsupdates |
| | *[[Start Toaster in managed mode]] |
