Yocto Project - User contributions [en]

Testing Toaster

2015-08-19T09:52:56Z

Ddalex: Created page with "Category:Toaster Toaster has 4 testing suites, targeted at verifying different parts of the system. This document briefly describes each testing suite. == Django Unit T..."

[[Category:Toaster]]

Toaster has 4 testing suites, targeted at verifying different parts of the system. This document briefly describes each testing suite.

== Django Unit Tests ==

Toaster is primarely a Django application. As such, it makes use of the Django test system that runs unit tests on mock data to ensure pieces of functionality work as expected, and prevent regressions.

To run the django unit tests, invoke them as with any Django test suite

./bitbake/lib/toaster/manage.py test

To add unit tests, simply add needed tests to the _tests.py_ file in the module you're editing.

== Toaster Test System (TTS) ==

The TTS is a suite aimed at running integration and acceptance tests. Currently, it verifying the correct syntax and form of the Python code, tries to start Toaster in both modes, and runs HTML5 validation on HTML-returning pages.

The tests are designed to be triggered through a continuous integration system.

To start manually it, run the tts test runner:

./bitbake/lib/toaster/contrib/tts/runner.py

== The oe-selftest toaster tests ==

These tests are verifying the correctness of the data gathered by the build process - it tests the bitbake/lib/bb/ui/toasterui.py and bitbake/lib/bb/ui/buildinfohelper.py files and the interface to the bitbake server process.

== The selenium automated UI tests ==

The automated UI tests are targeted at validating and identifying regressions in the UI pages. These tests are based on the Selenium framework to drive interactions in the interface.

To run the selenium tests, please use:

./ bitbake/lib/toaster/contrib/tts/toasteruitest

./bitbake/lib/toaster/contrib/tts/toasteruitest/run_toastertests.py

Toaster

2015-08-19T09:17:10Z

Ddalex: /* Toaster How-to's */

[[Category:Toaster]]
[[Toaster]] is a web-based interface to OpenEmbedded and BitBake. You might have heard it is a replacement for [https://www.yoctoproject.org/documentation/hob-manual Hob]. The answer is "it will be at some point, but not yet".

General discussion about '''Toaster''' happens on a dedicated mailing list: [https://lists.yoctoproject.org/listinfo/toaster https://lists.yoctoproject.org/listinfo/toaster]

=== Using Toaster ===

Toaster can run in various modes and setups. To ease out the understanding of documentation, we review here the terminology used throughout the documentation.

* Interactive mode - this is the mode released with Yocto Project Release 1.6. Functional components consist of a build recording component, 'toasterui', and a build stats and inspection user interface, the 'toastergui'. It is started with the command sequence listed below, and the builds are started using normal bitbake command line

$ source oe-init-build-env
$ source toaster start

* Managed mode - in this mode Toaster handles the build configuration GUI (through Project pages), and build scheduling and execution, in addition to the features launched with Yocto Project Release 1.6. The builds are triggered through the web interface,
the user as no direct access to the bitbake command line.

** Local managed mode, in short _local_, is the out-of-box mode available after a poky/ checkout. It allows the user to perform build on his local machine source code, with a local build directory, and is intended to help the user discover the interface, and configure and run local builds. You can launch it with

$ ./bitbake/bin/toaster

** Remote managed mode, or [[hosted Toaster]], is intended to be the production setup for running Toaster in organizations supporting multiple users and using customized Toaster installations.

=== Toaster How-to's ===

Specific pages with Toaster how-tos are available below.

* [[Contribute to Toaster]]
* [[Testing Toaster]]
* [[Setting up a local instance of Toaster]]
* [[Setting up a production instance of Toaster]] - documentation for Interactive mode
* [[Setting up a hosted managed mode for Toaster]] - configure and run build through Toaster itself, providing a service for your users
* [https://www.yoctoproject.org/documentation/toaster-manual-17 How to use the Toaster web interface]
* [[How to delete information from the Toaster database]]

=== About Toaster ===

* [[File:Working_with_design.pdf]]
* [https://bugzilla.yoctoproject.org/buglist.cgi?list_id=213820&columnlist=status_whiteboard%2Cassigned_to%2Ctarget_milestone%2Cbug_status%2Cshort_desc%2Cbug_severity%2Cpriority&classification=Build%20System%20%26%20Metadata&query_based_on=Toaster-Opens&query_format=advanced&bug_status=NEW&bug_status=ACCEPTED&bug_status=IN%20PROGRESS%20DESIGN&bug_status=IN%20PROGRESS%20DESIGN%20COMPLETE&bug_status=IN%20PROGRESS%20IMPLEMENTATION&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=WaitForUpstream&component=toaster&product=Toaster&known_name=Toaster-Opens Bug list]
* [[Toaster architecture design]]
* [[Toaster testing plan]]
* [[Toaster 1.9 design | Scope and design (1.9)]]
* [[Toaster 1.7 design | Scope and design (1.7 - 1.8)]]
* [[Toaster archive | Archive]]

=== In progress documentation ===

We are currently preparing the documentation for the Toaster build functionality. The content here is just a brain dump of what we need to cover (in no particular order). Feel free to add and create content as you see fit:

*[[Using virtualenv]]
*[[Toaster configuration]] - explain releases, layer sources, BitBake versions and default project configurations
*[[Set up Toaster locally]] - explain the set up process and the local version of the localconf.json file
*[[Set up production instance]] - explain the Django admin interface and the remove version of the localconf.json file
*[[manage.py commands]] - this should include an explanation of lsupdates
*[[Custom layer index]] - explain how to create your own layer index
*[[Start Toaster in managed mode]]

Toaster

2015-08-19T09:16:50Z

Ddalex: /* About Toaster */

[[Category:Toaster]]
[[Toaster]] is a web-based interface to OpenEmbedded and BitBake. You might have heard it is a replacement for [https://www.yoctoproject.org/documentation/hob-manual Hob]. The answer is "it will be at some point, but not yet".

General discussion about '''Toaster''' happens on a dedicated mailing list: [https://lists.yoctoproject.org/listinfo/toaster https://lists.yoctoproject.org/listinfo/toaster]

=== Using Toaster ===

Toaster can run in various modes and setups. To ease out the understanding of documentation, we review here the terminology used throughout the documentation.

* Interactive mode - this is the mode released with Yocto Project Release 1.6. Functional components consist of a build recording component, 'toasterui', and a build stats and inspection user interface, the 'toastergui'. It is started with the command sequence listed below, and the builds are started using normal bitbake command line

$ source oe-init-build-env
$ source toaster start

* Managed mode - in this mode Toaster handles the build configuration GUI (through Project pages), and build scheduling and execution, in addition to the features launched with Yocto Project Release 1.6. The builds are triggered through the web interface,
the user as no direct access to the bitbake command line.

** Local managed mode, in short _local_, is the out-of-box mode available after a poky/ checkout. It allows the user to perform build on his local machine source code, with a local build directory, and is intended to help the user discover the interface, and configure and run local builds. You can launch it with

$ ./bitbake/bin/toaster

** Remote managed mode, or [[hosted Toaster]], is intended to be the production setup for running Toaster in organizations supporting multiple users and using customized Toaster installations.

=== Toaster How-to's ===

Specific pages with Toaster how-tos are available below.

* [[Setting up a local instance of Toaster]]
* [[Setting up a production instance of Toaster]] - documentation for Interactive mode
* [[Setting up a hosted managed mode for Toaster]] - configure and run build through Toaster itself, providing a service for your users
* [https://www.yoctoproject.org/documentation/toaster-manual-17 How to use the Toaster web interface]
* [[How to delete information from the Toaster database]]

=== About Toaster ===

* [[File:Working_with_design.pdf]]
* [https://bugzilla.yoctoproject.org/buglist.cgi?list_id=213820&columnlist=status_whiteboard%2Cassigned_to%2Ctarget_milestone%2Cbug_status%2Cshort_desc%2Cbug_severity%2Cpriority&classification=Build%20System%20%26%20Metadata&query_based_on=Toaster-Opens&query_format=advanced&bug_status=NEW&bug_status=ACCEPTED&bug_status=IN%20PROGRESS%20DESIGN&bug_status=IN%20PROGRESS%20DESIGN%20COMPLETE&bug_status=IN%20PROGRESS%20IMPLEMENTATION&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=WaitForUpstream&component=toaster&product=Toaster&known_name=Toaster-Opens Bug list]
* [[Toaster architecture design]]
* [[Toaster testing plan]]
* [[Toaster 1.9 design | Scope and design (1.9)]]
* [[Toaster 1.7 design | Scope and design (1.7 - 1.8)]]
* [[Toaster archive | Archive]]

=== In progress documentation ===

We are currently preparing the documentation for the Toaster build functionality. The content here is just a brain dump of what we need to cover (in no particular order). Feel free to add and create content as you see fit:

*[[Using virtualenv]]
*[[Toaster configuration]] - explain releases, layer sources, BitBake versions and default project configurations
*[[Set up Toaster locally]] - explain the set up process and the local version of the localconf.json file
*[[Set up production instance]] - explain the Django admin interface and the remove version of the localconf.json file
*[[manage.py commands]] - this should include an explanation of lsupdates
*[[Custom layer index]] - explain how to create your own layer index
*[[Start Toaster in managed mode]]

Contribute to Toaster

2015-07-15T17:15:24Z

Ddalex: /* Gotchas */

[[Category:Toaster]]
This page summarises the Toaster development process. We hope this will help you start contributing to the project.

== What can I do? ==

The [https://bugzilla.yoctoproject.org/buglist.cgi?product=Toaster Yocto Project Bugzilla instance] lists all the things that need to be done:

* If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list]
* If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list] to find out why the design work is not done yet

== Set up the local repository ==

For development of Toaster we recommend setting up a local install of Toaster. General install instructions are available in the main [https://www.yoctoproject.org/documentation/toaster-manual Toaster documentation]

Clone the poky repository
<code>
$ git clone git://git.yoctoproject.org/poky
$ cd poky
</code>

Install a python virtual environment to sandbox the python modules from your OS.
Enter Activate the python virtual environment in your current shell.
<code>
$ virtualenv venv
$ source ./venv/bin/activate
</code>

Install the python module dependencies for Toaster
<code>
$ pip install -r ./bitbake/toaster-requirements.txt
</code>

Run the setup and start script, follow instructions displayed
<code>
$ TOASTER_MANAGED=1 TOASTER_DEVEL=1 ./bitbake/bin/toaster
</code>

== Submitting patches ==

Publishing your patches to Toaster is a two step process.
# Sending patches to Toaster Project for review
# Submitting the patches that you reviewed to the upstream repository

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].
All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

=== Sending patches to Toaster Project ===

NOTE: The format of the commit message should be like this

<pre>
bitbake: toaster: <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] by "git send-email" please include in your subject line "[review-request][PATCH]"

e.g.
<code>
$ git send-email HEAD^ --subject-prefix="review-request][PATCH"
</code>

A comprehensive document about commit messages is available on the [http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines openembedded wiki]

More help learning git is available on [https://try.github.io github] and [http://git-scm.com/documentation/ the official documentation]

=== Sending branches to Toaster Project ===

If you wish to submit whole branches please use the poky-contrib repository see [[Poky Contributions#Poky_Contrib_Branch]] for setup guide.

Once you have pushed a branch please then send an email to the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] with the subject in the following format:

<code>
[review-request] my_branch_name
</code>

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

=== Submitting patch sets for integration upstream ===

All Toaster patches need to be submitted upstream to the Bitbake repository. Since development happens on the poky repository, but the patches need to be merged to the Bitbake repository, the following process should be executed.

* You will need a bitbake/ checkout separated from your current poky/ checkout. You can use one at the same level, e.g. ~/poky/ and ~/bitbake/ are your checkout directories.
* Make sure you have a clean directory which you can use to store the patches during the transition from poky/ to bitbake/. e.g. ~/patches
* Let's assume you are on the "ownname/work_branch" branch in the poky/ tree at the and on the "master" branch in the bitbake/ tree

Procedure:

# Make sure that the ~/patches directory exists and it is clean
#: <code>
mkdir -p ~/patches && rm -f ~/patches/*
</code>
# Export the toaster patches from the "ownname/work_branch" bitbake directory to the patches/ directory
#: <code>
cd ~/poky/bitbake && git format-patch -o ~/patches/ --relative
</code>
# Create a submission branch in the bitbake/ git checkout and import the patches exported from poky/; NOTE: Make sure that the patchset is rebased on top of the latest master branch.
#: <code>
cd ~/bitbake/ && git checkout origin/master -b submission && git pull
find ~/patches/ -type f -name *patch | sort -n | xargs -n1 git am
</code>
# Now we need to review this branch. All patches must be signed-off as reviewed by the upstreamer, which cannot be author of the patch.
#: <code>
git rebase -i origin/master
</code>
#: and edit each patch in part (s/^pick/edit/) to add your signature
#: <code>
git log -n1 -p
# review the changes; if all ok, sign off the patch, and continue
git commit --amend -s
git rebase --continue
</code>
# push the submission patch to poky-contrib (yes, the bitbake/ tree can be pushed as branch to the poky-contrib repo
#: <code>
git push poky-contrib submission:ownname/bitbake_submission
</code>
# and use the "create-pull-request" and "send-pull-request" from the poky/ repository in the poky/scripts/ directory to create a pull request and submit it to bitbake-devel@lists.openembedded.org; yes, you use the poky scripts in the bitbake/ directory
#: <code>
~/poky/scripts/create-pull-request -u poky-contrib -b ownname/bitbake_submission
# edit the patchset message in the pull-XXXX/0000-*.patch
~/poky/scripts/send-pull-request pull-XXXX/
</code>

==== Submitting patches for prior releases ====

The procedure is the same, but using the prior release as the base branch instead of the "master" branch in bitbake.

Also, make sure that you add the name of the prior release for which the patchset is intended in the prefix of the patchset, as parameter to the "create-pull-request" command, e.g. '''-p 1.26''' for the 1.26 branch.

==== Gotchas ====

Sometimes the mailer will refuse to send patches, especially on binary or long-line files. The proper way to go around that is to reply to the patchset you've submitted to the mailing list, asking for a git pull directly from the poky-contrib branch.

== Code syle guide ==

=== Templates ===

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

'''Yes please:'''

<pre>
{{var}}

<div>
{# Maintaining indentation #}
{% if %}
this
{% else %}
that
{% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
{{layer}}
{% endfor %}

</pre>

'''No thank you:'''
<pre>
{{var}}
<div>
{# Maintaining indentation #}
{%if%}this{%else%}that{%endif%}
</div>

{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}
</pre>

Note:
* Maintain indentation as you would with other languages
* White space after '%'
* Comment blocks for longer comments

=== Javascript ===

'''Yes please:'''
<pre>

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
cheddar : 1,
stilton : 2,
emmental : 3,
};

function doThingsHere(){
return 1;
}

/* If one equals two do some other things and make sure that
* if the the click handler is setup correctly.
*/
if (one === two) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function (event){
alert("Hello");
});
}

$("#little-mouse").focusout(function(){
alert("bye")
});

if (oneVar)
noThingHere();
else
doThingHere();
</pre>

'''No thank you:'''
<pre>
// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2, emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
alert("bye")
});

if (oneVar)
{
noThingHere();
} else { doThingHere(); }
</pre>

Note:
* Variables should be marked with "var"
* Semicolons should be used
* Keep as close to 80 cols as possible
* Use 2 space per indentation
* Open curly braces after parenthesis for functions and close on a new line
* Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:
<code>
$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js
</code>

=== HTML ===

'''Yes please:'''
<pre>
<div id="something-area">
This is some text
</div>

<div>
This is some text
</div>

</pre>

'''No thank you:'''
<pre>
<div id="somethingarea">
This is some text</div>
<div id="somethingarea">
This is
some text


</div>
</pre>

Note:
* 2 space indentation
* Lower case, ids hyphenated when multiple words
* No duplicate ids

* Run your HTML through a [http://validator.w3.org/#validate_by_input HTML validator] available for [http://validator.w3.org/source/ local install]. The w3c validator it's self doesn't currently validate html5, it uses as a back end [https://validator.github.io/validator/ Nu Html Checker] which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:
<pre>
$ mkdir html5-validator && cd html5-validator
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$ git clone https://github.com/validator/validator.git
$ python build/build.py all
$ python build/build.py all
</pre>

HTML can be indented quickly using tidy, for example:
<pre>
tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes ./index.html
</pre>

=== Python ===

Lenient [https://www.python.org/dev/peps/pep-0008 pep8]
Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and [http://pep8.readthedocs.org/en/latest/intro.html#error-codes error code list]

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.
<code>
$ pep8 ./toastergui/urls.py
</code>

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

<code>
$ pylint ./toastergui/views.py
</code>

Contribute to Toaster

2015-07-15T17:14:37Z

Ddalex: /* Submitting patch sets for integration upstream */

[[Category:Toaster]]
This page summarises the Toaster development process. We hope this will help you start contributing to the project.

== What can I do? ==

The [https://bugzilla.yoctoproject.org/buglist.cgi?product=Toaster Yocto Project Bugzilla instance] lists all the things that need to be done:

* If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list]
* If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list] to find out why the design work is not done yet

== Set up the local repository ==

For development of Toaster we recommend setting up a local install of Toaster. General install instructions are available in the main [https://www.yoctoproject.org/documentation/toaster-manual Toaster documentation]

Clone the poky repository
<code>
$ git clone git://git.yoctoproject.org/poky
$ cd poky
</code>

Install a python virtual environment to sandbox the python modules from your OS.
Enter Activate the python virtual environment in your current shell.
<code>
$ virtualenv venv
$ source ./venv/bin/activate
</code>

Install the python module dependencies for Toaster
<code>
$ pip install -r ./bitbake/toaster-requirements.txt
</code>

Run the setup and start script, follow instructions displayed
<code>
$ TOASTER_MANAGED=1 TOASTER_DEVEL=1 ./bitbake/bin/toaster
</code>

== Submitting patches ==

Publishing your patches to Toaster is a two step process.
# Sending patches to Toaster Project for review
# Submitting the patches that you reviewed to the upstream repository

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].
All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

=== Sending patches to Toaster Project ===

NOTE: The format of the commit message should be like this

<pre>
bitbake: toaster: <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] by "git send-email" please include in your subject line "[review-request][PATCH]"

e.g.
<code>
$ git send-email HEAD^ --subject-prefix="review-request][PATCH"
</code>

A comprehensive document about commit messages is available on the [http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines openembedded wiki]

More help learning git is available on [https://try.github.io github] and [http://git-scm.com/documentation/ the official documentation]

=== Sending branches to Toaster Project ===

If you wish to submit whole branches please use the poky-contrib repository see [[Poky Contributions#Poky_Contrib_Branch]] for setup guide.

Once you have pushed a branch please then send an email to the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] with the subject in the following format:

<code>
[review-request] my_branch_name
</code>

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

=== Submitting patch sets for integration upstream ===

All Toaster patches need to be submitted upstream to the Bitbake repository. Since development happens on the poky repository, but the patches need to be merged to the Bitbake repository, the following process should be executed.

* You will need a bitbake/ checkout separated from your current poky/ checkout. You can use one at the same level, e.g. ~/poky/ and ~/bitbake/ are your checkout directories.
* Make sure you have a clean directory which you can use to store the patches during the transition from poky/ to bitbake/. e.g. ~/patches
* Let's assume you are on the "ownname/work_branch" branch in the poky/ tree at the and on the "master" branch in the bitbake/ tree

Procedure:

# Make sure that the ~/patches directory exists and it is clean
#: <code>
mkdir -p ~/patches && rm -f ~/patches/*
</code>
# Export the toaster patches from the "ownname/work_branch" bitbake directory to the patches/ directory
#: <code>
cd ~/poky/bitbake && git format-patch -o ~/patches/ --relative
</code>
# Create a submission branch in the bitbake/ git checkout and import the patches exported from poky/; NOTE: Make sure that the patchset is rebased on top of the latest master branch.
#: <code>
cd ~/bitbake/ && git checkout origin/master -b submission && git pull
find ~/patches/ -type f -name *patch | sort -n | xargs -n1 git am
</code>
# Now we need to review this branch. All patches must be signed-off as reviewed by the upstreamer, which cannot be author of the patch.
#: <code>
git rebase -i origin/master
</code>
#: and edit each patch in part (s/^pick/edit/) to add your signature
#: <code>
git log -n1 -p
# review the changes; if all ok, sign off the patch, and continue
git commit --amend -s
git rebase --continue
</code>
# push the submission patch to poky-contrib (yes, the bitbake/ tree can be pushed as branch to the poky-contrib repo
#: <code>
git push poky-contrib submission:ownname/bitbake_submission
</code>
# and use the "create-pull-request" and "send-pull-request" from the poky/ repository in the poky/scripts/ directory to create a pull request and submit it to bitbake-devel@lists.openembedded.org; yes, you use the poky scripts in the bitbake/ directory
#: <code>
~/poky/scripts/create-pull-request -u poky-contrib -b ownname/bitbake_submission
# edit the patchset message in the pull-XXXX/0000-*.patch
~/poky/scripts/send-pull-request pull-XXXX/
</code>

==== Submitting patches for prior releases ====

The procedure is the same, but using the prior release as the base branch instead of the "master" branch in bitbake.

Also, make sure that you add the name of the prior release for which the patchset is intended in the prefix of the patchset, as parameter to the "create-pull-request" command, e.g. '''-p 1.26''' for the 1.26 branch.

==== Gotchas ====

Sometimes the mailer will refuse to send patches, especially on binary or long-line files. The proper way to go around that is to reply to the patchset you've submitted to the mailing list

== Code syle guide ==

=== Templates ===

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

'''Yes please:'''

<pre>
{{var}}

<div>
{# Maintaining indentation #}
{% if %}
this
{% else %}
that
{% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
{{layer}}
{% endfor %}

</pre>

'''No thank you:'''
<pre>
{{var}}
<div>
{# Maintaining indentation #}
{%if%}this{%else%}that{%endif%}
</div>

{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}
</pre>

Note:
* Maintain indentation as you would with other languages
* White space after '%'
* Comment blocks for longer comments

=== Javascript ===

'''Yes please:'''
<pre>

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
cheddar : 1,
stilton : 2,
emmental : 3,
};

function doThingsHere(){
return 1;
}

/* If one equals two do some other things and make sure that
* if the the click handler is setup correctly.
*/
if (one === two) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function (event){
alert("Hello");
});
}

$("#little-mouse").focusout(function(){
alert("bye")
});

if (oneVar)
noThingHere();
else
doThingHere();
</pre>

'''No thank you:'''
<pre>
// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2, emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
alert("bye")
});

if (oneVar)
{
noThingHere();
} else { doThingHere(); }
</pre>

Note:
* Variables should be marked with "var"
* Semicolons should be used
* Keep as close to 80 cols as possible
* Use 2 space per indentation
* Open curly braces after parenthesis for functions and close on a new line
* Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:
<code>
$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js
</code>

=== HTML ===

'''Yes please:'''
<pre>
<div id="something-area">
This is some text
</div>

<div>
This is some text
</div>

</pre>

'''No thank you:'''
<pre>
<div id="somethingarea">
This is some text</div>
<div id="somethingarea">
This is
some text


</div>
</pre>

Note:
* 2 space indentation
* Lower case, ids hyphenated when multiple words
* No duplicate ids

* Run your HTML through a [http://validator.w3.org/#validate_by_input HTML validator] available for [http://validator.w3.org/source/ local install]. The w3c validator it's self doesn't currently validate html5, it uses as a back end [https://validator.github.io/validator/ Nu Html Checker] which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:
<pre>
$ mkdir html5-validator && cd html5-validator
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$ git clone https://github.com/validator/validator.git
$ python build/build.py all
$ python build/build.py all
</pre>

HTML can be indented quickly using tidy, for example:
<pre>
tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes ./index.html
</pre>

=== Python ===

Lenient [https://www.python.org/dev/peps/pep-0008 pep8]
Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and [http://pep8.readthedocs.org/en/latest/intro.html#error-codes error code list]

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.
<code>
$ pep8 ./toastergui/urls.py
</code>

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

<code>
$ pylint ./toastergui/views.py
</code>

Contribute to Toaster

2015-07-15T17:08:44Z

Ddalex: /* Submitting patch sets for integration upstream */

[[Category:Toaster]]
This page summarises the Toaster development process. We hope this will help you start contributing to the project.

== What can I do? ==

The [https://bugzilla.yoctoproject.org/buglist.cgi?product=Toaster Yocto Project Bugzilla instance] lists all the things that need to be done:

* If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list]
* If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list] to find out why the design work is not done yet

== Set up the local repository ==

For development of Toaster we recommend setting up a local install of Toaster. General install instructions are available in the main [https://www.yoctoproject.org/documentation/toaster-manual Toaster documentation]

Clone the poky repository
<code>
$ git clone git://git.yoctoproject.org/poky
$ cd poky
</code>

Install a python virtual environment to sandbox the python modules from your OS.
Enter Activate the python virtual environment in your current shell.
<code>
$ virtualenv venv
$ source ./venv/bin/activate
</code>

Install the python module dependencies for Toaster
<code>
$ pip install -r ./bitbake/toaster-requirements.txt
</code>

Run the setup and start script, follow instructions displayed
<code>
$ TOASTER_MANAGED=1 TOASTER_DEVEL=1 ./bitbake/bin/toaster
</code>

== Submitting patches ==

Publishing your patches to Toaster is a two step process.
# Sending patches to Toaster Project for review
# Submitting the patches that you reviewed to the upstream repository

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].
All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

=== Sending patches to Toaster Project ===

NOTE: The format of the commit message should be like this

<pre>
bitbake: toaster: <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] by "git send-email" please include in your subject line "[review-request][PATCH]"

e.g.
<code>
$ git send-email HEAD^ --subject-prefix="review-request][PATCH"
</code>

A comprehensive document about commit messages is available on the [http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines openembedded wiki]

More help learning git is available on [https://try.github.io github] and [http://git-scm.com/documentation/ the official documentation]

=== Sending branches to Toaster Project ===

If you wish to submit whole branches please use the poky-contrib repository see [[Poky Contributions#Poky_Contrib_Branch]] for setup guide.

Once you have pushed a branch please then send an email to the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] with the subject in the following format:

<code>
[review-request] my_branch_name
</code>

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

=== Submitting patch sets for integration upstream ===

All Toaster patches need to be submitted upstream to the Bitbake repository. Since development happens on the poky repository, but the patches need to be merged to the Bitbake repository, the following process should be executed.

* You will need a bitbake/ checkout separated from your current poky/ checkout. You can use one at the same level, e.g. ~/poky/ and ~/bitbake/ are your checkout directories.
* Make sure you have a clean directory which you can use to store the patches during the transition from poky/ to bitbake/. e.g. ~/patches
* Let's assume you are on the "ownname/work_branch" branch in the poky/ tree at the and on the "master" branch in the bitbake/ tree

Procedure:

# Make sure that the ~/patches directory exists and it is clean
#: <code>
mkdir -p ~/patches && rm -f ~/patches/*
</code>
# Export the toaster patches from the "ownname/work_branch" bitbake directory to the patches/ directory
#: <code>
cd ~/poky/bitbake && git format-patch -o ~/patches/ --relative
</code>
# Create a submission branch in the bitbake/ git checkout and import the patches exported from poky/; NOTE: Make sure that the patchset is rebased on top of the latest master branch.
#: <code>
cd ~/bitbake/ && git checkout origin/master -b submission && git pull
find ~/patches/ -type f -name *patch | sort -n | xargs -n1 git am
</code>
# Now we need to review this branch. All patches must be signed-off as reviewed by the upstreamer, which cannot be author of the patch.
#: <code>
git rebase -i origin/master
</code>
#: and edit each patch in part (s/^pick/edit/) to add your signature
#: <code>
git log -n1 -p
# review the changes; if all ok, sign off the patch, and continue
git commit --amend -s
git rebase --continue
</code>
# push the submission patch to poky-contrib (yes, the bitbake/ tree can be pushed as branch to the poky-contrib repo
#: <code>
git push poky-contrib submission:ownname/bitbake_submission
</code>
# and use the "create-pull-request" and "send-pull-request" from the poky/ repository in the poky/scripts/ directory to create a pull request and submit it to bitbake-devel@lists.openembedded.org; yes, you use the poky scripts in the bitbake/ directory
#: <code>
~/poky/scripts/create-pull-request -u poky-contrib -b ownname/bitbake_submission
# edit the patchset message in the pull-XXXX/0000-*.patch
~/poky/scripts/send-pull-request pull-XXXX/
</code>

== Code syle guide ==

=== Templates ===

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

'''Yes please:'''

<pre>
{{var}}

<div>
{# Maintaining indentation #}
{% if %}
this
{% else %}
that
{% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
{{layer}}
{% endfor %}

</pre>

'''No thank you:'''
<pre>
{{var}}
<div>
{# Maintaining indentation #}
{%if%}this{%else%}that{%endif%}
</div>

{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}
</pre>

Note:
* Maintain indentation as you would with other languages
* White space after '%'
* Comment blocks for longer comments

=== Javascript ===

'''Yes please:'''
<pre>

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
cheddar : 1,
stilton : 2,
emmental : 3,
};

function doThingsHere(){
return 1;
}

/* If one equals two do some other things and make sure that
* if the the click handler is setup correctly.
*/
if (one === two) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function (event){
alert("Hello");
});
}

$("#little-mouse").focusout(function(){
alert("bye")
});

if (oneVar)
noThingHere();
else
doThingHere();
</pre>

'''No thank you:'''
<pre>
// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2, emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
alert("bye")
});

if (oneVar)
{
noThingHere();
} else { doThingHere(); }
</pre>

Note:
* Variables should be marked with "var"
* Semicolons should be used
* Keep as close to 80 cols as possible
* Use 2 space per indentation
* Open curly braces after parenthesis for functions and close on a new line
* Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:
<code>
$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js
</code>

=== HTML ===

'''Yes please:'''
<pre>
<div id="something-area">
This is some text
</div>

<div>
This is some text
</div>

</pre>

'''No thank you:'''
<pre>
<div id="somethingarea">
This is some text</div>
<div id="somethingarea">
This is
some text


</div>
</pre>

Note:
* 2 space indentation
* Lower case, ids hyphenated when multiple words
* No duplicate ids

* Run your HTML through a [http://validator.w3.org/#validate_by_input HTML validator] available for [http://validator.w3.org/source/ local install]. The w3c validator it's self doesn't currently validate html5, it uses as a back end [https://validator.github.io/validator/ Nu Html Checker] which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:
<pre>
$ mkdir html5-validator && cd html5-validator
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$ git clone https://github.com/validator/validator.git
$ python build/build.py all
$ python build/build.py all
</pre>

HTML can be indented quickly using tidy, for example:
<pre>
tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes ./index.html
</pre>

=== Python ===

Lenient [https://www.python.org/dev/peps/pep-0008 pep8]
Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and [http://pep8.readthedocs.org/en/latest/intro.html#error-codes error code list]

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.
<code>
$ pep8 ./toastergui/urls.py
</code>

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

<code>
$ pylint ./toastergui/views.py
</code>

Contribute to Toaster

2015-07-15T17:07:58Z

Ddalex: /* Submitting patch sets for integration upstream */

[[Category:Toaster]]
This page summarises the Toaster development process. We hope this will help you start contributing to the project.

== What can I do? ==

The [https://bugzilla.yoctoproject.org/buglist.cgi?product=Toaster Yocto Project Bugzilla instance] lists all the things that need to be done:

* If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list]
* If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list] to find out why the design work is not done yet

== Set up the local repository ==

For development of Toaster we recommend setting up a local install of Toaster. General install instructions are available in the main [https://www.yoctoproject.org/documentation/toaster-manual Toaster documentation]

Clone the poky repository
<code>
$ git clone git://git.yoctoproject.org/poky
$ cd poky
</code>

Install a python virtual environment to sandbox the python modules from your OS.
Enter Activate the python virtual environment in your current shell.
<code>
$ virtualenv venv
$ source ./venv/bin/activate
</code>

Install the python module dependencies for Toaster
<code>
$ pip install -r ./bitbake/toaster-requirements.txt
</code>

Run the setup and start script, follow instructions displayed
<code>
$ TOASTER_MANAGED=1 TOASTER_DEVEL=1 ./bitbake/bin/toaster
</code>

== Submitting patches ==

Publishing your patches to Toaster is a two step process.
# Sending patches to Toaster Project for review
# Submitting the patches that you reviewed to the upstream repository

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].
All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

=== Sending patches to Toaster Project ===

NOTE: The format of the commit message should be like this

<pre>
bitbake: toaster: <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] by "git send-email" please include in your subject line "[review-request][PATCH]"

e.g.
<code>
$ git send-email HEAD^ --subject-prefix="review-request][PATCH"
</code>

A comprehensive document about commit messages is available on the [http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines openembedded wiki]

More help learning git is available on [https://try.github.io github] and [http://git-scm.com/documentation/ the official documentation]

=== Sending branches to Toaster Project ===

If you wish to submit whole branches please use the poky-contrib repository see [[Poky Contributions#Poky_Contrib_Branch]] for setup guide.

Once you have pushed a branch please then send an email to the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] with the subject in the following format:

<code>
[review-request] my_branch_name
</code>

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

=== Submitting patch sets for integration upstream ===

All Toaster patches need to be submitted upstream to the Bitbake repository. Since development happens on the poky repository, but the patches need to be merged to the Bitbake repository, the following process should be executed.

* You will need a bitbake/ checkout separated from your current poky/ checkout. You can use one at the same level, e.g. ~/poky/ and ~/bitbake/ are your checkout directories.
* Make sure you have a clean directory which you can use to store the patches during the transition from poky/ to bitbake/. e.g. ~/patches
* Let's assume you are on the "ownname/work_branch" branch in the poky/ tree at the and on the "master" branch in the bitbake/ tree

Procedure:

# Make sure that the ~/patches directory exists and it is clean
#: <code>
mkdir -p ~/patches && rm -f ~/patches/*
</code>
# Export the toaster patches from the "ownname/work_branch" bitbake directory to the patches/ directory
#: <code>
cd ~/poky/bitbake && git format-patch -o ~/patches/ --relative
</code>

# Create a submission branch in the bitbake/ git checkout and import the patches exported from poky/; NOTE: Make sure that the patchset is rebased on top of the latest master branch.
#: <code>
cd ~/bitbake/ && git checkout origin/master -b submission && git pull
find ~/patches/ -type f -name *patch | sort -n | xargs -n1 git am
</code>

# Now we need to review this branch. All patches must be signed-off as reviewed by the upstreamer, which cannot be author of the patch.
#: <code>
git rebase -i origin/master
</code>
#: and edit each patch in part (s/^pick/edit/) to add your signature
#: <code>
git log -n1 -p
# review the changes; if all ok, sign off the patch, and continue
git commit --amend -s
git rebase --continue
</code>

# push the submission patch to poky-contrib (yes, the bitbake/ tree can be pushed as branch to the poky-contrib repo
#: <code>
git push poky-contrib submission:ownname/bitbake_submission
</code>

# and use the "create-pull-request" and "send-pull-request" from the poky/ repository in the poky/scripts/ directory to create a pull request and submit it to bitbake-devel@lists.openembedded.org; yes, you use the poky scripts in the bitbake/ directory
#: <code>
~/poky/scripts/create-pull-request -u poky-contrib -b ownname/bitbake_submission
# edit the patchset message in the pull-XXXX/0000-*.patch
~/poky/scripts/send-pull-request pull-XXXX/
</code>

== Code syle guide ==

=== Templates ===

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

'''Yes please:'''

<pre>
{{var}}

<div>
{# Maintaining indentation #}
{% if %}
this
{% else %}
that
{% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
{{layer}}
{% endfor %}

</pre>

'''No thank you:'''
<pre>
{{var}}
<div>
{# Maintaining indentation #}
{%if%}this{%else%}that{%endif%}
</div>

{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}
</pre>

Note:
* Maintain indentation as you would with other languages
* White space after '%'
* Comment blocks for longer comments

=== Javascript ===

'''Yes please:'''
<pre>

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
cheddar : 1,
stilton : 2,
emmental : 3,
};

function doThingsHere(){
return 1;
}

/* If one equals two do some other things and make sure that
* if the the click handler is setup correctly.
*/
if (one === two) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function (event){
alert("Hello");
});
}

$("#little-mouse").focusout(function(){
alert("bye")
});

if (oneVar)
noThingHere();
else
doThingHere();
</pre>

'''No thank you:'''
<pre>
// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2, emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
alert("bye")
});

if (oneVar)
{
noThingHere();
} else { doThingHere(); }
</pre>

Note:
* Variables should be marked with "var"
* Semicolons should be used
* Keep as close to 80 cols as possible
* Use 2 space per indentation
* Open curly braces after parenthesis for functions and close on a new line
* Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:
<code>
$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js
</code>

=== HTML ===

'''Yes please:'''
<pre>
<div id="something-area">
This is some text
</div>

<div>
This is some text
</div>

</pre>

'''No thank you:'''
<pre>
<div id="somethingarea">
This is some text</div>
<div id="somethingarea">
This is
some text


</div>
</pre>

Note:
* 2 space indentation
* Lower case, ids hyphenated when multiple words
* No duplicate ids

* Run your HTML through a [http://validator.w3.org/#validate_by_input HTML validator] available for [http://validator.w3.org/source/ local install]. The w3c validator it's self doesn't currently validate html5, it uses as a back end [https://validator.github.io/validator/ Nu Html Checker] which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:
<pre>
$ mkdir html5-validator && cd html5-validator
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$ git clone https://github.com/validator/validator.git
$ python build/build.py all
$ python build/build.py all
</pre>

HTML can be indented quickly using tidy, for example:
<pre>
tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes ./index.html
</pre>

=== Python ===

Lenient [https://www.python.org/dev/peps/pep-0008 pep8]
Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and [http://pep8.readthedocs.org/en/latest/intro.html#error-codes error code list]

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.
<code>
$ pep8 ./toastergui/urls.py
</code>

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

<code>
$ pylint ./toastergui/views.py
</code>

Contribute to Toaster

2015-07-15T17:07:26Z

Ddalex: /* Submitting patch sets for integration upstream */

[[Category:Toaster]]
This page summarises the Toaster development process. We hope this will help you start contributing to the project.

== What can I do? ==

The [https://bugzilla.yoctoproject.org/buglist.cgi?product=Toaster Yocto Project Bugzilla instance] lists all the things that need to be done:

* If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list]
* If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list] to find out why the design work is not done yet

== Set up the local repository ==

For development of Toaster we recommend setting up a local install of Toaster. General install instructions are available in the main [https://www.yoctoproject.org/documentation/toaster-manual Toaster documentation]

Clone the poky repository
<code>
$ git clone git://git.yoctoproject.org/poky
$ cd poky
</code>

Install a python virtual environment to sandbox the python modules from your OS.
Enter Activate the python virtual environment in your current shell.
<code>
$ virtualenv venv
$ source ./venv/bin/activate
</code>

Install the python module dependencies for Toaster
<code>
$ pip install -r ./bitbake/toaster-requirements.txt
</code>

Run the setup and start script, follow instructions displayed
<code>
$ TOASTER_MANAGED=1 TOASTER_DEVEL=1 ./bitbake/bin/toaster
</code>

== Submitting patches ==

Publishing your patches to Toaster is a two step process.
# Sending patches to Toaster Project for review
# Submitting the patches that you reviewed to the upstream repository

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].
All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

=== Sending patches to Toaster Project ===

NOTE: The format of the commit message should be like this

<pre>
bitbake: toaster: <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] by "git send-email" please include in your subject line "[review-request][PATCH]"

e.g.
<code>
$ git send-email HEAD^ --subject-prefix="review-request][PATCH"
</code>

A comprehensive document about commit messages is available on the [http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines openembedded wiki]

More help learning git is available on [https://try.github.io github] and [http://git-scm.com/documentation/ the official documentation]

=== Sending branches to Toaster Project ===

If you wish to submit whole branches please use the poky-contrib repository see [[Poky Contributions#Poky_Contrib_Branch]] for setup guide.

Once you have pushed a branch please then send an email to the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] with the subject in the following format:

<code>
[review-request] my_branch_name
</code>

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

=== Submitting patch sets for integration upstream ===

All Toaster patches need to be submitted upstream to the Bitbake repository. Since development happens on the poky repository, but the patches need to be merged to the Bitbake repository, the following process should be executed.

* You will need a bitbake/ checkout separated from your current poky/ checkout. You can use one at the same level, e.g. ~/poky/ and ~/bitbake/ are your checkout directories.
* Make sure you have a clean directory which you can use to store the patches during the transition from poky/ to bitbake/. e.g. ~/patches
* Let's assume you are on the "ownname/work_branch" branch in the poky/ tree at the and on the "master" branch in the bitbake/ tree

# Make sure that the ~/patches directory exists and it is clean
#: <code>
mkdir -p ~/patches && rm -f ~/patches/*
</code>
# Export the toaster patches from the "ownname/work_branch" bitbake directory to the patches/ directory
#: <code>
cd ~/poky/bitbake && git format-patch -o ~/patches/ --relative
</code>

# Create a submission branch in the bitbake/ git checkout and import the patches exported from poky/; NOTE: Make sure that the patchset is rebased on top of the latest master branch.
#: <code>
cd ~/bitbake/ && git checkout origin/master -b submission && git pull
find ~/patches/ -type f -name *patch | sort -n | xargs -n1 git am
</code>

# Now we need to review this branch. All patches must be signed-off as reviewed by the upstreamer, which cannot be author of the patch.
#: <code>
git rebase -i origin/master
</code>
#: and edit each patch in part (s/^pick/edit/) to add your signature
#: <code>
git log -n1 -p
# review the changes; if all ok, sign off the patch, and continue
git commit --amend -s
git rebase --continue
</code>

# push the submission patch to poky-contrib (yes, the bitbake/ tree can be pushed as branch to the poky-contrib repo
#: <code>
git push poky-contrib submission:ownname/bitbake_submission
</code>

# and use the "create-pull-request" and "send-pull-request" from the poky/ repository in the poky/scripts/ directory to create a pull request and submit it to bitbake-devel@lists.openembedded.org; yes, you use the poky scripts in the bitbake/ directory
#: <code>
~/poky/scripts/create-pull-request -u poky-contrib -b ownname/bitbake_submission
# edit the patchset message in the pull-XXXX/0000-*.patch
~/poky/scripts/send-pull-request pull-XXXX/
</code>

== Code syle guide ==

=== Templates ===

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

'''Yes please:'''

<pre>
{{var}}

<div>
{# Maintaining indentation #}
{% if %}
this
{% else %}
that
{% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
{{layer}}
{% endfor %}

</pre>

'''No thank you:'''
<pre>
{{var}}
<div>
{# Maintaining indentation #}
{%if%}this{%else%}that{%endif%}
</div>

{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}
</pre>

Note:
* Maintain indentation as you would with other languages
* White space after '%'
* Comment blocks for longer comments

=== Javascript ===

'''Yes please:'''
<pre>

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
cheddar : 1,
stilton : 2,
emmental : 3,
};

function doThingsHere(){
return 1;
}

/* If one equals two do some other things and make sure that
* if the the click handler is setup correctly.
*/
if (one === two) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function (event){
alert("Hello");
});
}

$("#little-mouse").focusout(function(){
alert("bye")
});

if (oneVar)
noThingHere();
else
doThingHere();
</pre>

'''No thank you:'''
<pre>
// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2, emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
alert("bye")
});

if (oneVar)
{
noThingHere();
} else { doThingHere(); }
</pre>

Note:
* Variables should be marked with "var"
* Semicolons should be used
* Keep as close to 80 cols as possible
* Use 2 space per indentation
* Open curly braces after parenthesis for functions and close on a new line
* Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:
<code>
$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js
</code>

=== HTML ===

'''Yes please:'''
<pre>
<div id="something-area">
This is some text
</div>

<div>
This is some text
</div>

</pre>

'''No thank you:'''
<pre>
<div id="somethingarea">
This is some text</div>
<div id="somethingarea">
This is
some text


</div>
</pre>

Note:
* 2 space indentation
* Lower case, ids hyphenated when multiple words
* No duplicate ids

* Run your HTML through a [http://validator.w3.org/#validate_by_input HTML validator] available for [http://validator.w3.org/source/ local install]. The w3c validator it's self doesn't currently validate html5, it uses as a back end [https://validator.github.io/validator/ Nu Html Checker] which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:
<pre>
$ mkdir html5-validator && cd html5-validator
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$ git clone https://github.com/validator/validator.git
$ python build/build.py all
$ python build/build.py all
</pre>

HTML can be indented quickly using tidy, for example:
<pre>
tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes ./index.html
</pre>

=== Python ===

Lenient [https://www.python.org/dev/peps/pep-0008 pep8]
Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and [http://pep8.readthedocs.org/en/latest/intro.html#error-codes error code list]

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.
<code>
$ pep8 ./toastergui/urls.py
</code>

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

<code>
$ pylint ./toastergui/views.py
</code>

Contribute to Toaster

2015-07-15T17:06:56Z

Ddalex: /* Submitting patch sets for integration upstream */

[[Category:Toaster]]
This page summarises the Toaster development process. We hope this will help you start contributing to the project.

== What can I do? ==

The [https://bugzilla.yoctoproject.org/buglist.cgi?product=Toaster Yocto Project Bugzilla instance] lists all the things that need to be done:

* If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list]
* If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list] to find out why the design work is not done yet

== Set up the local repository ==

For development of Toaster we recommend setting up a local install of Toaster. General install instructions are available in the main [https://www.yoctoproject.org/documentation/toaster-manual Toaster documentation]

Clone the poky repository
<code>
$ git clone git://git.yoctoproject.org/poky
$ cd poky
</code>

Install a python virtual environment to sandbox the python modules from your OS.
Enter Activate the python virtual environment in your current shell.
<code>
$ virtualenv venv
$ source ./venv/bin/activate
</code>

Install the python module dependencies for Toaster
<code>
$ pip install -r ./bitbake/toaster-requirements.txt
</code>

Run the setup and start script, follow instructions displayed
<code>
$ TOASTER_MANAGED=1 TOASTER_DEVEL=1 ./bitbake/bin/toaster
</code>

== Submitting patches ==

Publishing your patches to Toaster is a two step process.
# Sending patches to Toaster Project for review
# Submitting the patches that you reviewed to the upstream repository

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].
All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

=== Sending patches to Toaster Project ===

NOTE: The format of the commit message should be like this

<pre>
bitbake: toaster: <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] by "git send-email" please include in your subject line "[review-request][PATCH]"

e.g.
<code>
$ git send-email HEAD^ --subject-prefix="review-request][PATCH"
</code>

A comprehensive document about commit messages is available on the [http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines openembedded wiki]

More help learning git is available on [https://try.github.io github] and [http://git-scm.com/documentation/ the official documentation]

=== Sending branches to Toaster Project ===

If you wish to submit whole branches please use the poky-contrib repository see [[Poky Contributions#Poky_Contrib_Branch]] for setup guide.

Once you have pushed a branch please then send an email to the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] with the subject in the following format:

<code>
[review-request] my_branch_name
</code>

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

=== Submitting patch sets for integration upstream ===

All Toaster patches need to be submitted upstream to the Bitbake repository. Since development happens on the poky repository, but the patches need to be merged to the Bitbake repository, the following process should be executed.

* You will need a bitbake/ checkout separated from your current poky/ checkout. You can use one at the same level, e.g. ~/poky/ and ~/bitbake/ are your checkout directories.
* Make sure you have a clean directory which you can use to store the patches during the transition from poky/ to bitbake/. e.g. ~/patches
* Let's assume you are on the "ownname/work_branch" branch in the poky/ tree at the and on the "master" branch in the bitbake/ tree

# Make sure that the ~/patches directory exists and it is clean
#: <code>
mkdir -p ~/patches && rm -f ~/patches/*
</code>
# Export the toaster patches from the "ownname/work_branch" bitbake directory to the patches/ directory
#: <code>
cd ~/poky/bitbake && git format-patch -o ~/patches/ --relative
</code>

# Create a submission branch in the bitbake/ git checkout and import the patches exported from poky/; NOTE: Make sure that the patchset is rebased on top of the latest master branch.
#: <code>
cd ~/bitbake/ && git checkout origin/master -b submission && git pull
find ~/patches/ -type f -name *patch | sort -n | xargs -n1 git am
</code>

# Now we need to review this branch. All patches must be signed-off as reviewed by the upstreamer, which cannot be author of the patch.
#: <code>
git rebase -i origin/master
</code>
#: and edit each patch in part (s/^pick/edit/) to add your signature
#: <code>
git log -n1 -p
# if ok
git commit --amend -s
git rebase --continue
</code>

# push the submission patch to poky-contrib (yes, the bitbake/ tree can be pushed as branch to the poky-contrib repo
#: <code>
git push poky-contrib submission:ownname/bitbake_submission
</code>

# and use the "create-pull-request" and "send-pull-request" from the poky/ repository in the poky/scripts/ directory to create a pull request and submit it to bitbake-devel@lists.openembedded.org; yes, you use the poky scripts in the bitbake/ directory
#: <code>
~/poky/scripts/create-pull-request -u poky-contrib -b ownname/bitbake_submission
# edit the patchset message in the pull-XXXX/0000-*.patch
~/poky/scripts/send-pull-request pull-XXXX/
</code>

== Code syle guide ==

=== Templates ===

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

'''Yes please:'''

<pre>
{{var}}

<div>
{# Maintaining indentation #}
{% if %}
this
{% else %}
that
{% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
{{layer}}
{% endfor %}

</pre>

'''No thank you:'''
<pre>
{{var}}
<div>
{# Maintaining indentation #}
{%if%}this{%else%}that{%endif%}
</div>

{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}
</pre>

Note:
* Maintain indentation as you would with other languages
* White space after '%'
* Comment blocks for longer comments

=== Javascript ===

'''Yes please:'''
<pre>

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
cheddar : 1,
stilton : 2,
emmental : 3,
};

function doThingsHere(){
return 1;
}

/* If one equals two do some other things and make sure that
* if the the click handler is setup correctly.
*/
if (one === two) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function (event){
alert("Hello");
});
}

$("#little-mouse").focusout(function(){
alert("bye")
});

if (oneVar)
noThingHere();
else
doThingHere();
</pre>

'''No thank you:'''
<pre>
// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2, emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
alert("bye")
});

if (oneVar)
{
noThingHere();
} else { doThingHere(); }
</pre>

Note:
* Variables should be marked with "var"
* Semicolons should be used
* Keep as close to 80 cols as possible
* Use 2 space per indentation
* Open curly braces after parenthesis for functions and close on a new line
* Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:
<code>
$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js
</code>

=== HTML ===

'''Yes please:'''
<pre>
<div id="something-area">
This is some text
</div>

<div>
This is some text
</div>

</pre>

'''No thank you:'''
<pre>
<div id="somethingarea">
This is some text</div>
<div id="somethingarea">
This is
some text


</div>
</pre>

Note:
* 2 space indentation
* Lower case, ids hyphenated when multiple words
* No duplicate ids

* Run your HTML through a [http://validator.w3.org/#validate_by_input HTML validator] available for [http://validator.w3.org/source/ local install]. The w3c validator it's self doesn't currently validate html5, it uses as a back end [https://validator.github.io/validator/ Nu Html Checker] which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:
<pre>
$ mkdir html5-validator && cd html5-validator
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$ git clone https://github.com/validator/validator.git
$ python build/build.py all
$ python build/build.py all
</pre>

HTML can be indented quickly using tidy, for example:
<pre>
tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes ./index.html
</pre>

=== Python ===

Lenient [https://www.python.org/dev/peps/pep-0008 pep8]
Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and [http://pep8.readthedocs.org/en/latest/intro.html#error-codes error code list]

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.
<code>
$ pep8 ./toastergui/urls.py
</code>

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

<code>
$ pylint ./toastergui/views.py
</code>

Contribute to Toaster

2015-07-15T17:06:18Z

Ddalex:

[[Category:Toaster]]
This page summarises the Toaster development process. We hope this will help you start contributing to the project.

== What can I do? ==

The [https://bugzilla.yoctoproject.org/buglist.cgi?product=Toaster Yocto Project Bugzilla instance] lists all the things that need to be done:

* If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list]
* If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list] to find out why the design work is not done yet

== Set up the local repository ==

For development of Toaster we recommend setting up a local install of Toaster. General install instructions are available in the main [https://www.yoctoproject.org/documentation/toaster-manual Toaster documentation]

Clone the poky repository
<code>
$ git clone git://git.yoctoproject.org/poky
$ cd poky
</code>

Install a python virtual environment to sandbox the python modules from your OS.
Enter Activate the python virtual environment in your current shell.
<code>
$ virtualenv venv
$ source ./venv/bin/activate
</code>

Install the python module dependencies for Toaster
<code>
$ pip install -r ./bitbake/toaster-requirements.txt
</code>

Run the setup and start script, follow instructions displayed
<code>
$ TOASTER_MANAGED=1 TOASTER_DEVEL=1 ./bitbake/bin/toaster
</code>

== Submitting patches ==

Publishing your patches to Toaster is a two step process.
# Sending patches to Toaster Project for review
# Submitting the patches that you reviewed to the upstream repository

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].
All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

=== Sending patches to Toaster Project ===

NOTE: The format of the commit message should be like this

<pre>
bitbake: toaster: <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] by "git send-email" please include in your subject line "[review-request][PATCH]"

e.g.
<code>
$ git send-email HEAD^ --subject-prefix="review-request][PATCH"
</code>

A comprehensive document about commit messages is available on the [http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines openembedded wiki]

More help learning git is available on [https://try.github.io github] and [http://git-scm.com/documentation/ the official documentation]

=== Sending branches to Toaster Project ===

If you wish to submit whole branches please use the poky-contrib repository see [[Poky Contributions#Poky_Contrib_Branch]] for setup guide.

Once you have pushed a branch please then send an email to the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] with the subject in the following format:

<code>
[review-request] my_branch_name
</code>

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

=== Submitting patch sets for integration upstream ===

All Toaster patches need to be submitted upstream to the Bitbake repository. Since development happens on the poky repository, but the patches need to be merged to the Bitbake repository, the following process should be executed.

* You will need a bitbake/ checkout separated from your current poky/ checkout. You can use one at the same level, e.g. ~/poky/ and ~/bitbake/ are your checkout directories.
* Make sure you have a clean directory which you can use to store the patches during the transition from poky/ to bitbake/. e.g. ~/patches
* Let's assume you are on the "ownname/work_branch" branch in the poky/ tree at the and on the "master" branch in the bitbake/ tree

# Make sure that the ~/patches directory exists and it is clean
#: <code>
mkdir -p ~/patches && rm -f ~/patches/*
</code>
# Export the toaster patches from the "ownname/work_branch" bitbake directory to the patches/ directory
#: <code>
cd ~/poky/bitbake && git format-patch -o ~/patches/ --relative
</code>

# Create a submission branch in the bitbake/ git checkout and import the patches exported from poky/; NOTE: Make sure that the patchset is rebased on top of the latest master branch.
#: <code>
cd ~/bitbake/ && git checkout origin/master -b submission && git pull
find ~/patches/ -type f -name *patch | sort -n | xargs -n1 git am
</code>

# Now we need to review this branch. All patches must be signed-off as reviewed by the upstreamer, which cannot be author of the patch.
#: <code>
git rebase -i origin/master
</code>
#: and edit each patch in part (s/^pick/edit/) to add your signature
#: <code>
git log -n1 -p
# if ok
git commit --amend -s
git rebase --continue
</code>

# push the submission patch to poky-contrib (yes, the bitbake/ tree can be pushed as branch to the poky-contrib repo
#: <code>
git push poky-contrib submission:ownname/bitbake_submission
</code>

# and use the "create-pull-request" and "send-pull-request" from the poky/ repository in the poky/scripts/ directory to create a pull request and submit it to bitbake-devel@lists.openembedded.org; yes, you use the poky scripts in the bitbake/ directory
#: <code>
~/poky/scripts/create-pull-request -u poky-contrib -b ownname/bitbake_submission
# edit the patchset message in the pull-XXXX/0000-*.patch
~/poky/scripts/send-pull-request pull-XXXX/
</code>
== Code syle guide ==

=== Templates ===

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

'''Yes please:'''

<pre>
{{var}}

<div>
{# Maintaining indentation #}
{% if %}
this
{% else %}
that
{% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
{{layer}}
{% endfor %}

</pre>

'''No thank you:'''
<pre>
{{var}}
<div>
{# Maintaining indentation #}
{%if%}this{%else%}that{%endif%}
</div>

{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}
</pre>

Note:
* Maintain indentation as you would with other languages
* White space after '%'
* Comment blocks for longer comments

=== Javascript ===

'''Yes please:'''
<pre>

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
cheddar : 1,
stilton : 2,
emmental : 3,
};

function doThingsHere(){
return 1;
}

/* If one equals two do some other things and make sure that
* if the the click handler is setup correctly.
*/
if (one === two) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function (event){
alert("Hello");
});
}

$("#little-mouse").focusout(function(){
alert("bye")
});

if (oneVar)
noThingHere();
else
doThingHere();
</pre>

'''No thank you:'''
<pre>
// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2, emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
alert("bye")
});

if (oneVar)
{
noThingHere();
} else { doThingHere(); }
</pre>

Note:
* Variables should be marked with "var"
* Semicolons should be used
* Keep as close to 80 cols as possible
* Use 2 space per indentation
* Open curly braces after parenthesis for functions and close on a new line
* Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:
<code>
$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js
</code>

=== HTML ===

'''Yes please:'''
<pre>
<div id="something-area">
This is some text
</div>

<div>
This is some text
</div>

</pre>

'''No thank you:'''
<pre>
<div id="somethingarea">
This is some text</div>
<div id="somethingarea">
This is
some text


</div>
</pre>

Note:
* 2 space indentation
* Lower case, ids hyphenated when multiple words
* No duplicate ids

* Run your HTML through a [http://validator.w3.org/#validate_by_input HTML validator] available for [http://validator.w3.org/source/ local install]. The w3c validator it's self doesn't currently validate html5, it uses as a back end [https://validator.github.io/validator/ Nu Html Checker] which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:
<pre>
$ mkdir html5-validator && cd html5-validator
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$ git clone https://github.com/validator/validator.git
$ python build/build.py all
$ python build/build.py all
</pre>

HTML can be indented quickly using tidy, for example:
<pre>
tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes ./index.html
</pre>

=== Python ===

Lenient [https://www.python.org/dev/peps/pep-0008 pep8]
Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and [http://pep8.readthedocs.org/en/latest/intro.html#error-codes error code list]

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.
<code>
$ pep8 ./toastergui/urls.py
</code>

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

<code>
$ pylint ./toastergui/views.py
</code>

Contribute to Toaster

2015-07-15T16:31:32Z

Ddalex: /* Submitting patch sets for integration upstream */

[[Category:Toaster]]
This page summarises the Toaster development process. We hope this will help you start contributing to the project.

== What can I do? ==

The [https://bugzilla.yoctoproject.org/buglist.cgi?product=Toaster Yocto Project Bugzilla instance] lists all the things that need to be done:

* If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list]
* If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list] to find out why the design work is not done yet

== Set up the local repository ==

For development of Toaster we recommend setting up a local install of Toaster. General install instructions are available in the main [https://www.yoctoproject.org/documentation/toaster-manual Toaster documentation]

Clone the poky repository
<code>
$ git clone git://git.yoctoproject.org/poky
$ cd poky
</code>

Install a python virtual environment to sandbox the python modules from your OS.
Enter Activate the python virtual environment in your current shell.
<code>
$ virtualenv venv
$ source ./venv/bin/activate
</code>

Install the python module dependencies for Toaster
<code>
$ pip install -r ./bitbake/toaster-requirements.txt
</code>

Run the setup and start script, follow instructions displayed
<code>
$ TOASTER_MANAGED=1 TOASTER_DEVEL=1 ./bitbake/bin/toaster
</code>

== Submitting patches ==

Publishing your patches to Toaster is a two step process.
# Sending patches to Toaster Project for review
# Submitting the patches that you reviewed to the upstream repository

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].
All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

=== Sending patches to Toaster Project ===

NOTE: The format of the commit message should be like this

<pre>
bitbake: toaster: <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] by "git send-email" please include in your subject line "[review-request][PATCH]"

e.g.
<code>
$ git send-email HEAD^ --subject-prefix="review-request][PATCH"
</code>

A comprehensive document about commit messages is available on the [http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines openembedded wiki]

More help learning git is available on [https://try.github.io github] and [http://git-scm.com/documentation/ the official documentation]

=== Sending branches to Toaster Project ===

If you wish to submit whole branches please use the poky-contrib repository see [[Poky Contributions#Poky_Contrib_Branch]] for setup guide.

Once you have pushed a branch please then send an email to the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] with the subject in the following format:

<code>
[review-request] my_branch_name
</code>

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

=== Submitting patch sets for integration upstream ===

All Toaster patches need to be submitted upstream to the Bitbake repository. Since development happens on the poky repository, but the patches need to be merged to the Bitbake repository, the following process should be executed.

* You will need a bitbake/ checkout separated from your current poky/ checkout. You can use one at the same level, e.g. ~/poky/ and ~/bitbake/ are your checkout directories.
* Make sure you have a clean directory which you can use to store the patches during the transition from poky/ to bitbake/. e.g. ~/patches
* Let's assume you are on the "ownname/work_branch" branch in the poky/ tree at the and on the "master" branch in the bitbake/ tree

# Make sure that the ~/patches directory exists and it is clean
#:
<nowiki>
mkdir -p ~/patches && rm -f ~/patches/*

#

== Code syle guide ==

=== Templates ===

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

'''Yes please:'''

<pre>
{{var}}

<div>
{# Maintaining indentation #}
{% if %}
this
{% else %}
that
{% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
{{layer}}
{% endfor %}

</pre>

'''No thank you:'''
<pre>
{{var}}
<div>
{# Maintaining indentation #}
{%if%}this{%else%}that{%endif%}
</div>

{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}
</pre>

Note:
* Maintain indentation as you would with other languages
* White space after '%'
* Comment blocks for longer comments

=== Javascript ===

'''Yes please:'''
<pre>

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
cheddar : 1,
stilton : 2,
emmental : 3,
};

function doThingsHere(){
return 1;
}

/* If one equals two do some other things and make sure that
* if the the click handler is setup correctly.
*/
if (one === two) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function (event){
alert("Hello");
});
}

$("#little-mouse").focusout(function(){
alert("bye")
});

if (oneVar)
noThingHere();
else
doThingHere();
</pre>

'''No thank you:'''
<pre>
// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2, emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
alert("bye")
});

if (oneVar)
{
noThingHere();
} else { doThingHere(); }
</pre>

Note:
* Variables should be marked with "var"
* Semicolons should be used
* Keep as close to 80 cols as possible
* Use 2 space per indentation
* Open curly braces after parenthesis for functions and close on a new line
* Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:
<code>
$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js
</code>

=== HTML ===

'''Yes please:'''
<pre>
<div id="something-area">
This is some text
</div>

<div>
This is some text
</div>

</pre>

'''No thank you:'''
<pre>
<div id="somethingarea">
This is some text</div>
<div id="somethingarea">
This is
some text


</div>
</pre>

Note:
* 2 space indentation
* Lower case, ids hyphenated when multiple words
* No duplicate ids

* Run your HTML through a [http://validator.w3.org/#validate_by_input HTML validator] available for [http://validator.w3.org/source/ local install]. The w3c validator it's self doesn't currently validate html5, it uses as a back end [https://validator.github.io/validator/ Nu Html Checker] which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:
<pre>
$ mkdir html5-validator && cd html5-validator
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$ git clone https://github.com/validator/validator.git
$ python build/build.py all
$ python build/build.py all
</pre>

HTML can be indented quickly using tidy, for example:
<pre>
tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes ./index.html
</pre>

=== Python ===

Lenient [https://www.python.org/dev/peps/pep-0008 pep8]
Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and [http://pep8.readthedocs.org/en/latest/intro.html#error-codes error code list]

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.
<code>
$ pep8 ./toastergui/urls.py
</code>

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

<code>
$ pylint ./toastergui/views.py
</code>

Contribute to Toaster

2015-07-15T16:30:28Z

Ddalex:

[[Category:Toaster]]
This page summarises the Toaster development process. We hope this will help you start contributing to the project.

== What can I do? ==

The [https://bugzilla.yoctoproject.org/buglist.cgi?product=Toaster Yocto Project Bugzilla instance] lists all the things that need to be done:

* If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list]
* If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list] to find out why the design work is not done yet

== Set up the local repository ==

For development of Toaster we recommend setting up a local install of Toaster. General install instructions are available in the main [https://www.yoctoproject.org/documentation/toaster-manual Toaster documentation]

Clone the poky repository
<code>
$ git clone git://git.yoctoproject.org/poky
$ cd poky
</code>

Install a python virtual environment to sandbox the python modules from your OS.
Enter Activate the python virtual environment in your current shell.
<code>
$ virtualenv venv
$ source ./venv/bin/activate
</code>

Install the python module dependencies for Toaster
<code>
$ pip install -r ./bitbake/toaster-requirements.txt
</code>

Run the setup and start script, follow instructions displayed
<code>
$ TOASTER_MANAGED=1 TOASTER_DEVEL=1 ./bitbake/bin/toaster
</code>

== Submitting patches ==

Publishing your patches to Toaster is a two step process.
# Sending patches to Toaster Project for review
# Submitting the patches that you reviewed to the upstream repository

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].
All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

=== Sending patches to Toaster Project ===

NOTE: The format of the commit message should be like this

<pre>
bitbake: toaster: <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] by "git send-email" please include in your subject line "[review-request][PATCH]"

e.g.
<code>
$ git send-email HEAD^ --subject-prefix="review-request][PATCH"
</code>

A comprehensive document about commit messages is available on the [http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines openembedded wiki]

More help learning git is available on [https://try.github.io github] and [http://git-scm.com/documentation/ the official documentation]

=== Sending branches to Toaster Project ===

If you wish to submit whole branches please use the poky-contrib repository see [[Poky Contributions#Poky_Contrib_Branch]] for setup guide.

Once you have pushed a branch please then send an email to the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] with the subject in the following format:

<code>
[review-request] my_branch_name
</code>

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

=== Submitting patch sets for integration upstream ===

All Toaster patches need to be submitted upstream to the Bitbake repository. Since development happens on the poky repository, but the patches need to be merged to the Bitbake repository, the following process should be executed.

* You will need a bitbake/ checkout separated from your current poky/ checkout. You can use one at the same level, e.g. ~/poky/ and ~/bitbake/ are your checkout directories.
* Make sure you have a clean directory which you can use to store the patches during the transition from poky/ to bitbake/. e.g. ~/patches
* Let's assume you are on the "ownname/work_branch" branch in the poky/ tree at the and on the "master" branch in the bitbake/ tree

# Make sure that the ~/patches directory exists and it is clean
#: <code>
mkdir -p ~/patches && rm -f ~/patches/*
</code>
#

== Code syle guide ==

=== Templates ===

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

'''Yes please:'''

<pre>
{{var}}

<div>
{# Maintaining indentation #}
{% if %}
this
{% else %}
that
{% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
{{layer}}
{% endfor %}

</pre>

'''No thank you:'''
<pre>
{{var}}
<div>
{# Maintaining indentation #}
{%if%}this{%else%}that{%endif%}
</div>

{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}
</pre>

Note:
* Maintain indentation as you would with other languages
* White space after '%'
* Comment blocks for longer comments

=== Javascript ===

'''Yes please:'''
<pre>

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
cheddar : 1,
stilton : 2,
emmental : 3,
};

function doThingsHere(){
return 1;
}

/* If one equals two do some other things and make sure that
* if the the click handler is setup correctly.
*/
if (one === two) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function (event){
alert("Hello");
});
}

$("#little-mouse").focusout(function(){
alert("bye")
});

if (oneVar)
noThingHere();
else
doThingHere();
</pre>

'''No thank you:'''
<pre>
// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2, emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
alert("bye")
});

if (oneVar)
{
noThingHere();
} else { doThingHere(); }
</pre>

Note:
* Variables should be marked with "var"
* Semicolons should be used
* Keep as close to 80 cols as possible
* Use 2 space per indentation
* Open curly braces after parenthesis for functions and close on a new line
* Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:
<code>
$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js
</code>

=== HTML ===

'''Yes please:'''
<pre>
<div id="something-area">
This is some text
</div>

<div>
This is some text
</div>

</pre>

'''No thank you:'''
<pre>
<div id="somethingarea">
This is some text</div>
<div id="somethingarea">
This is
some text


</div>
</pre>

Note:
* 2 space indentation
* Lower case, ids hyphenated when multiple words
* No duplicate ids

* Run your HTML through a [http://validator.w3.org/#validate_by_input HTML validator] available for [http://validator.w3.org/source/ local install]. The w3c validator it's self doesn't currently validate html5, it uses as a back end [https://validator.github.io/validator/ Nu Html Checker] which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:
<pre>
$ mkdir html5-validator && cd html5-validator
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$ git clone https://github.com/validator/validator.git
$ python build/build.py all
$ python build/build.py all
</pre>

HTML can be indented quickly using tidy, for example:
<pre>
tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes ./index.html
</pre>

=== Python ===

Lenient [https://www.python.org/dev/peps/pep-0008 pep8]
Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and [http://pep8.readthedocs.org/en/latest/intro.html#error-codes error code list]

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.
<code>
$ pep8 ./toastergui/urls.py
</code>

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

<code>
$ pylint ./toastergui/views.py
</code>

Contribute to Toaster

2015-07-15T16:04:32Z

Ddalex:

[[Category:Toaster]]
This page summarises the Toaster development process. We hope this will help you start contributing to the project.

== What can I do? ==

The [https://bugzilla.yoctoproject.org/buglist.cgi?product=Toaster Yocto Project Bugzilla instance] lists all the things that need to be done:

* If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list]
* If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list] to find out why the design work is not done yet

== Set up the local repository ==

For development of Toaster we recommend setting up a local install of Toaster. General install instructions are available in the main [https://www.yoctoproject.org/documentation/toaster-manual Toaster documentation]

Clone the poky repository
<code>
$ git clone git://git.yoctoproject.org/poky
$ cd poky
</code>

Install a python virtual environment to sandbox the python modules from your OS.
Enter Activate the python virtual environment in your current shell.
<code>
$ virtualenv venv
$ source ./venv/bin/activate
</code>

Install the python module dependencies for Toaster
<code>
$ pip install -r ./bitbake/toaster-requirements.txt
</code>

Run the setup and start script, follow instructions displayed
<code>
$ TOASTER_MANAGED=1 TOASTER_DEVEL=1 ./bitbake/bin/toaster
</code>

== Sending patches to Toaster Project ==

NOTE: The format of the commit message should be like this

<pre>
bitbake: toaster: <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] by "git send-email" please include in your subject line "[review-request][PATCH]"

e.g.
<code>
$ git send-email HEAD^ --subject-prefix="review-request][PATCH"
</code>

A comprehensive document about commit messages is available on the [http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines openembedded wiki]

More help learning git is available on [https://try.github.io github] and [http://git-scm.com/documentation/ the official documentation]

== Sending branches to Toaster Project ==

If you wish to submit whole branches please use the poky-contrib repository see [[Poky Contributions#Poky_Contrib_Branch]] for setup guide.

Once you have pushed a branch please then send an email to the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] with the subject in the following format:

<code>
[review-request] my_branch_name
</code>

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

=== Submitting changes upstream ===

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].

== Code syle guide ==

=== Templates ===

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

'''Yes please:'''

<pre>
{{var}}

<div>
{# Maintaining indentation #}
{% if %}
this
{% else %}
that
{% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
{{layer}}
{% endfor %}

</pre>

'''No thank you:'''
<pre>
{{var}}
<div>
{# Maintaining indentation #}
{%if%}this{%else%}that{%endif%}
</div>

{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}
</pre>

Note:
* Maintain indentation as you would with other languages
* White space after '%'
* Comment blocks for longer comments

=== Javascript ===

'''Yes please:'''
<pre>

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
cheddar : 1,
stilton : 2,
emmental : 3,
};

function doThingsHere(){
return 1;
}

/* If one equals two do some other things and make sure that
* if the the click handler is setup correctly.
*/
if (one === two) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function (event){
alert("Hello");
});
}

$("#little-mouse").focusout(function(){
alert("bye")
});

if (oneVar)
noThingHere();
else
doThingHere();
</pre>

'''No thank you:'''
<pre>
// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2, emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
alert("bye")
});

if (oneVar)
{
noThingHere();
} else { doThingHere(); }
</pre>

Note:
* Variables should be marked with "var"
* Semicolons should be used
* Keep as close to 80 cols as possible
* Use 2 space per indentation
* Open curly braces after parenthesis for functions and close on a new line
* Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:
<code>
$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js
</code>

=== HTML ===

'''Yes please:'''
<pre>
<div id="something-area">
This is some text
</div>

<div>
This is some text
</div>

</pre>

'''No thank you:'''
<pre>
<div id="somethingarea">
This is some text</div>
<div id="somethingarea">
This is
some text


</div>
</pre>

Note:
* 2 space indentation
* Lower case, ids hyphenated when multiple words
* No duplicate ids

* Run your HTML through a [http://validator.w3.org/#validate_by_input HTML validator] available for [http://validator.w3.org/source/ local install]. The w3c validator it's self doesn't currently validate html5, it uses as a back end [https://validator.github.io/validator/ Nu Html Checker] which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:
<pre>
$ mkdir html5-validator && cd html5-validator
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$ git clone https://github.com/validator/validator.git
$ python build/build.py all
$ python build/build.py all
</pre>

HTML can be indented quickly using tidy, for example:
<pre>
tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes ./index.html
</pre>

=== Python ===

Lenient [https://www.python.org/dev/peps/pep-0008 pep8]
Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and [http://pep8.readthedocs.org/en/latest/intro.html#error-codes error code list]

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.
<code>
$ pep8 ./toastergui/urls.py
</code>

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

<code>
$ pylint ./toastergui/views.py
</code>

Design feedback

2015-07-06T12:21:48Z

Ddalex: /* "Normal" builds vs. "custom image recipe" builds */

== Feedback ==

=== Content and labeling ===

MICHAEL: creating an image recipe, I find the text in the modal too long, anything more than 2 sentences and my attention span is over!

BELEN: I am sure we can improve that text to make it shorter.

----

ALEX: BTW, thinking of the naming, I would suggest "Custom recipes", because it's not clear from the links that I can actually configure something there.

BELEN: Happy to review the labeling, as usual. I would add the word "image" though, because you cannot create custom software recipes or package groups: only image recipes. So it would be something like "Custom image recipes". Would that work?

----

ALEX: maybe we should change the terminology from Add/Delete?

BELEN: yes, everybody is screaming for this :) I will change it, but it needs to be changed across Toaster, not just in the image customisation process.

=== Creating an image recipe ===

MICHAEL: I found it confusing that I entered a name and clicked create but have not actually created anything

BELEN: mmm, this one is interesting. What makes you think that you have not created anything?

----

MICHAEL: what about something like suggestion1.png (attached) this brings it more into line with import layer/create new project/form based activity.

BELEN: that is quite similar to Tiago's sketches. I shied away from that approach for 2 reasons:

# Because I find it crowded and lacking on clear priorities. There is a lot of content and controls in this screen. Most of them we need to deactivate until a name for the image recipe has been set. I am sure there are things we could do in visual design to improve the balance in such a screen, but not in this first go. The question that immediately comes to my mind when I see it is: if I cannot manipulate these content and controls, why are you showing them to me? That's why I sticked to the 2-step approach: give this a name first, so that we can create the image recipe, then, once created, you can proceed to "configure" it.
#Because it meant we need 2 different custom image recipe pages: one for when you are creating the image for the first time (with the steps numbered and the name field at the top), and a second one for when you are editing the image recipe after creating it (with the name you gave before as the heading 1, and no steps). Splitting the naming step from the custom image recipe page means that we can use exactly the same page for both circumstances, which I think is good.

The above is just the rationale behind my design, just to make this clear. I believe your suggestion would also work. To figure out which one would work better, we would need to test them, and unfortunately we don't have time for that right now. But I am sure we can work together to come up with something. It would be good to hear a bit more about the reasons why you prefer your suggestion.

=== Select a base image recipe ===

MICHAEL: I was looking for a way to "de/re/select" an image, we don't quite have the same concept here as the machines selection that I was expecting, where you can always select regardless of whether it's already selected.

BELEN: just to make sure I understand, do you mean the fact that you get a 'selected' badge next to the selected base image recipe? The initial designs for machines also indicated which machine was selected with a similar badge, until it was pointed out to me that there is no way for us to know which layer is actually providing the machine you are setting. The only thing we know for sure is the name of the machine. I do not think is critical to indicate which is the selected base image recipe in the table: you already have it on the image summary (the well1 on the right hand side). So maybe we can survive without that.

----

ALEX: When configuring an image, I would suggest dropping the "Base image recipe" concept, because we can't follow on updates from the base image recipe after the custom recipe was created, and this will confuse the user.

BELEN: and what do users do instead? Do they start they custom image recipes from a blank slate? That will most likely result in an image that doesn't build. Do you have an alternative starting point in mind?

ALEX: The users would start from an already existing image, as they do now. What I'm suggesting is dropping the tab of possible base images, and the ability to change the base image for a custom image once the custom image recipe is created. This is the same point as below, just pointing to the tab.

----

ALEX: Also, the tab with the Base image recipe in My Image details, allowing the user to change the base image for a recipe, should be taken out, since we cannot change the base image recipe after the initial selection.

BELEN: Not allowing people to change their minds about the selected base image recipe is a bit draconian. It means that if you select the wrong base image by mistake, you will be forced to discard the custom image recipe, and restart from scratch. A bit harsh. After discussing this with Paul, it turns out the agreement was the following: if you change the base image recipe, we'll drop the initial list of packages and any deletions you might have done on it (we will warn you about that before you change the base image). Ideally we would be able to remember any packages you have added, but if this is not feasible, we might be able to survive without it. I need to design this.

ALEX: When storing a custom image recipe, we have two options: store an absolute list of packages, or store a reference to a base image and a difference for the list of packages. These options are available only while working within the Toaster instance - once you export your recipe, you have to export an absolute list of packages.

Each approach has advantages and disadvantages. The functionality described above requires storing a reference + difference, in order to be able to change the base image. The reference+difference approach has the advantage of changing the package list based on the image features and current configuration, and following the upstream changes, and the disadvantage of needing parsing on each project configuration change, and possibly changing the image package list outside user's control (when the upstream base image changes).

=== Custom image summary (right column) ===

MICHAEL: I was expecting the pencil to do the same as other pencils and activate text input boxes. Obviously if you're on the Add/Delete packages page you can't change the base image like that so maybe not having these pencils would be better. I was also unsure as what the change version/licence would do.

BELEN: Most of the pencil icons will activate text input boxes, it's just that I didn't prototype the interaction because we already know how they work (we are just reusing the existing interaction). The pencils are the way you can change the name, the version and the license for your custom image recipe. The only exception is the pencil icon next to the base image recipe. That would be a link to the 'Select a base image recipe' page. I agree this is not consistent. In general, I think the idea of that image summary is good, but the content could be improved: so maybe we'll look into a better way to organise it and better labelling for the different items.

=== Adding / removing layers ===

MICHAEL: If you've selected an image recipe and then remove the layer that provides it... what happens?

BELEN: heh, didn't think of this one :) Any changes in the layer list, adding or removing, will trigger a parsing. Once the parsing is finished, we could flag the fact that the base image recipe is no longer provided by any project layers. But, if you have removed the layer that provides the base image recipe, I would like to think nothing happens, since we would have created a new .bb file that doesn't depend on the original image recipe. If we decide that the custom image recipe should inherit the base image recipe, well, then we would have a problem: we would need to force people to select a new base image recipe. Paul can probably clarify what would really happen, so I'll make sure to ask him.

----

ALEX: The "Add A Layer" panel on the right hand side, I would've expected it to be a pop-up like in the configuration page, not take me to the Compatible layers. It completely takes me out of the context of what I was doing - I want to customize a image for MinnowMax, I want to add minnowmax and be done, not go to a different page, and then have no idea how to return in a single click.

BELEN: you are right. We can probably just replicate the mechanism we have in the basic configuration page: an 'add layer' form with links to view all the layers and to import a layer. So if you know what you are doing, you can add the layer you need without exiting the image configuration page.

----

DAVID: The new page can add layers in place which is nice, and I see how you can use it to parse and show that layer's package count. However, it appears that if I change my mind about the layer I have to go out to the layer page to remove it? Could perhaps the "undo" feature also be made available here? Or a layer delete icon?

BELEN: See the [[#The 'Undo' feature]] below. You can delete layers in these pages using the 'delete' icon in the Project Layers section (right-hand column). The ability to delete layers creates other issues though, as pointed by Michael above.

----

ALEX: Also, when deleting / adding a layer in-page, the recipes available would just appear/disappear from the "available recipes" table.

BELEN: Which one is the "available recipes" table? The one listing all the base image recipes you can choose from? If yes, those don't disappear if you delete a layer. Because we will know about them from the layer index, we list them always, and we allow you to select them by adding the required layer first. When you remove a layer, what changes is the button that you see next to the base image recipe.

=== Actions ===

MICHAEL: The Build Image recipe/ Download recipe file / Delete image recipe buttons are somewhat easy to miss, they look a bit like progress buttons or other tabs. I wonder if they would be better off in '.well' 1. The Build button could be more noticeable.

ALEX: In the "My image recipe", I am not sure why the primary actions that you can take on the recipe (Build, Download, Delete) are tucked away in the right hand side, when my focus is on the left hand side, where I get drawn to the image name. Maybe we can move the buttons, make them bigger, as to be clear they are the primary action you do with a customized image ?

BELEN: agreed. I tried many options, but I wasn't particularly happy with any of them. I'll give this another go.

=== Dependency size ===

MICHAEL: In general if you can avoid having data in a table that for each one requires extra data looking up the tables will be much faster/efficient. For example we have the dependencies button with total size displayed. It should be really quick to count the number of dependences, but much slower if we also have to retrieve the objects to get the data in them, such as "name" and "size". If we can do that by making it "on demand" that's much better, e.g. you click the button and it fetches this extra data.

BELEN: this was my initial approach. From the UI perspective, it is also tidier. Then Ed asked to see this information without having to click, and I decided to give it a try to see how it would look like. I agree it is better to present it when you select a certain dependency, so I will revert to that. Sorry Ed :/

=== Displaying image contents ===

DAVID: Can we add a filter so that we can show just the packages that we can delete? The use case is trimming the image, where we want the 100 packages that we wish to examine for removal not to be lost in the possible 1000's packages that are available to add.

BELEN: yes, absolutely. The prototype does not include filters, but the intention is adding at least the one you are asking for.

----

ALEX: How do I know what's in my image in at a certain moment ?

BELEN: We will provide a filter for that, as explained in above.

----

ALEX: It would be great if I could have two tables in the image customization page - the current contents of the image, and the packages that are available. When selecting/removing a package, a package would disappear from a list and appear in the second one.

BELEN: This is a nice idea, but I don't think we have space for 2 tables. I toyed with the concept of a list of added packages that populates as you add things, to provide better visibility of your changes. This list would be on the right hand column, with the other custom image recipe information (name, version, etc). Would that do?

=== Notifications ===

DAVID: I will admit that I was thrown by the new status pop-up, because not only does it cover things up on my page, I also generally associate them with spam and ad-ware. I understand your use for showing an action in progress, but we already had a metaphor for that in the progress/status bars inserted (when applicable) at the top of the page. Why a new metaphor? What does it add that the regular model does not? I know that it does stay visible while you for example scroll, but is that a hard requirement?

BELEN: Just to be sure, I guess you are talking about the fixed-positioned notifications that appear when you perform an action (for example, adding a layer). Their main advantage over our current notification system is that they are always visible. Since these notifications are the way Toaster provides feedback to users about their actions, that's a significant advantage. They are also a relatively common way of presenting notification nowadays (Dropbox uses them, just to give a random example, but so do lots of other web apps). Another common way is "pop from top" messages, like these ones:

https://css-tricks.com/pop-from-top-notification/

They have an example here:

https://css-tricks.com/examples/PopFromTopMessage/

I gave those a quick try, but didn't play nice with the fixed top bar we are currently using in Toaster.

I can tweak the design a bit to make the fixed-positioned notifications less offensive (smaller, with less shadow, for example). If you would like to find a completely different way to handle notifications, let's open a Bugzilla entry and I can definitely do that at a later stage.

----

DAVID: I also do not understand the dangling part, where I have to manual cancel it to make it go away. For example, I understand for example its use in the add layer parsing progress, but when it is done and says "Layer Information updated" why do I have to manually kill it? Could it at least go away on its own after a moment, and let that be the clue that the process completed?

BELEN: yes, this we can definitely do. We just need to come up with the right time interval.

----

ALEX: I subscribe to David's view that the popups are annoying and unnecessary.

BELEN: they are not popups, just to be clear. Popups are modal. These things do not prevent you from interacting with the interface until you take action on them.

----

ALEX: I would suggest using box alerts as they are currently implemented, to avoid obscuring the screen, and divert user's attention. The feedback for immediate actions should not be in your face.

BELEN: well, it kind of needs to be. Feedback is incredibly important in interfaces: users must know the state of the system and the outcome of their actions. Notifications should be therefore visible (the ones we have right now not always are), and that's why so many web applications are using fix-positioned ones like these to provide feedback to users.

----

ALEX: Ditto for the "data loading" spinner, let's make it a bit more obscure.

BELEN: Michael and I spent a good deal of time trying to work out the best way to present this using the standard UI widgets that come with Bootstrap. I still think the way it is right now is the best way we can currently make it. That doesn't mean it cannot be improved. I am sure Levi will be ok with including a 'data loading' UI component in the library he is creating for Toaster.

=== The 'Undo' feature ===

DAVID: The "Undo" feature though is nice!

BELEN: "undo" features are very, very nice, I agree. I've added it here as suggested by the rest of the design contributors, but the reality is that I don't expect it to be part of the image customisation feature, that's why I don't really mention it in the video. "Undo" should probably be designed and implemented and an application-wide feature, which means it should work for pretty much all actions. This poses a certain problem for us, since "undoing" adding a layer might not be as easy as "undoing" removing a package from a custom image recipe. What I mean by this is that 'undo' requires some thought. We can open a Bugzilla entry to design an "Undo" feature for Toaster, but it will not be part of the image customisation work.

=== Layer parsing and package count ===

DAVID: Speaking of layer parsing, don't we already have an idea of the package count per layer from the "all packages" table?

BELEN: I am a bit confused by the "package count per layer" sentence. In the image customisation process, what matters is not the package count per layer, but the package count per base image recipe, which is the very important information when you need to decide which image recipe you should use as a starting point.

=== Package groups ===

DAVID: How does this interface deal with "package groups"? People will ask.

http://www.yoctoproject.org/docs/1.8/dev-manual/dev-manual.html#usingpoky-extend-customimage-customtasks

BELEN: I'm afraid it doesn't. We know that at some point we'll need to distinguish package groups from other recipes, and break them up, i.e. allow people to remove packages from package groups. But during the high level design discussions we came to the conclusion that it would be too hard to do in version 1 - see page 11 of the initial design document

https://wiki.yoctoproject.org/wiki/images/3/31/Image_customisation_in_Toaster.pdf

=== Concerns about package removal ===

DAVID: Adding packages is easy, but we (WR) have found that removing them is weirdly hard. I am very curious how the backend package removal/exclusion is being done, and who is doing it. In any case, we should have disclaimers in place.

* Some removed packages appear anyway in the image, because sometimes the dependency information in the packages are not complete nor correct.

* Some removed packages will break the build or the runtime, even though the dependencies say it should be ok. Users should be encouraged to test their changes early and often, and we may want to help save their bacon with checkpoints (based on the last successful build?) or multiple undo's so that they can back up to a working state rather than re-starting from scratch. Just saying.

BELEN: I cannot answer your question about package removal (maybe Paul?), but good to be warned about potential problems and possible solutions. They might not make version 1, but we can definitely enhance the functionality in subsequent releases.

ALEX: This is a limitation of the way metadata is structured and tested. The YP developers often miss correct dependency linking because the actual dependencies for a package (e.g. installed libraries) were already satisfied by another chain, so they miss specifying the dependency in the metadata. This problem will be re-occuring during Toaster use, and until we clean up the dependencies metadata, we need to tell the users exactly what's going on - and enable them to timely submit error reports about the invalid metadata. This is the only way we can actually clean everything up.

=== "Normal" builds vs. "custom image recipe" builds ===

ALEX: I think it should be easy to build your own recipes in the Configuration page. The text entry with the build button should be no longer the primary means of starting a build, now that we can configure the build itself. I would suggest adding a "My image recipes" box in the style of "Most build recipes" in the Configuration, with two big buttons : "Build selected" and "New image recipe".

BELEN: right, I am a bit confused about this suggestion. We decided to "isolate" the builds of custom image recipes from "normal" builds to make sure we didn't have to deal with pushing layers to Git repos. To make this separation (which is by no means ideal) as clear as possible, we agreed to create a separate "My image recipes" section, from which you cannot start "normal" builds, and from where you start only builds of you custom image recipes. If you want to integrate your custom image recipe into your "normal" builds you need to download the .bb file and add it to your custom layer, then import that custom layer into Toaster. Please read page 5 of the high-level design document available here

https://wiki.yoctoproject.org/wiki/images/3/31/Image_customisation_in_Toaster.pdf

The above is what the initial design workshop concluded. I think we are all most definitely open to new ideas, but they must involve a solution to the problem of which layer do custom image recipes go to.

ALEX: Maybe I overstated my suggestion - it was asking simply to add an extra box launch already-configured custom images in the Configuration page, not changing the page structure.
----

ALEX: For a new user, it's not obvious how to start configuring the image contents, in the sense that "My image recipes" isn't the most obvious place one would start to configure a build, which is why people actually look for into Toaster. I would suggest making "Type the target you want to build" and Build button in a section similar to, and on top of the "Latest builds" section. And add, in that section, in the project page, a bit link "Configure the image contents".

BELEN: Again, I am confused. There is now a build form not only above the "latest builds" section, but also in all project configuration pages, in exactly the same place so that it remains easy to find for users. That component is removed from the "My image recipes" pages because we concluded we should separate their builds from "normal" builds as explained above. Your "configure the image contents" suggestion would conflate both types of builds. Again, this is something I would love to do, but we would need a way to handle the question of which layer do custom image recipes go to.

ALEX: I am not sure I fully understand the separation between the two types of builds - I did not understand the separation. In my mind, there is one single type of builds.

The custom recipes will got in an ad-hoc temporary layer created by Toaster in all cases, and this layer will show up in the build data. I suppose that there will be no obscuring of this information.

----

ALEX: It seems a bit strange that I can navigate to most of the Project options via the left hand menu, but not to image recipes. The user-defined image recipes are Project-specific, and I think they are at the same level with the "Image recipes" and "Software recipes", but in the "CONFIGURATION" category. I don't think that having a duplicate link as a tab is a problem, but I get frustrated by the disappearance of the left-hand-menu in the My Image Recipes.

BELEN: This is once more a consequence of the agreed separation between "normal" and custom image recipe builds. In the project page: the "builds" content spans both, the "configuration" content belongs to the "normal" builds context, and the "my image recipes" content belongs to the custom image recipes context. The latter 2 are kept separate as the outcome of the initial design workshop suggested should be done.

ALEX: Maybe we should revisit this in a face-to-face meeting ?

=== New build button ===

ALEX: Please drop the "New build" button. That button is a stop-gap measure to launching build commands, but now we're not simply issue-ing build commands, we are actually configuring the build. Even if you are an experienced user, it's opaque form - small and with very little information - makes it difficult to use.

BELEN: mmm, drop it from where? It is nowhere in the image customisation process. The purpose of the 'new build' button is allow you to build for any project from anywhere. I still think that's a handy thing to have, although I have no evidence to back that up. I am ok with reconsidering that feature, but not as part of this review, which is supposed to be about the image customisation design.

ALEX: ahm, right, dropping it from the top toolbar, next to the New Project button. Not part of the image customization though, but the thought was triggered by being in the Custom image page and seeing multiple Build buttons.

== Design ARs ==

* Find a way to show that the number of packages shown after parsing is a best guess
* In the 'my image recipes' table, see if there is a way we can see a build button in the last table column, consistent with how all other recipes are displayed
* Review the content in the 'name your image recipe' modal. The help text is too long, and the heading should match somehow the button clicked ('new image recipe'). Consider change to a page, like in projects.
* Change 'my image recipes' to 'custom image recipes'
* Change 'add / delete' to 'add / remove' across Toaster
* Talk to Michael about his suggested alternative to the image customisation page (with the name at the top as step 1, and the tabs afterwards as step 2)
* Confirm with Michael if I should remove the 'selected' badge in the base image recipes table
* Design the 'changing my base image' workflow. All the initial packages and any of them you have deleted will be lost. We should ideally keep the list of packages you added, but what if any of them already exists in the new base image? Think all this through.
* Remove the pencil next to the base image recipe in the right column summary: its behaviour would be inconsistent with all other pencils in Toaster. Also, changing the base image recipe has big impact, so it should probably not be too easy.
* Review the content of the right column summary
* Asking Paul what would happen in the image customisation process if you remove the layer providing the base image for a custom image recipe
* Add the 'add layer' form as is in the configuration page to the Project layers section in the right hand column
* Review the presentation of the main page actions (build / download / delete)
* Remove the dependency size info from the button and put it inside the popover
* Add the needed filters
* Add a list of packages added to the image summary in the right column
* Make the fixed-positioned notifications less offensive (check the shadow and size)
* Define the time interval after which the fixed notifications fade out
* Ask Levi to design a 'loading' component for the UI library

Design feedback

2015-07-06T11:03:52Z

Ddalex: /* New build button */

== Feedback ==

=== Content and labeling ===

MICHAEL: creating an image recipe, I find the text in the modal too long, anything more than 2 sentences and my attention span is over!

BELEN: I am sure we can improve that text to make it shorter.

----

ALEX: BTW, thinking of the naming, I would suggest "Custom recipes", because it's not clear from the links that I can actually configure something there.

BELEN: Happy to review the labeling, as usual. I would add the word "image" though, because you cannot create custom software recipes or package groups: only image recipes. So it would be something like "Custom image recipes". Would that work?

----

ALEX: maybe we should change the terminology from Add/Delete?

BELEN: yes, everybody is screaming for this :) I will change it, but it needs to be changed across Toaster, not just in the image customisation process.

=== Creating an image recipe ===

MICHAEL: I found it confusing that I entered a name and clicked create but have not actually created anything

BELEN: mmm, this one is interesting. What makes you think that you have not created anything?

----

MICHAEL: what about something like suggestion1.png (attached) this brings it more into line with import layer/create new project/form based activity.

BELEN: that is quite similar to Tiago's sketches. I shied away from that approach for 2 reasons:

# Because I find it crowded and lacking on clear priorities. There is a lot of content and controls in this screen. Most of them we need to deactivate until a name for the image recipe has been set. I am sure there are things we could do in visual design to improve the balance in such a screen, but not in this first go. The question that immediately comes to my mind when I see it is: if I cannot manipulate these content and controls, why are you showing them to me? That's why I sticked to the 2-step approach: give this a name first, so that we can create the image recipe, then, once created, you can proceed to "configure" it.
#Because it meant we need 2 different custom image recipe pages: one for when you are creating the image for the first time (with the steps numbered and the name field at the top), and a second one for when you are editing the image recipe after creating it (with the name you gave before as the heading 1, and no steps). Splitting the naming step from the custom image recipe page means that we can use exactly the same page for both circumstances, which I think is good.

The above is just the rationale behind my design, just to make this clear. I believe your suggestion would also work. To figure out which one would work better, we would need to test them, and unfortunately we don't have time for that right now. But I am sure we can work together to come up with something. It would be good to hear a bit more about the reasons why you prefer your suggestion.

=== Select a base image recipe ===

MICHAEL: I was looking for a way to "de/re/select" an image, we don't quite have the same concept here as the machines selection that I was expecting, where you can always select regardless of whether it's already selected.

BELEN: just to make sure I understand, do you mean the fact that you get a 'selected' badge next to the selected base image recipe? The initial designs for machines also indicated which machine was selected with a similar badge, until it was pointed out to me that there is no way for us to know which layer is actually providing the machine you are setting. The only thing we know for sure is the name of the machine. I do not think is critical to indicate which is the selected base image recipe in the table: you already have it on the image summary (the well1 on the right hand side). So maybe we can survive without that.

----

ALEX: When configuring an image, I would suggest dropping the "Base image recipe" concept, because we can't follow on updates from the base image recipe after the custom recipe was created, and this will confuse the user.

BELEN: and what do users do instead? Do they start they custom image recipes from a blank slate? That will most likely result in an image that doesn't build. Do you have an alternative starting point in mind?

ALEX: The users would start from an already existing image, as they do now. What I'm suggesting is dropping the tab of possible base images, and the ability to change the base image for a custom image once the custom image recipe is created. This is the same point as below, just pointing to the tab.

----

ALEX: Also, the tab with the Base image recipe in My Image details, allowing the user to change the base image for a recipe, should be taken out, since we cannot change the base image recipe after the initial selection.

BELEN: Not allowing people to change their minds about the selected base image recipe is a bit draconian. It means that if you select the wrong base image by mistake, you will be forced to discard the custom image recipe, and restart from scratch. A bit harsh. After discussing this with Paul, it turns out the agreement was the following: if you change the base image recipe, we'll drop the initial list of packages and any deletions you might have done on it (we will warn you about that before you change the base image). Ideally we would be able to remember any packages you have added, but if this is not feasible, we might be able to survive without it. I need to design this.

ALEX: When storing a custom image recipe, we have two options: store an absolute list of packages, or store a reference to a base image and a difference for the list of packages. These options are available only while working within the Toaster instance - once you export your recipe, you have to export an absolute list of packages.

Each approach has advantages and disadvantages. The functionality described above requires storing a reference + difference, in order to be able to change the base image. The reference+difference approach has the advantage of changing the package list based on the image features and current configuration, and following the upstream changes, and the disadvantage of needing parsing on each project configuration change, and possibly changing the image package list outside user's control (when the upstream base image changes).

=== Custom image summary (right column) ===

MICHAEL: I was expecting the pencil to do the same as other pencils and activate text input boxes. Obviously if you're on the Add/Delete packages page you can't change the base image like that so maybe not having these pencils would be better. I was also unsure as what the change version/licence would do.

BELEN: Most of the pencil icons will activate text input boxes, it's just that I didn't prototype the interaction because we already know how they work (we are just reusing the existing interaction). The pencils are the way you can change the name, the version and the license for your custom image recipe. The only exception is the pencil icon next to the base image recipe. That would be a link to the 'Select a base image recipe' page. I agree this is not consistent. In general, I think the idea of that image summary is good, but the content could be improved: so maybe we'll look into a better way to organise it and better labelling for the different items.

=== Adding / removing layers ===

MICHAEL: If you've selected an image recipe and then remove the layer that provides it... what happens?

BELEN: heh, didn't think of this one :) Any changes in the layer list, adding or removing, will trigger a parsing. Once the parsing is finished, we could flag the fact that the base image recipe is no longer provided by any project layers. But, if you have removed the layer that provides the base image recipe, I would like to think nothing happens, since we would have created a new .bb file that doesn't depend on the original image recipe. If we decide that the custom image recipe should inherit the base image recipe, well, then we would have a problem: we would need to force people to select a new base image recipe. Paul can probably clarify what would really happen, so I'll make sure to ask him.

----

ALEX: The "Add A Layer" panel on the right hand side, I would've expected it to be a pop-up like in the configuration page, not take me to the Compatible layers. It completely takes me out of the context of what I was doing - I want to customize a image for MinnowMax, I want to add minnowmax and be done, not go to a different page, and then have no idea how to return in a single click.

BELEN: you are right. We can probably just replicate the mechanism we have in the basic configuration page: an 'add layer' form with links to view all the layers and to import a layer. So if you know what you are doing, you can add the layer you need without exiting the image configuration page.

----

DAVID: The new page can add layers in place which is nice, and I see how you can use it to parse and show that layer's package count. However, it appears that if I change my mind about the layer I have to go out to the layer page to remove it? Could perhaps the "undo" feature also be made available here? Or a layer delete icon?

BELEN: See the [[#The 'Undo' feature]] below. You can delete layers in these pages using the 'delete' icon in the Project Layers section (right-hand column). The ability to delete layers creates other issues though, as pointed by Michael above.

----

ALEX: Also, when deleting / adding a layer in-page, the recipes available would just appear/disappear from the "available recipes" table.

BELEN: Which one is the "available recipes" table? The one listing all the base image recipes you can choose from? If yes, those don't disappear if you delete a layer. Because we will know about them from the layer index, we list them always, and we allow you to select them by adding the required layer first. When you remove a layer, what changes is the button that you see next to the base image recipe.

=== Actions ===

MICHAEL: The Build Image recipe/ Download recipe file / Delete image recipe buttons are somewhat easy to miss, they look a bit like progress buttons or other tabs. I wonder if they would be better off in '.well' 1. The Build button could be more noticeable.

ALEX: In the "My image recipe", I am not sure why the primary actions that you can take on the recipe (Build, Download, Delete) are tucked away in the right hand side, when my focus is on the left hand side, where I get drawn to the image name. Maybe we can move the buttons, make them bigger, as to be clear they are the primary action you do with a customized image ?

BELEN: agreed. I tried many options, but I wasn't particularly happy with any of them. I'll give this another go.

=== Dependency size ===

MICHAEL: In general if you can avoid having data in a table that for each one requires extra data looking up the tables will be much faster/efficient. For example we have the dependencies button with total size displayed. It should be really quick to count the number of dependences, but much slower if we also have to retrieve the objects to get the data in them, such as "name" and "size". If we can do that by making it "on demand" that's much better, e.g. you click the button and it fetches this extra data.

BELEN: this was my initial approach. From the UI perspective, it is also tidier. Then Ed asked to see this information without having to click, and I decided to give it a try to see how it would look like. I agree it is better to present it when you select a certain dependency, so I will revert to that. Sorry Ed :/

=== Displaying image contents ===

DAVID: Can we add a filter so that we can show just the packages that we can delete? The use case is trimming the image, where we want the 100 packages that we wish to examine for removal not to be lost in the possible 1000's packages that are available to add.

BELEN: yes, absolutely. The prototype does not include filters, but the intention is adding at least the one you are asking for.

----

ALEX: How do I know what's in my image in at a certain moment ?

BELEN: We will provide a filter for that, as explained in above.

----

ALEX: It would be great if I could have two tables in the image customization page - the current contents of the image, and the packages that are available. When selecting/removing a package, a package would disappear from a list and appear in the second one.

BELEN: This is a nice idea, but I don't think we have space for 2 tables. I toyed with the concept of a list of added packages that populates as you add things, to provide better visibility of your changes. This list would be on the right hand column, with the other custom image recipe information (name, version, etc). Would that do?

=== Notifications ===

DAVID: I will admit that I was thrown by the new status pop-up, because not only does it cover things up on my page, I also generally associate them with spam and ad-ware. I understand your use for showing an action in progress, but we already had a metaphor for that in the progress/status bars inserted (when applicable) at the top of the page. Why a new metaphor? What does it add that the regular model does not? I know that it does stay visible while you for example scroll, but is that a hard requirement?

BELEN: Just to be sure, I guess you are talking about the fixed-positioned notifications that appear when you perform an action (for example, adding a layer). Their main advantage over our current notification system is that they are always visible. Since these notifications are the way Toaster provides feedback to users about their actions, that's a significant advantage. They are also a relatively common way of presenting notification nowadays (Dropbox uses them, just to give a random example, but so do lots of other web apps). Another common way is "pop from top" messages, like these ones:

https://css-tricks.com/pop-from-top-notification/

They have an example here:

https://css-tricks.com/examples/PopFromTopMessage/

I gave those a quick try, but didn't play nice with the fixed top bar we are currently using in Toaster.

I can tweak the design a bit to make the fixed-positioned notifications less offensive (smaller, with less shadow, for example). If you would like to find a completely different way to handle notifications, let's open a Bugzilla entry and I can definitely do that at a later stage.

----

DAVID: I also do not understand the dangling part, where I have to manual cancel it to make it go away. For example, I understand for example its use in the add layer parsing progress, but when it is done and says "Layer Information updated" why do I have to manually kill it? Could it at least go away on its own after a moment, and let that be the clue that the process completed?

BELEN: yes, this we can definitely do. We just need to come up with the right time interval.

----

ALEX: I subscribe to David's view that the popups are annoying and unnecessary.

BELEN: they are not popups, just to be clear. Popups are modal. These things do not prevent you from interacting with the interface until you take action on them.

----

ALEX: I would suggest using box alerts as they are currently implemented, to avoid obscuring the screen, and divert user's attention. The feedback for immediate actions should not be in your face.

BELEN: well, it kind of needs to be. Feedback is incredibly important in interfaces: users must know the state of the system and the outcome of their actions. Notifications should be therefore visible (the ones we have right now not always are), and that's why so many web applications are using fix-positioned ones like these to provide feedback to users.

----

ALEX: Ditto for the "data loading" spinner, let's make it a bit more obscure.

BELEN: Michael and I spent a good deal of time trying to work out the best way to present this using the standard UI widgets that come with Bootstrap. I still think the way it is right now is the best way we can currently make it. That doesn't mean it cannot be improved. I am sure Levi will be ok with including a 'data loading' UI component in the library he is creating for Toaster.

=== The 'Undo' feature ===

DAVID: The "Undo" feature though is nice!

BELEN: "undo" features are very, very nice, I agree. I've added it here as suggested by the rest of the design contributors, but the reality is that I don't expect it to be part of the image customisation feature, that's why I don't really mention it in the video. "Undo" should probably be designed and implemented and an application-wide feature, which means it should work for pretty much all actions. This poses a certain problem for us, since "undoing" adding a layer might not be as easy as "undoing" removing a package from a custom image recipe. What I mean by this is that 'undo' requires some thought. We can open a Bugzilla entry to design an "Undo" feature for Toaster, but it will not be part of the image customisation work.

=== Layer parsing and package count ===

DAVID: Speaking of layer parsing, don't we already have an idea of the package count per layer from the "all packages" table?

BELEN: I am a bit confused by the "package count per layer" sentence. In the image customisation process, what matters is not the package count per layer, but the package count per base image recipe, which is the very important information when you need to decide which image recipe you should use as a starting point.

=== Package groups ===

DAVID: How does this interface deal with "package groups"? People will ask.

http://www.yoctoproject.org/docs/1.8/dev-manual/dev-manual.html#usingpoky-extend-customimage-customtasks

BELEN: I'm afraid it doesn't. We know that at some point we'll need to distinguish package groups from other recipes, and break them up, i.e. allow people to remove packages from package groups. But during the high level design discussions we came to the conclusion that it would be too hard to do in version 1 - see page 11 of the initial design document

https://wiki.yoctoproject.org/wiki/images/3/31/Image_customisation_in_Toaster.pdf

=== Concerns about package removal ===

DAVID: Adding packages is easy, but we (WR) have found that removing them is weirdly hard. I am very curious how the backend package removal/exclusion is being done, and who is doing it. In any case, we should have disclaimers in place.

* Some removed packages appear anyway in the image, because sometimes the dependency information in the packages are not complete nor correct.

* Some removed packages will break the build or the runtime, even though the dependencies say it should be ok. Users should be encouraged to test their changes early and often, and we may want to help save their bacon with checkpoints (based on the last successful build?) or multiple undo's so that they can back up to a working state rather than re-starting from scratch. Just saying.

BELEN: I cannot answer your question about package removal (maybe Paul?), but good to be warned about potential problems and possible solutions. They might not make version 1, but we can definitely enhance the functionality in subsequent releases.

ALEX: This is a limitation of the way metadata is structured and tested. The YP developers often miss correct dependency linking because the actual dependencies for a package (e.g. installed libraries) were already satisfied by another chain, so they miss specifying the dependency in the metadata. This problem will be re-occuring during Toaster use, and until we clean up the dependencies metadata, we need to tell the users exactly what's going on - and enable them to timely submit error reports about the invalid metadata. This is the only way we can actually clean everything up.

=== "Normal" builds vs. "custom image recipe" builds ===

ALEX: I think it should be easy to build your own recipes in the Configuration page. The text entry with the build button should be no longer the primary means of starting a build, now that we can configure the build itself. I would suggest adding a "My image recipes" box in the style of "Most build recipes" in the Configuration, with two big buttons : "Build selected" and "New image recipe".

BELEN: right, I am a bit confused about this suggestion. We decided to "isolate" the builds of custom image recipes from "normal" builds to make sure we didn't have to deal with pushing layers to Git repos. To make this separation (which is by no means ideal) as clear as possible, we agreed to create a separate "My image recipes" section, from which you cannot start "normal" builds, and from where you start only builds of you custom image recipes. If you want to integrate your custom image recipe into your "normal" builds you need to download the .bb file and add it to your custom layer, then import that custom layer into Toaster. Please read page 5 of the high-level design document available here

https://wiki.yoctoproject.org/wiki/images/3/31/Image_customisation_in_Toaster.pdf

The above is what the initial design workshop concluded. I think we are all most definitely open to new ideas, but they must involve a solution to the problem of which layer do custom image recipes go to.

----

ALEX: For a new user, it's not obvious how to start configuring the image contents, in the sense that "My image recipes" isn't the most obvious place one would start to configure a build, which is why people actually look for into Toaster. I would suggest making "Type the target you want to build" and Build button in a section similar to, and on top of the "Latest builds" section. And add, in that section, in the project page, a bit link "Configure the image contents".

BELEN: Again, I am confused. There is now a build form not only above the "latest builds" section, but also in all project configuration pages, in exactly the same place so that it remains easy to find for users. That component is removed from the "My image recipes" pages because we concluded we should separate their builds from "normal" builds as explained above. Your "configure the image contents" suggestion would conflate both types of builds. Again, this is something I would love to do, but we would need a way to handle the question of which layer do custom image recipes go to.

----

ALEX: It seems a bit strange that I can navigate to most of the Project options via the left hand menu, but not to image recipes. The user-defined image recipes are Project-specific, and I think they are at the same level with the "Image recipes" and "Software recipes", but in the "CONFIGURATION" category. I don't think that having a duplicate link as a tab is a problem, but I get frustrated by the disappearance of the left-hand-menu in the My Image Recipes.

BELEN: This is once more a consequence of the agreed separation between "normal" and custom image recipe builds. In the project page: the "builds" content spans both, the "configuration" content belongs to the "normal" builds context, and the "my image recipes" content belongs to the custom image recipes context. The latter 2 are kept separate as the outcome of the initial design workshop suggested should be done.

=== New build button ===

ALEX: Please drop the "New build" button. That button is a stop-gap measure to launching build commands, but now we're not simply issue-ing build commands, we are actually configuring the build. Even if you are an experienced user, it's opaque form - small and with very little information - makes it difficult to use.

BELEN: mmm, drop it from where? It is nowhere in the image customisation process. The purpose of the 'new build' button is allow you to build for any project from anywhere. I still think that's a handy thing to have, although I have no evidence to back that up. I am ok with reconsidering that feature, but not as part of this review, which is supposed to be about the image customisation design.

ALEX: ahm, right, dropping it from the top toolbar, next to the New Project button. Not part of the image customization though, but the thought was triggered by being in the Custom image page and seeing multiple Build buttons.

== Design ARs ==

* Find a way to show that the number of packages shown after parsing is a best guess
* In the 'my image recipes' table, see if there is a way we can see a build button in the last table column, consistent with how all other recipes are displayed
* Review the content in the 'name your image recipe' modal. The help text is too long, and the heading should match somehow the button clicked ('new image recipe'). Consider change to a page, like in projects.
* Change 'my image recipes' to 'custom image recipes'
* Change 'add / delete' to 'add / remove' across Toaster
* Talk to Michael about his suggested alternative to the image customisation page (with the name at the top as step 1, and the tabs afterwards as step 2)
* Confirm with Michael if I should remove the 'selected' badge in the base image recipes table
* Design the 'changing my base image' workflow. All the initial packages and any of them you have deleted will be lost. We should ideally keep the list of packages you added, but what if any of them already exists in the new base image? Think all this through.
* Remove the pencil next to the base image recipe in the right column summary: its behaviour would be inconsistent with all other pencils in Toaster. Also, changing the base image recipe has big impact, so it should probably not be too easy.
* Review the content of the right column summary
* Asking Paul what would happen in the image customisation process if you remove the layer providing the base image for a custom image recipe
* Add the 'add layer' form as is in the configuration page to the Project layers section in the right hand column
* Review the presentation of the main page actions (build / download / delete)
* Remove the dependency size info from the button and put it inside the popover
* Add the needed filters
* Add a list of packages added to the image summary in the right column
* Make the fixed-positioned notifications less offensive (check the shadow and size)
* Define the time interval after which the fixed notifications fade out
* Ask Levi to design a 'loading' component for the UI library

Design feedback

2015-07-06T10:59:07Z

Ddalex: /* Select a base image recipe */

== Feedback ==

=== Content and labeling ===

MICHAEL: creating an image recipe, I find the text in the modal too long, anything more than 2 sentences and my attention span is over!

BELEN: I am sure we can improve that text to make it shorter.

----

ALEX: BTW, thinking of the naming, I would suggest "Custom recipes", because it's not clear from the links that I can actually configure something there.

BELEN: Happy to review the labeling, as usual. I would add the word "image" though, because you cannot create custom software recipes or package groups: only image recipes. So it would be something like "Custom image recipes". Would that work?

----

ALEX: maybe we should change the terminology from Add/Delete?

BELEN: yes, everybody is screaming for this :) I will change it, but it needs to be changed across Toaster, not just in the image customisation process.

=== Creating an image recipe ===

MICHAEL: I found it confusing that I entered a name and clicked create but have not actually created anything

BELEN: mmm, this one is interesting. What makes you think that you have not created anything?

----

MICHAEL: what about something like suggestion1.png (attached) this brings it more into line with import layer/create new project/form based activity.

BELEN: that is quite similar to Tiago's sketches. I shied away from that approach for 2 reasons:

# Because I find it crowded and lacking on clear priorities. There is a lot of content and controls in this screen. Most of them we need to deactivate until a name for the image recipe has been set. I am sure there are things we could do in visual design to improve the balance in such a screen, but not in this first go. The question that immediately comes to my mind when I see it is: if I cannot manipulate these content and controls, why are you showing them to me? That's why I sticked to the 2-step approach: give this a name first, so that we can create the image recipe, then, once created, you can proceed to "configure" it.
#Because it meant we need 2 different custom image recipe pages: one for when you are creating the image for the first time (with the steps numbered and the name field at the top), and a second one for when you are editing the image recipe after creating it (with the name you gave before as the heading 1, and no steps). Splitting the naming step from the custom image recipe page means that we can use exactly the same page for both circumstances, which I think is good.

The above is just the rationale behind my design, just to make this clear. I believe your suggestion would also work. To figure out which one would work better, we would need to test them, and unfortunately we don't have time for that right now. But I am sure we can work together to come up with something. It would be good to hear a bit more about the reasons why you prefer your suggestion.

=== Select a base image recipe ===

MICHAEL: I was looking for a way to "de/re/select" an image, we don't quite have the same concept here as the machines selection that I was expecting, where you can always select regardless of whether it's already selected.

BELEN: just to make sure I understand, do you mean the fact that you get a 'selected' badge next to the selected base image recipe? The initial designs for machines also indicated which machine was selected with a similar badge, until it was pointed out to me that there is no way for us to know which layer is actually providing the machine you are setting. The only thing we know for sure is the name of the machine. I do not think is critical to indicate which is the selected base image recipe in the table: you already have it on the image summary (the well1 on the right hand side). So maybe we can survive without that.

----

ALEX: When configuring an image, I would suggest dropping the "Base image recipe" concept, because we can't follow on updates from the base image recipe after the custom recipe was created, and this will confuse the user.

BELEN: and what do users do instead? Do they start they custom image recipes from a blank slate? That will most likely result in an image that doesn't build. Do you have an alternative starting point in mind?

ALEX: The users would start from an already existing image, as they do now. What I'm suggesting is dropping the tab of possible base images, and the ability to change the base image for a custom image once the custom image recipe is created. This is the same point as below, just pointing to the tab.

----

ALEX: Also, the tab with the Base image recipe in My Image details, allowing the user to change the base image for a recipe, should be taken out, since we cannot change the base image recipe after the initial selection.

BELEN: Not allowing people to change their minds about the selected base image recipe is a bit draconian. It means that if you select the wrong base image by mistake, you will be forced to discard the custom image recipe, and restart from scratch. A bit harsh. After discussing this with Paul, it turns out the agreement was the following: if you change the base image recipe, we'll drop the initial list of packages and any deletions you might have done on it (we will warn you about that before you change the base image). Ideally we would be able to remember any packages you have added, but if this is not feasible, we might be able to survive without it. I need to design this.

ALEX: When storing a custom image recipe, we have two options: store an absolute list of packages, or store a reference to a base image and a difference for the list of packages. These options are available only while working within the Toaster instance - once you export your recipe, you have to export an absolute list of packages.

Each approach has advantages and disadvantages. The functionality described above requires storing a reference + difference, in order to be able to change the base image. The reference+difference approach has the advantage of changing the package list based on the image features and current configuration, and following the upstream changes, and the disadvantage of needing parsing on each project configuration change, and possibly changing the image package list outside user's control (when the upstream base image changes).

=== Custom image summary (right column) ===

MICHAEL: I was expecting the pencil to do the same as other pencils and activate text input boxes. Obviously if you're on the Add/Delete packages page you can't change the base image like that so maybe not having these pencils would be better. I was also unsure as what the change version/licence would do.

BELEN: Most of the pencil icons will activate text input boxes, it's just that I didn't prototype the interaction because we already know how they work (we are just reusing the existing interaction). The pencils are the way you can change the name, the version and the license for your custom image recipe. The only exception is the pencil icon next to the base image recipe. That would be a link to the 'Select a base image recipe' page. I agree this is not consistent. In general, I think the idea of that image summary is good, but the content could be improved: so maybe we'll look into a better way to organise it and better labelling for the different items.

=== Adding / removing layers ===

MICHAEL: If you've selected an image recipe and then remove the layer that provides it... what happens?

BELEN: heh, didn't think of this one :) Any changes in the layer list, adding or removing, will trigger a parsing. Once the parsing is finished, we could flag the fact that the base image recipe is no longer provided by any project layers. But, if you have removed the layer that provides the base image recipe, I would like to think nothing happens, since we would have created a new .bb file that doesn't depend on the original image recipe. If we decide that the custom image recipe should inherit the base image recipe, well, then we would have a problem: we would need to force people to select a new base image recipe. Paul can probably clarify what would really happen, so I'll make sure to ask him.

----

ALEX: The "Add A Layer" panel on the right hand side, I would've expected it to be a pop-up like in the configuration page, not take me to the Compatible layers. It completely takes me out of the context of what I was doing - I want to customize a image for MinnowMax, I want to add minnowmax and be done, not go to a different page, and then have no idea how to return in a single click.

BELEN: you are right. We can probably just replicate the mechanism we have in the basic configuration page: an 'add layer' form with links to view all the layers and to import a layer. So if you know what you are doing, you can add the layer you need without exiting the image configuration page.

----

DAVID: The new page can add layers in place which is nice, and I see how you can use it to parse and show that layer's package count. However, it appears that if I change my mind about the layer I have to go out to the layer page to remove it? Could perhaps the "undo" feature also be made available here? Or a layer delete icon?

BELEN: See the [[#The 'Undo' feature]] below. You can delete layers in these pages using the 'delete' icon in the Project Layers section (right-hand column). The ability to delete layers creates other issues though, as pointed by Michael above.

----

ALEX: Also, when deleting / adding a layer in-page, the recipes available would just appear/disappear from the "available recipes" table.

BELEN: Which one is the "available recipes" table? The one listing all the base image recipes you can choose from? If yes, those don't disappear if you delete a layer. Because we will know about them from the layer index, we list them always, and we allow you to select them by adding the required layer first. When you remove a layer, what changes is the button that you see next to the base image recipe.

=== Actions ===

MICHAEL: The Build Image recipe/ Download recipe file / Delete image recipe buttons are somewhat easy to miss, they look a bit like progress buttons or other tabs. I wonder if they would be better off in '.well' 1. The Build button could be more noticeable.

ALEX: In the "My image recipe", I am not sure why the primary actions that you can take on the recipe (Build, Download, Delete) are tucked away in the right hand side, when my focus is on the left hand side, where I get drawn to the image name. Maybe we can move the buttons, make them bigger, as to be clear they are the primary action you do with a customized image ?

BELEN: agreed. I tried many options, but I wasn't particularly happy with any of them. I'll give this another go.

=== Dependency size ===

MICHAEL: In general if you can avoid having data in a table that for each one requires extra data looking up the tables will be much faster/efficient. For example we have the dependencies button with total size displayed. It should be really quick to count the number of dependences, but much slower if we also have to retrieve the objects to get the data in them, such as "name" and "size". If we can do that by making it "on demand" that's much better, e.g. you click the button and it fetches this extra data.

BELEN: this was my initial approach. From the UI perspective, it is also tidier. Then Ed asked to see this information without having to click, and I decided to give it a try to see how it would look like. I agree it is better to present it when you select a certain dependency, so I will revert to that. Sorry Ed :/

=== Displaying image contents ===

DAVID: Can we add a filter so that we can show just the packages that we can delete? The use case is trimming the image, where we want the 100 packages that we wish to examine for removal not to be lost in the possible 1000's packages that are available to add.

BELEN: yes, absolutely. The prototype does not include filters, but the intention is adding at least the one you are asking for.

----

ALEX: How do I know what's in my image in at a certain moment ?

BELEN: We will provide a filter for that, as explained in above.

----

ALEX: It would be great if I could have two tables in the image customization page - the current contents of the image, and the packages that are available. When selecting/removing a package, a package would disappear from a list and appear in the second one.

BELEN: This is a nice idea, but I don't think we have space for 2 tables. I toyed with the concept of a list of added packages that populates as you add things, to provide better visibility of your changes. This list would be on the right hand column, with the other custom image recipe information (name, version, etc). Would that do?

=== Notifications ===

DAVID: I will admit that I was thrown by the new status pop-up, because not only does it cover things up on my page, I also generally associate them with spam and ad-ware. I understand your use for showing an action in progress, but we already had a metaphor for that in the progress/status bars inserted (when applicable) at the top of the page. Why a new metaphor? What does it add that the regular model does not? I know that it does stay visible while you for example scroll, but is that a hard requirement?

BELEN: Just to be sure, I guess you are talking about the fixed-positioned notifications that appear when you perform an action (for example, adding a layer). Their main advantage over our current notification system is that they are always visible. Since these notifications are the way Toaster provides feedback to users about their actions, that's a significant advantage. They are also a relatively common way of presenting notification nowadays (Dropbox uses them, just to give a random example, but so do lots of other web apps). Another common way is "pop from top" messages, like these ones:

https://css-tricks.com/pop-from-top-notification/

They have an example here:

https://css-tricks.com/examples/PopFromTopMessage/

I gave those a quick try, but didn't play nice with the fixed top bar we are currently using in Toaster.

I can tweak the design a bit to make the fixed-positioned notifications less offensive (smaller, with less shadow, for example). If you would like to find a completely different way to handle notifications, let's open a Bugzilla entry and I can definitely do that at a later stage.

----

DAVID: I also do not understand the dangling part, where I have to manual cancel it to make it go away. For example, I understand for example its use in the add layer parsing progress, but when it is done and says "Layer Information updated" why do I have to manually kill it? Could it at least go away on its own after a moment, and let that be the clue that the process completed?

BELEN: yes, this we can definitely do. We just need to come up with the right time interval.

----

ALEX: I subscribe to David's view that the popups are annoying and unnecessary.

BELEN: they are not popups, just to be clear. Popups are modal. These things do not prevent you from interacting with the interface until you take action on them.

----

ALEX: I would suggest using box alerts as they are currently implemented, to avoid obscuring the screen, and divert user's attention. The feedback for immediate actions should not be in your face.

BELEN: well, it kind of needs to be. Feedback is incredibly important in interfaces: users must know the state of the system and the outcome of their actions. Notifications should be therefore visible (the ones we have right now not always are), and that's why so many web applications are using fix-positioned ones like these to provide feedback to users.

----

ALEX: Ditto for the "data loading" spinner, let's make it a bit more obscure.

BELEN: Michael and I spent a good deal of time trying to work out the best way to present this using the standard UI widgets that come with Bootstrap. I still think the way it is right now is the best way we can currently make it. That doesn't mean it cannot be improved. I am sure Levi will be ok with including a 'data loading' UI component in the library he is creating for Toaster.

=== The 'Undo' feature ===

DAVID: The "Undo" feature though is nice!

BELEN: "undo" features are very, very nice, I agree. I've added it here as suggested by the rest of the design contributors, but the reality is that I don't expect it to be part of the image customisation feature, that's why I don't really mention it in the video. "Undo" should probably be designed and implemented and an application-wide feature, which means it should work for pretty much all actions. This poses a certain problem for us, since "undoing" adding a layer might not be as easy as "undoing" removing a package from a custom image recipe. What I mean by this is that 'undo' requires some thought. We can open a Bugzilla entry to design an "Undo" feature for Toaster, but it will not be part of the image customisation work.

=== Layer parsing and package count ===

DAVID: Speaking of layer parsing, don't we already have an idea of the package count per layer from the "all packages" table?

BELEN: I am a bit confused by the "package count per layer" sentence. In the image customisation process, what matters is not the package count per layer, but the package count per base image recipe, which is the very important information when you need to decide which image recipe you should use as a starting point.

=== Package groups ===

DAVID: How does this interface deal with "package groups"? People will ask.

http://www.yoctoproject.org/docs/1.8/dev-manual/dev-manual.html#usingpoky-extend-customimage-customtasks

BELEN: I'm afraid it doesn't. We know that at some point we'll need to distinguish package groups from other recipes, and break them up, i.e. allow people to remove packages from package groups. But during the high level design discussions we came to the conclusion that it would be too hard to do in version 1 - see page 11 of the initial design document

https://wiki.yoctoproject.org/wiki/images/3/31/Image_customisation_in_Toaster.pdf

=== Concerns about package removal ===

DAVID: Adding packages is easy, but we (WR) have found that removing them is weirdly hard. I am very curious how the backend package removal/exclusion is being done, and who is doing it. In any case, we should have disclaimers in place.

* Some removed packages appear anyway in the image, because sometimes the dependency information in the packages are not complete nor correct.

* Some removed packages will break the build or the runtime, even though the dependencies say it should be ok. Users should be encouraged to test their changes early and often, and we may want to help save their bacon with checkpoints (based on the last successful build?) or multiple undo's so that they can back up to a working state rather than re-starting from scratch. Just saying.

BELEN: I cannot answer your question about package removal (maybe Paul?), but good to be warned about potential problems and possible solutions. They might not make version 1, but we can definitely enhance the functionality in subsequent releases.

ALEX: This is a limitation of the way metadata is structured and tested. The YP developers often miss correct dependency linking because the actual dependencies for a package (e.g. installed libraries) were already satisfied by another chain, so they miss specifying the dependency in the metadata. This problem will be re-occuring during Toaster use, and until we clean up the dependencies metadata, we need to tell the users exactly what's going on - and enable them to timely submit error reports about the invalid metadata. This is the only way we can actually clean everything up.

=== "Normal" builds vs. "custom image recipe" builds ===

ALEX: I think it should be easy to build your own recipes in the Configuration page. The text entry with the build button should be no longer the primary means of starting a build, now that we can configure the build itself. I would suggest adding a "My image recipes" box in the style of "Most build recipes" in the Configuration, with two big buttons : "Build selected" and "New image recipe".

BELEN: right, I am a bit confused about this suggestion. We decided to "isolate" the builds of custom image recipes from "normal" builds to make sure we didn't have to deal with pushing layers to Git repos. To make this separation (which is by no means ideal) as clear as possible, we agreed to create a separate "My image recipes" section, from which you cannot start "normal" builds, and from where you start only builds of you custom image recipes. If you want to integrate your custom image recipe into your "normal" builds you need to download the .bb file and add it to your custom layer, then import that custom layer into Toaster. Please read page 5 of the high-level design document available here

https://wiki.yoctoproject.org/wiki/images/3/31/Image_customisation_in_Toaster.pdf

The above is what the initial design workshop concluded. I think we are all most definitely open to new ideas, but they must involve a solution to the problem of which layer do custom image recipes go to.

----

ALEX: For a new user, it's not obvious how to start configuring the image contents, in the sense that "My image recipes" isn't the most obvious place one would start to configure a build, which is why people actually look for into Toaster. I would suggest making "Type the target you want to build" and Build button in a section similar to, and on top of the "Latest builds" section. And add, in that section, in the project page, a bit link "Configure the image contents".

BELEN: Again, I am confused. There is now a build form not only above the "latest builds" section, but also in all project configuration pages, in exactly the same place so that it remains easy to find for users. That component is removed from the "My image recipes" pages because we concluded we should separate their builds from "normal" builds as explained above. Your "configure the image contents" suggestion would conflate both types of builds. Again, this is something I would love to do, but we would need a way to handle the question of which layer do custom image recipes go to.

----

ALEX: It seems a bit strange that I can navigate to most of the Project options via the left hand menu, but not to image recipes. The user-defined image recipes are Project-specific, and I think they are at the same level with the "Image recipes" and "Software recipes", but in the "CONFIGURATION" category. I don't think that having a duplicate link as a tab is a problem, but I get frustrated by the disappearance of the left-hand-menu in the My Image Recipes.

BELEN: This is once more a consequence of the agreed separation between "normal" and custom image recipe builds. In the project page: the "builds" content spans both, the "configuration" content belongs to the "normal" builds context, and the "my image recipes" content belongs to the custom image recipes context. The latter 2 are kept separate as the outcome of the initial design workshop suggested should be done.

=== New build button ===

ALEX: Please drop the "New build" button. That button is a stop-gap measure to launching build commands, but now we're not simply issue-ing build commands, we are actually configuring the build. Even if you are an experienced user, it's opaque form - small and with very little information - makes it difficult to use.

BELEN: mmm, drop it from where? It is nowhere in the image customisation process. The purpose of the 'new build' button is allow you to build for any project from anywhere. I still think that's a handy thing to have, although I have no evidence to back that up. I am ok with reconsidering that feature, but not as part of this review, which is supposed to be about the image customisation design.

== Design ARs ==

* Find a way to show that the number of packages shown after parsing is a best guess
* In the 'my image recipes' table, see if there is a way we can see a build button in the last table column, consistent with how all other recipes are displayed
* Review the content in the 'name your image recipe' modal. The help text is too long, and the heading should match somehow the button clicked ('new image recipe'). Consider change to a page, like in projects.
* Change 'my image recipes' to 'custom image recipes'
* Change 'add / delete' to 'add / remove' across Toaster
* Talk to Michael about his suggested alternative to the image customisation page (with the name at the top as step 1, and the tabs afterwards as step 2)
* Confirm with Michael if I should remove the 'selected' badge in the base image recipes table
* Design the 'changing my base image' workflow. All the initial packages and any of them you have deleted will be lost. We should ideally keep the list of packages you added, but what if any of them already exists in the new base image? Think all this through.
* Remove the pencil next to the base image recipe in the right column summary: its behaviour would be inconsistent with all other pencils in Toaster. Also, changing the base image recipe has big impact, so it should probably not be too easy.
* Review the content of the right column summary
* Asking Paul what would happen in the image customisation process if you remove the layer providing the base image for a custom image recipe
* Add the 'add layer' form as is in the configuration page to the Project layers section in the right hand column
* Review the presentation of the main page actions (build / download / delete)
* Remove the dependency size info from the button and put it inside the popover
* Add the needed filters
* Add a list of packages added to the image summary in the right column
* Make the fixed-positioned notifications less offensive (check the shadow and size)
* Define the time interval after which the fixed notifications fade out
* Ask Levi to design a 'loading' component for the UI library

File:Toasterconf.json.txt.patch

2015-02-09T17:39:55Z

Ddalex: uploaded a new version of "File:Toasterconf.json.txt.patch": Fixed to proper JSON

Sample configuration file for setting up hosted-version managed-mode Toaster.

Available as a patch.

Toaster

2014-11-26T14:02:08Z

Ddalex: /* Using Toaster */

[[Category:Toaster]]
[[Toaster]] is a web-based interface to OpenEmbedded and BitBake. You might have heard it is a replacement for [https://www.yoctoproject.org/documentation/hob-manual Hob]. The answer is "it will be at some point, but not yet".

General discussion about '''Toaster''' happens on a dedicated mailing list: [https://lists.yoctoproject.org/listinfo/toaster https://lists.yoctoproject.org/listinfo/toaster]

=== Using Toaster ===

Toaster can run in various modes and setups. To ease out the understanding of documentation, we review here the terminology used throughout the documentation.

* Interactive mode - this is the mode released with Yocto Project Release 1.6. Functional components consist of a build recording component, 'toasterui', and a build stats and inspection user interface, the 'toastergui'. It is started with the command sequence listed below, and the builds are started using normal bitbake command line

$ source oe-init-build-env
$ source toaster start

* Managed mode - in this mode Toaster handles the build configuration GUI (through Project pagess), and build scheduling and execution, in addition to the features launched with Yocto Project Release 1.6. The builds are triggered through the web interface,
the user as no direct access to the bitbake command line.

** Local managed mode, in short _local_, is the out-of-box mode available after a poky/ checkout. It allows the user to perform build on his local machine source code, with a local build directory, and is intended to help the user discover the interface, and configure and run local builds. You can launch it with

$ ./bitbake/bin/toaster

** Remote managed mode, or [[hosted Toaster]], is intended to be the production setup for running Toaster in organizations supporting multiple users and using customized Toaster installations.

=== Toaster How-to's ===

Specific pages with Toaster how-tos are available below.

* [[Setting up a local instance of Toaster]]
* [[Setting up a production instance of Toaster]] - documentation for Interactive mode, for building with the Autobuilder/BuildBot/Jenkins
* [[Setting up a hosted managed mode for Toaster]] - configure and run build through Toaster itself, providing a service for your users
* [https://www.yoctoproject.org/documentation/toaster-manual-17 How to use the Toaster web interface]
* [[How to delete information from the Toaster database]]

=== About Toaster ===

* [[Contribute to Toaster]]
* [https://bugzilla.yoctoproject.org/buglist.cgi?list_id=213820&columnlist=status_whiteboard%2Cassigned_to%2Ctarget_milestone%2Cbug_status%2Cshort_desc%2Cbug_severity%2Cpriority&classification=Build%20System%20%26%20Metadata&query_based_on=Toaster-Opens&query_format=advanced&bug_status=NEW&bug_status=ACCEPTED&bug_status=IN%20PROGRESS%20DESIGN&bug_status=IN%20PROGRESS%20DESIGN%20COMPLETE&bug_status=IN%20PROGRESS%20IMPLEMENTATION&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=WaitForUpstream&component=toaster&product=Toaster&known_name=Toaster-Opens Bug list]
* [[Toaster architecture design]]
* [[Toaster testing plan]]
* [[Toaster 1.7 design | Scope and design]]
* [[Toaster archive | Archive]]

=== In progress documentation ===

We are currently preparing the documentation for the Toaster build functionality. The content here is just a brain dump of what we need to cover (in no particular order). Feel free to add and create content as you see fit:

*[[Using virtualenv]]
*[[Toaster configuration]] - explain releases, layer sources, BitBake versions and default project configurations
*[[Set up Toaster locally]] - explain the set up process and the local version of the localconf.json file
*[[Set up production instance]] - explain the Django admin interface and the remove version of the localconf.json file
*[[manage.py commands]] - this should include an explanation of lsupdates
*[[Custom layer index]] - explain how to create your own layer index
*[[Start Toaster in managed mode]]

Toaster

2014-11-26T14:01:51Z

Ddalex:

[[Category:Toaster]]
[[Toaster]] is a web-based interface to OpenEmbedded and BitBake. You might have heard it is a replacement for [https://www.yoctoproject.org/documentation/hob-manual Hob]. The answer is "it will be at some point, but not yet".

General discussion about '''Toaster''' happens on a dedicated mailing list: [https://lists.yoctoproject.org/listinfo/toaster https://lists.yoctoproject.org/listinfo/toaster]

=== Using Toaster ===

Toaster can run in various modes and setups. To ease out the understanding of documentation, we review here the terminology used throughout the documentation.

* Interactive mode - this is the mode released with Yocto Project Release 1.6. Functional components consist of a build recording component, 'toasterui', and a build stats and inspection user interface, the 'toastergui'. It is started with the command sequence listed below, and the builds are started using normal bitbake command line

$ source oe-init-build-env
$ source toaster start

* Managed mode - in this mode Toaster handles the build configuration GUI (through Project pagess), and build scheduling and execution, in addition to the features launched with Yocto Project Release 1.6. The builds are triggered through the web interface,
the user as no direct access to the bitbake command line.

** Local managed mode, in short _local_, is the out-of-box mode available after a poky/ checkout. It allows the user to perform build on his local machine source code, with a local build directory, and is intended to help the user discover the interface, and configure and run local builds. You can launch it with

$ ./bitbake/bin/toaster

** Remote managed mode, or [[hosted Toaster]], is intended to be the production setup for running Toaster in organizations supporting multiple users and using customized Toaster installations.

=== Toaster How-to

Specific pages with Toaster how-tos are available below.

* [[Setting up a local instance of Toaster]]
* [[Setting up a production instance of Toaster]] - documentation for Interactive mode, for building with the Autobuilder/BuildBot/Jenkins
* [[Setting up a hosted managed mode for Toaster]] - configure and run build through Toaster itself, providing a service for your users
* [https://www.yoctoproject.org/documentation/toaster-manual-17 How to use the Toaster web interface]
* [[How to delete information from the Toaster database]]

=== About Toaster ===

* [[Contribute to Toaster]]
* [https://bugzilla.yoctoproject.org/buglist.cgi?list_id=213820&columnlist=status_whiteboard%2Cassigned_to%2Ctarget_milestone%2Cbug_status%2Cshort_desc%2Cbug_severity%2Cpriority&classification=Build%20System%20%26%20Metadata&query_based_on=Toaster-Opens&query_format=advanced&bug_status=NEW&bug_status=ACCEPTED&bug_status=IN%20PROGRESS%20DESIGN&bug_status=IN%20PROGRESS%20DESIGN%20COMPLETE&bug_status=IN%20PROGRESS%20IMPLEMENTATION&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=WaitForUpstream&component=toaster&product=Toaster&known_name=Toaster-Opens Bug list]
* [[Toaster architecture design]]
* [[Toaster testing plan]]
* [[Toaster 1.7 design | Scope and design]]
* [[Toaster archive | Archive]]

=== In progress documentation ===

We are currently preparing the documentation for the Toaster build functionality. The content here is just a brain dump of what we need to cover (in no particular order). Feel free to add and create content as you see fit:

*[[Using virtualenv]]
*[[Toaster configuration]] - explain releases, layer sources, BitBake versions and default project configurations
*[[Set up Toaster locally]] - explain the set up process and the local version of the localconf.json file
*[[Set up production instance]] - explain the Django admin interface and the remove version of the localconf.json file
*[[manage.py commands]] - this should include an explanation of lsupdates
*[[Custom layer index]] - explain how to create your own layer index
*[[Start Toaster in managed mode]]

Setting up a production instance of Toaster

2014-11-26T13:53:26Z

Ddalex: /* 2. Enable build logging to the common SQL server for each build directory you are using */

[[Category:Toaster]]
This page explains how to set up Toaster separately from the hardware running your Yocto Project builds. Do it this way if you want to share a single instance of Toaster across multiple users and build servers.

== Setting up a Toaster Instance on a Remote Host ==

Under normal circumstances, starting Toaster causes three things happen:

* A BitBake server starts

* The Toaster UI starts, which connects to the BitBake server on one side and to the SQL database on the other side

* The web server starts, which reads the database and displays the web user interface

Situations exist, however, where you might want to have multiple instances of Toaster running on various remote machines. You can create this situation by basically modifying how Toaster starts and where the common SQL database resides. You are able to do this because it is not required that Toaster starts the above set of components in order to run. Minimally, an instance of Toaster requires just one of the components to run. Consequently, you are free to manually start as many or few of the components as you need rather than using the Toaster script to cause all three things to happen.

The concepts for setting up multiple instances of Toaster revolve around maintaining a common SQL database and Web server that show data from that common database and then setting up separate instances of BitBake servers and Toaster user interfaces for each separate BitBake build directory. Note that the common SQL database and the Web server show data from all the various BitBake builds. Setting the SQL database outside of any BitBake build directories maintains a
separation layer between the various builds.

The database is persistent because the logging database is set up external to the database server (e.g. MySQL). It is not even necessary to run the BitBake servers, the SQL server, and the Web server on the same machine. Each component can be run on its own machine.

== Steps to get set up ==

=== 1. Set up the SQL Logging Server and the Web Server ===

You can use any SQL server out of the box (e.g. <code>apt-get install mysgl-server</code> works for an Ubuntu system). If you are concerned about performance, you might want to hand-tune the server. You must set up proper username and password access for the server. You need administration rights for the mysql root account. Realize that this is not the same thing as root access on the machine.

Clone a separate, local Git repository of the Toaster master branch to use for running the Web server. You do not perform builds on this tree. You need to create this local repository away from any build areas.

In the separately cloned tree for the Web server, edit the <code>bitbake/lib/toaster/toastermain/settings.py</code> file so that the <code>DATABASES</code> value points to the previously created database server. Use the username and password you established earlier.

Run the database sync scripts to create the needed tables as follows:

$ python bitbake/lib/toaster/manage.py syncdb
$ python bitbake/lib/toaster/manage.py migrate orm

Note: It is important, for toaster running in 1.6 mode, to not sync bldcontrol (only used in 1.7 mode).

Start the Web server using the following command:

$ python bitbake/lib/toaster/manage.py runserver

... or in the background:

$ nohup python bitbake/lib/toaster/manage.py runserver 2>toaster_web.log >toaster_web.log &

=== 2. Enable build logging to the common SQL server for each build directory you are using ===

Edit <code>_build local_ bitbake/lib/toaster/toastermain/settings.py</code> to alter the <code>DATABASES</code> value to point to the common SQL logging server.

Create the required conf/toaster.conf file as per Bitbake extra options on "https://wiki.yoctoproject.org/wiki/Setting_up_a_local_instance_of_Toaster".

Start the BitBake server using the following command:

$ bitbake --postread conf/toaster.conf --server-only -t xmlrpc -B localhost:0 && export BBSERVER=localhost:-1

Start the logging user interface using the following command:

$ nohup bitbake --observe-only -u toasterui >toaster_ui.log &

NOTE: No hard-coded ports are used as there is enough code to run autodiscovery for ports to prevent collisions.

At this point, you are ready to run your builds using commands such as:

$ bitbake core-image-minimal

When you are finished, you need to kill the BitBake server for that particular build area:

$ bitbake -m

=== 3. Verify that it all works ===

You should examine the logs and be sure that the logging worked, that data is persistent, and that data from multiple builds from different areas was supported.

Toaster

2014-11-17T18:15:04Z

Ddalex: /* Using Toaster */

[[Category:Toaster]]
[[Toaster]] is a web-based interface to OpenEmbedded and BitBake. You might have heard it is a replacement for [https://www.yoctoproject.org/documentation/hob-manual Hob]. The answer is "it will be at some point, but not yet".

General discussion about '''Toaster''' happens on a dedicated mailing list: [https://lists.yoctoproject.org/listinfo/toaster https://lists.yoctoproject.org/listinfo/toaster]

=== Using Toaster ===

Toaster can run in various modes and setups. To ease out the understanding of documentation, we review here the terminology used throughout the documentation.

* Interactive mode - this is the mode released with Yocto Project Release 1.6. Functional components consist of a build recording component, 'toasterui', and a build stats and inspection user interface, the 'toastergui'. It is started with the command sequence listed below, and the builds are started using normal bitbake command line

$ source oe-init-build-env
$ source toaster start

* Managed mode - in this mode Toaster handles the build configuration GUI (through Project pagess), and build scheduling and execution, in addition to the features launched with Yocto Project Release 1.6. The builds are triggered through the web interface,
the user as no direct access to the bitbake command line.

** Local managed mode, in short _local_, is the out-of-box mode available after a poky/ checkout. It allows the user to perform build on his local machine source code, with a local build directory, and is intended to help the user discover the interface, and configure and run local builds. You can launch it with

$ ./bitbake/bin/toaster

** Remote managed mode, or [[hosted Toaster]], is intended to be the production setup for running Toaster in organizations supporting multiple users and using customized Toaster installations.

Specific pages with Toaster how-tos are available below.

* [[Setting up a local instance of Toaster]]
* [[Setting up a production instance of Toaster]]
* [https://www.yoctoproject.org/documentation/toaster-manual-17 How to use the Toaster web interface]
* [[How to delete information from the Toaster database]]

=== About Toaster ===

* [[Contribute to Toaster]]
* [https://bugzilla.yoctoproject.org/buglist.cgi?list_id=213820&columnlist=status_whiteboard%2Cassigned_to%2Ctarget_milestone%2Cbug_status%2Cshort_desc%2Cbug_severity%2Cpriority&classification=Build%20System%20%26%20Metadata&query_based_on=Toaster-Opens&query_format=advanced&bug_status=NEW&bug_status=ACCEPTED&bug_status=IN%20PROGRESS%20DESIGN&bug_status=IN%20PROGRESS%20DESIGN%20COMPLETE&bug_status=IN%20PROGRESS%20IMPLEMENTATION&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=WaitForUpstream&component=toaster&product=Toaster&known_name=Toaster-Opens Bug list]
* [[Toaster architecture design]]
* [[Toaster testing plan]]
* [[Toaster 1.7 design | Scope and design]]
* [[Toaster archive | Archive]]

=== In progress documentation ===

We are currently preparing the documentation for the Toaster build functionality. The content here is just a brain dump of what we need to cover (in no particular order). Feel free to add and create content as you see fit:

*[[Using virtualenv]]
*[[Toaster configuration]] - explain releases, layer sources, BitBake versions and default project configurations
*[[Set up Toaster locally]] - explain the set up process and the local version of the localconf.json file
*[[Set up production instance]] - explain the Django admin interface and the remove version of the localconf.json file
*[[manage.py commands]] - this should include an explanation of lsupdates
*[[Custom layer index]] - explain how to create your own layer index
*[[Start Toaster in managed mode]]

Main Page

2014-11-17T17:47:49Z

Ddalex:

<h1>Welcome to the Yocto Project Wiki!</h1>

== Project planning ==

=== 1.8 (Current) Project Status and Schedule ===

* [[Yocto 1.8 Schedule]]

=== 1.7 Project Status and Schedule ===
* [[Yocto 1.7 Status]]
* [[Yocto 1.7 Schedule]]
* [[1.7 qa run history]]
* [[1.7 QA Status]]
* [[meta-intel-2.0-dizzy-1.7 release planning]]

== Wiki reference sitemap ==

=== Main Wiki Sections ===

* [[Planning and Governance]]
* [[Community Guidelines]]
* [[Yocto Release Engineering | Yocto Project Release Engineering]]
* [[Processes and Activities]]
** [[YoctoCalendar]]
* [[Projects]]
* [[Yocto Interest Groups]]
* [[FAQ]]
* [[How do I]]
* [[Contributors]]
* [[Training]]
* [[Testopia]] - The Yocto Project's community-opened test case management platform

* [[Toaster]] - the web interface | [[Web UI]] - The design plans for the new Web UI

== Other resources ==
* [http://yoctoproject.org Yocto Project Front Page]

Main Page

2014-11-17T17:46:49Z

Ddalex: /* 1.8 (Current) Project Status and Schedule */

== Welcome to the Yocto Project Wiki! ==

=== 1.8 (Current) Project Status and Schedule ===

* [[Yocto 1.8 Schedule]]

=== 1.7 Project Status and Schedule ===
* [[Yocto 1.7 Status]]
* [[Yocto 1.7 Schedule]]
* [[1.7 qa run history]]
* [[1.7 QA Status]]
* [[meta-intel-2.0-dizzy-1.7 release planning]]

=== Main Wiki Sections ===
* [[Planning and Governance]]
* [[Community Guidelines]]
* [[Yocto Release Engineering | Yocto Project Release Engineering]]
* [[Processes and Activities]]
** [[YoctoCalendar]]
* [[Projects]]
* [[Yocto Interest Groups]]
* [[FAQ]]
* [[How do I]]
* [[Contributors]]
* [[Training]]
* [[Testopia]] - The Yocto Project's community-opened test case management platform

* [[Toaster]] - the web interface | [[Web UI]] - The design plans for the new Web UI

== Other resources ==
* [http://yoctoproject.org Yocto Project Front Page]

Main Page

2014-11-17T17:46:36Z

Ddalex: /* Welcome to the Yocto Project Wiki! */

== Welcome to the Yocto Project Wiki! ==

=== 1.8 (Current) Project Status and Schedule ===
* [[Yocto 1.8 Status]]
* [[Yocto 1.8 Schedule]]
* [[1.8 qa run history]]
* [[1.8 QA Status]]

=== 1.7 Project Status and Schedule ===
* [[Yocto 1.7 Status]]
* [[Yocto 1.7 Schedule]]
* [[1.7 qa run history]]
* [[1.7 QA Status]]
* [[meta-intel-2.0-dizzy-1.7 release planning]]

=== Main Wiki Sections ===
* [[Planning and Governance]]
* [[Community Guidelines]]
* [[Yocto Release Engineering | Yocto Project Release Engineering]]
* [[Processes and Activities]]
** [[YoctoCalendar]]
* [[Projects]]
* [[Yocto Interest Groups]]
* [[FAQ]]
* [[How do I]]
* [[Contributors]]
* [[Training]]
* [[Testopia]] - The Yocto Project's community-opened test case management platform

* [[Toaster]] - the web interface | [[Web UI]] - The design plans for the new Web UI

== Other resources ==
* [http://yoctoproject.org Yocto Project Front Page]

Main Page

2014-11-17T17:45:52Z

Ddalex: /* Current Project Status and Schedule */

== Welcome to the Yocto Project Wiki! ==
=== 1.7 Project Status and Schedule ===
* [[Yocto 1.7 Status]]
* [[Yocto 1.7 Schedule]]
* [[1.7 qa run history]]
* [[1.7 QA Status]]
* [[meta-intel-2.0-dizzy-1.7 release planning]]

=== Main Wiki Sections ===
* [[Planning and Governance]]
* [[Community Guidelines]]
* [[Yocto Release Engineering | Yocto Project Release Engineering]]
* [[Processes and Activities]]
** [[YoctoCalendar]]
* [[Projects]]
* [[Yocto Interest Groups]]
* [[FAQ]]
* [[How do I]]
* [[Contributors]]
* [[Training]]
* [[Testopia]] - The Yocto Project's community-opened test case management platform
* [[Toaster]] - the web interface | [[Web UI]] - The design plans for the new Web UI

== Other resources ==
* [http://yoctoproject.org Yocto Project Front Page]

Using virtualenv

2014-11-17T16:38:09Z

Ddalex: Created page with "Category:Toaster Toaster brings in extra Python dependencies that the Bitbake core doesn't need in order to run. As such, it is expected that some Python packages required b..."

[[Category:Toaster]]

Toaster brings in extra Python dependencies that the Bitbake core doesn't need in order to run. As such, it is expected that some Python packages required by Toaster may not be installed on the systems that currently run Bitbake without issues.

In order to make it easy to run Toaster, a requirements file is provided; this file, named "toaster-requirements.txt", in the root directory of bitbake, lists dependencies in a pip install-compatible format.

The recommend usage is to install these dependencies in a Python virtual environment using pip.

== How to ==

1. create a virtual environment and activate it:

$ virtualenv venv
$ source venv/bin/activate

2. use pip to install needed packages

$ pip install -r bitbake/toaster-requirements.txt

3. use toaster normally, either in

* interactive mode

$ source oe-init-build-env
$ source toaster start

* managed mode

$ bitbake/bin/toaster

Happy virtualenv-ing !

File:Toasterconf.json.txt.patch

2014-11-06T13:36:33Z

Ddalex: Sample configuration file for setting up hosted-version managed-mode Toaster. Available as a patch.

Sample configuration file for setting up hosted-version managed-mode Toaster.

Available as a patch.

Setting up a production instance of Toaster

2014-11-03T11:56:59Z

Ddalex: /* 1. Set up the SQL Logging Server and the Web Server */

[[Category:Toaster]]
This page explains how to set up Toaster separately from the hardware running your Yocto Project builds. Do it this way if you want to share a single instance of Toaster across multiple users and build servers.

== Setting up a Toaster Instance on a Remote Host ==

Under normal circumstances, starting Toaster causes three things happen:

* A BitBake server starts

* The Toaster UI starts, which connects to the BitBake server on one side and to the SQL database on the other side

* The web server starts, which reads the database and displays the web user interface

Situations exist, however, where you might want to have multiple instances of Toaster running on various remote machines. You can create this situation by basically modifying how Toaster starts and where the common SQL database resides. You are able to do this because it is not required that Toaster starts the above set of components in order to run. Minimally, an instance of Toaster requires just one of the components to run. Consequently, you are free to manually start as many or few of the components as you need rather than using the Toaster script to cause all three things to happen.

The concepts for setting up multiple instances of Toaster revolve around maintaining a common SQL database and Web server that show data from that common database and then setting up separate instances of BitBake servers and Toaster user interfaces for each separate BitBake build directory. Note that the common SQL database and the Web server show data from all the various BitBake builds. Setting the SQL database outside of any BitBake build directories maintains a
separation layer between the various builds.

The database is persistent because the logging database is set up external to the database server (e.g. MySQL). It is not even necessary to run the BitBake servers, the SQL server, and the Web server on the same machine. Each component can be run on its own machine.

== Steps to get set up ==

=== 1. Set up the SQL Logging Server and the Web Server ===

You can use any SQL server out of the box (e.g. <code>apt-get install mysgl-server</code> works for an Ubuntu system). If you are concerned about performance, you might want to hand-tune the server. You must set up proper username and password access for the server. You need administration rights for the mysql root account. Realize that this is not the same thing as root access on the machine.

Clone a separate, local Git repository of the Toaster master branch to use for running the Web server. You do not perform builds on this tree. You need to create this local repository away from any build areas.

In the separately cloned tree for the Web server, edit the <code>bitbake/lib/toaster/toastermain/settings.py</code> file so that the <code>DATABASES</code> value points to the previously created database server. Use the username and password you established earlier.

Run the database sync scripts to create the needed tables as follows:

$ python bitbake/lib/toaster/manage.py syncdb
$ python bitbake/lib/toaster/manage.py migrate orm

Note: It is important, for toaster running in 1.6 mode, to not sync bldcontrol (only used in 1.7 mode).

Start the Web server using the following command:

$ python bitbake/lib/toaster/manage.py runserver

... or in the background:

$ nohup python bitbake/lib/toaster/manage.py runserver 2>toaster_web.log >toaster_web.log &

=== 2. Enable build logging to the common SQL server for each build directory you are using ===

Edit <code>_build local_ bitbake/lib/toaster/toastermain/settings.py</code> to alter the <code>DATABASES</code> value to point to the common SQL logging server.

Create the required conf/toaster.conf file as per Bitbake extra options on "https://wiki.yoctoproject.org/wiki/Setting_up_a_local_instance_of_Toaster".

Start the BitBake server using the following command:

$ bitbake --postread conf/toaster.conf --server-only -t xmlrpc -B localhost:0 && export BBSERVER=localhost:-1

Start the logging user interface using the following command:

$ nohup bitbake --observe-only -u toasterui &

NOTE: No hard-coded ports are used as there is enough code to run autodiscovery for ports to prevent collisions.

At this point, you are ready to run your builds using commands such as:

$ bitbake core-image-minimal

When you are finished, you need to kill the BitBake server for that particular build area:

$ bitbake -m

=== 3. Verify that it all works ===

You should examine the logs and be sure that the logging worked, that data is persistent, and that data from multiple builds from different areas was supported.

Start Toaster in managed mode

2014-11-03T11:43:41Z

Ddalex: Created page with "Starting Toaster in managed mode: * For local instance, using a terminal where oe-init-build-env was NOT sourced, simply run Toaster from the POKY checkout directory. POKY/bit..."

Starting Toaster in managed mode:

* For local instance, using a terminal where oe-init-build-env was NOT sourced, simply run Toaster from the POKY checkout directory.

POKY/bitbake/bin/toaster

* For remote instance, use the instructions in the attached presentation [[https://wiki.yoctoproject.org/wiki/images/7/75/Toaster_Presentation_ELCE_2014_1.pdf]]

File:Toaster Presentation ELCE 2014 1.pdf

2014-11-03T11:42:40Z

Ddalex: Toaster presentation that was given at ELCE 2014, Toaster-managed variant. Includes setup instructions for hosted managed-build Toaster.

Toaster presentation that was given at ELCE 2014, Toaster-managed variant.

Includes setup instructions for hosted managed-build Toaster.

Toaster

2014-11-03T11:35:10Z

Ddalex: /* Using Toaster */

File:Toaster.sqlite

2014-09-25T14:49:02Z

Ddalex: uploaded a new version of "File:Toaster.sqlite": Prepopulated db file for Toaster 1.7

Sample Toaster data for development purposes.

Extending Toaster

2014-06-25T14:34:12Z

Ddalex: /* How to create a Toaster extension */

In order to promote customizability of Toaster, we have here a set of guidelines on how to add functionality and customize the Toaster web interface.

== Django extensibility model ==

Django, per se, does not offer any extensibility / customization model. Being a framework for hosted application, it is assumed that the host will have complete access for modification. Because Django is not envisioned to be distributed / portable / reusable, there is no need to support insertion of 3rd party content after deployment, or to alter the user flows by 3rd party contributors. However, we can use Django modularization support to add custom extensibility capabilities to our distributable project.

A Django project supports a number of "applications", intended to allow developers to separate different scopes of a web site in different applications. For example, the custom admin interface for a web site should live in a different application than the normal user-facing site.

Toaster consists of a basic set of applications - we have:
* bldviewer - the "simple" web interface, for development purposes only
* toastergui - the Toaster frontend for build inspection
* orm - a separate application just to hold the persistent object models
* bldcontrol - application containing the code to start and control bitbake in managed mode

== Features for extensibility ==

Toaster extensibility model is based on Django. To implement new functionality in Toaster, you have to add a new application containing the code implementing the functionality.

Keeping the code separated in a different application will make for painless code rebases, as your application may live as a separate set of patches based on top of upstream Toaster.

As a web application, the code is driven by HTTP requests. In order to use your handlers, Toaster will automatically redirect requests starting "{modulename}/" to the patterns listed in the "{modulename}/urls.py" file, where "{modulename}" is the name of the application you're creating.

If your {modulename} application contains either views.py or models.py files, it will automatically be added to INSTALLED_APPS list, as to allow database synchronization applications, such as South, to work with your application. This means that you can use fixtures and migration in your application, just like any of the default Toaster applications

In your application you can call any code and use any models from other default Toaster applications.

If your application needs another contributed application, you have to manage any dependencies manually.

== How to create a Toaster extension ==

This is a summary of what you have to do to add new functionality for Toaster:

* create a new application holding that functionality: <code>python manage.py startapp {modulename}</code>
* add in the application directory a urls.py file: <code> {modulename}/urls.py </code>
* start toaster, and verify your pages are accessible with the "/{modulename}/" prefix
* limit all patches touching your application only to your application; this will ease the rebase (see below)
* when updating Toaster, use rebase (as opposed to merge) to move your applications' patches on top of a new Toaster version
* database management functions will be automatically enabled for your applications

Extending Toaster

2014-06-25T14:33:07Z

Ddalex:

Extending Toaster

2014-06-25T14:10:42Z

Ddalex: /* Django extensibility model */

Toaster

2014-05-19T10:57:30Z

Ddalex: /* About Toaster */

[[Category:Toaster]]
[[Toaster]] is a web-based interface to the Bitbake build system and the Poky distribution inside the Yocto Project.
This project was formely known as Web Hob / Webhob / webhob, and you may still find references to the old name in the documentation.

General discussion about '''Toaster''' happens on a dedicated mailing list: [https://lists.yoctoproject.org/listinfo/toaster https://lists.yoctoproject.org/listinfo/toaster]

=== Using Toaster ===

* [[Setting up a local instance of Toaster]]
* [[Setting up a production instance of Toaster]]
* [https://www.yoctoproject.org/documentation/toaster-manual How to use the Toaster web interface]
* [[How to delete information from the Toaster database]]

=== About Toaster ===

* [[Contribute to Toaster]]
* [https://bugzilla.yoctoproject.org/buglist.cgi?list_id=213820&columnlist=status_whiteboard%2Cassigned_to%2Ctarget_milestone%2Cbug_status%2Cshort_desc%2Cbug_severity%2Cpriority&classification=Build%20System%20%26%20Metadata&query_based_on=Toaster-Opens&query_format=advanced&bug_status=NEW&bug_status=ACCEPTED&bug_status=IN%20PROGRESS%20DESIGN&bug_status=IN%20PROGRESS%20DESIGN%20COMPLETE&bug_status=IN%20PROGRESS%20IMPLEMENTATION&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=WaitForUpstream&component=toaster&product=Toaster&known_name=Toaster-Opens Bug list]
* [[Toaster architecture design]]
* [[Toaster testing plan]]
* [[Toaster 1.7 design | Scope and design]]
* [[Toaster archive | Archive]]

Toaster

2014-05-19T10:57:14Z

Ddalex: /* About Toaster */

[[Category:Toaster]]
[[Toaster]] is a web-based interface to the Bitbake build system and the Poky distribution inside the Yocto Project.
This project was formely known as Web Hob / Webhob / webhob, and you may still find references to the old name in the documentation.

General discussion about '''Toaster''' happens on a dedicated mailing list: [https://lists.yoctoproject.org/listinfo/toaster https://lists.yoctoproject.org/listinfo/toaster]

=== Using Toaster ===

* [[Setting up a local instance of Toaster]]
* [[Setting up a production instance of Toaster]]
* [https://www.yoctoproject.org/documentation/toaster-manual How to use the Toaster web interface]
* [[How to delete information from the Toaster database]]

=== About Toaster ===

* [[Contribute to Toaster]]
* [[https://bugzilla.yoctoproject.org/buglist.cgi?list_id=213820&columnlist=status_whiteboard%2Cassigned_to%2Ctarget_milestone%2Cbug_status%2Cshort_desc%2Cbug_severity%2Cpriority&classification=Build%20System%20%26%20Metadata&query_based_on=Toaster-Opens&query_format=advanced&bug_status=NEW&bug_status=ACCEPTED&bug_status=IN%20PROGRESS%20DESIGN&bug_status=IN%20PROGRESS%20DESIGN%20COMPLETE&bug_status=IN%20PROGRESS%20IMPLEMENTATION&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=WaitForUpstream&component=toaster&product=Toaster&known_name=Toaster-Opens Bug list]]
* [[Toaster architecture design]]
* [[Toaster testing plan]]
* [[Toaster 1.7 design | Scope and design]]
* [[Toaster archive | Archive]]

Toaster

2014-05-19T10:56:56Z

Ddalex: /* About Toaster */

[[Category:Toaster]]
[[Toaster]] is a web-based interface to the Bitbake build system and the Poky distribution inside the Yocto Project.
This project was formely known as Web Hob / Webhob / webhob, and you may still find references to the old name in the documentation.

General discussion about '''Toaster''' happens on a dedicated mailing list: [https://lists.yoctoproject.org/listinfo/toaster https://lists.yoctoproject.org/listinfo/toaster]

=== Using Toaster ===

* [[Setting up a local instance of Toaster]]
* [[Setting up a production instance of Toaster]]
* [https://www.yoctoproject.org/documentation/toaster-manual How to use the Toaster web interface]
* [[How to delete information from the Toaster database]]

=== About Toaster ===

* [[Contribute to Toaster]]
* [[https://bugzilla.yoctoproject.org/buglist.cgi?list_id=213820&columnlist=status_whiteboard%2Cassigned_to%2Ctarget_milestone%2Cbug_status%2Cshort_desc%2Cbug_severity%2Cpriority&classification=Build%20System%20%26%20Metadata&query_based_on=Toaster-Opens&query_format=advanced&bug_status=NEW&bug_status=ACCEPTED&bug_status=IN%20PROGRESS%20DESIGN&bug_status=IN%20PROGRESS%20DESIGN%20COMPLETE&bug_status=IN%20PROGRESS%20IMPLEMENTATION&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=WaitForUpstream&component=toaster&product=Toaster&known_name=Toaster-Opens | Bug list]]
* [[Toaster architecture design]]
* [[Toaster testing plan]]
* [[Toaster 1.7 design | Scope and design]]
* [[Toaster archive | Archive]]

Toaster

2014-05-19T10:56:39Z

Ddalex: /* About Toaster */

[[Category:Toaster]]
[[Toaster]] is a web-based interface to the Bitbake build system and the Poky distribution inside the Yocto Project.
This project was formely known as Web Hob / Webhob / webhob, and you may still find references to the old name in the documentation.

General discussion about '''Toaster''' happens on a dedicated mailing list: [https://lists.yoctoproject.org/listinfo/toaster https://lists.yoctoproject.org/listinfo/toaster]

=== Using Toaster ===

* [[Setting up a local instance of Toaster]]
* [[Setting up a production instance of Toaster]]
* [https://www.yoctoproject.org/documentation/toaster-manual How to use the Toaster web interface]
* [[How to delete information from the Toaster database]]

=== About Toaster ===

* [[Contribute to Toaster]]
* [[https://bugzilla.yoctoproject.org/buglist.cgi?list_id=213820&columnlist=status_whiteboard%2Cassigned_to%2Ctarget_milestone%2Cbug_status%2Cshort_desc%2Cbug_severity%2Cpriority&classification=Build%20System%20%26%20Metadata&query_based_on=Toaster-Opens&query_format=advanced&bug_status=NEW&bug_status=ACCEPTED&bug_status=IN%20PROGRESS%20DESIGN&bug_status=IN%20PROGRESS%20DESIGN%20COMPLETE&bug_status=IN%20PROGRESS%20IMPLEMENTATION&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=WaitForUpstream&component=toaster&product=Toaster&known_name=Toaster-Opens|Bug list]]
* [[Toaster architecture design]]
* [[Toaster testing plan]]
* [[Toaster 1.7 design | Scope and design]]
* [[Toaster archive | Archive]]

Toaster

2014-05-19T10:56:19Z

Ddalex: /* About Toaster */

[[Category:Toaster]]
[[Toaster]] is a web-based interface to the Bitbake build system and the Poky distribution inside the Yocto Project.
This project was formely known as Web Hob / Webhob / webhob, and you may still find references to the old name in the documentation.

General discussion about '''Toaster''' happens on a dedicated mailing list: [https://lists.yoctoproject.org/listinfo/toaster https://lists.yoctoproject.org/listinfo/toaster]

=== Using Toaster ===

* [[Setting up a local instance of Toaster]]
* [[Setting up a production instance of Toaster]]
* [https://www.yoctoproject.org/documentation/toaster-manual How to use the Toaster web interface]
* [[How to delete information from the Toaster database]]

=== About Toaster ===

* [[Contribute to Toaster]]
* [[Our bug list|https://bugzilla.yoctoproject.org/buglist.cgi?list_id=213820&columnlist=status_whiteboard%2Cassigned_to%2Ctarget_milestone%2Cbug_status%2Cshort_desc%2Cbug_severity%2Cpriority&classification=Build%20System%20%26%20Metadata&query_based_on=Toaster-Opens&query_format=advanced&bug_status=NEW&bug_status=ACCEPTED&bug_status=IN%20PROGRESS%20DESIGN&bug_status=IN%20PROGRESS%20DESIGN%20COMPLETE&bug_status=IN%20PROGRESS%20IMPLEMENTATION&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=WaitForUpstream&component=toaster&product=Toaster&known_name=Toaster-Opens]]
* [[Toaster architecture design]]
* [[Toaster testing plan]]
* [[Toaster 1.7 design | Scope and design]]
* [[Toaster archive | Archive]]

Extending Toaster

2014-05-15T15:56:16Z

Ddalex:

Extending Toaster

2014-05-15T15:55:41Z

Ddalex: Created page with "In order to promote customizability of Toaster, we have here a set of guidelines on how to add functionality and customize the Toaster web interface. * Django extensibility mode..."

In order to promote customizability of Toaster, we have here a set of guidelines on how to add functionality and customize the Toaster web interface.

* Django extensibility model *

Django, per se, does not offer any extensibility / customization model. Being a framework for hosted application, it is assumed that the host will have complete access for modification. Because Django is not envisioned to be distributed / portable / reusable, there is no need to support insertion of 3rd party content after deployment, or to alter the user flows by 3rd party contributors. However, we can use Django modularization support to add custom extensibility capabilities to our distributable project.

A Django project supports a number of "applications", intended to allow developers to separate different scopes of a web site in different applications. For example, the custom admin interface for a web site should live in a different application than the normal user-facing site.

Toaster consists of a basic set of applications - we have:
* bldviewer - the "simple" web interface, for development purposes only
* toastergui - the Toaster frontend for build inspection
* orm - a separate application just to hold the persistent object models

* Requirements for extensibility *

* Toaster points - a proposal *

Toaster 1.7 design

2014-04-25T09:45:37Z

Ddalex: /* Discussion */

== A couple of videos to kick off things ==

The following videos illustrate some of the features we are thinking of adding to Toaster during the 1.7 release cycle. When watching them, keep in mind that this is all very rough: more like sketches to help us start a conversation about scope than proper designs. Chances are the final designs will be completely different.

About projects - [[media:Building-1.7.mov]] 
Starting a build - [[media:How-to-build.mov]]

== Feature list ==

=== Ability to create new projects ===

This is the ability to use the Toaster interface to create new project directories, manage the content, and initiate the builds of selectable target images.

===== Summary of discussion =====

* A project is a set of layers and configuration variables values that describes how to do a build.
* Creating a project means writing up values for the above-mentioned set, and store these values in a persistent manner.
* A build for a project means effecting (instantiating) the build process for the current set of values.
* Build directory location (host/machine) is not relevant to Toaster.
* Saving the build artifacts and providing access to the build artifacts is critical for Toaster.

===== Discussion =====

[DAVE] What is a typical workflow that gets to the build project part? I see the configure screens but not the actually build launch.

[DAVE] Does clicking 'new build' fire off a build? If so, which configuration is built?

[BELEN] Heh, I completely forgot to show that in the first video. [http://wiki.yoctoproject.org/wiki/images/b/b1/How-to-build.mov Here is part 2] showing how to start a build and answering (I hope) both questions.

[DAVE] Can projects be configured, primed for builds, but not built?

[BELEN] Yes.

[DAVE] Will the user be able to configure several builds, then queue them all to build?

[BELEN] Users might create more than one project then start a build for each of them by clicking the "build" button in each project page, but nothing more sophisticated than that for the time being. Here is where things can get tricky. Where does Toaster stop and something like Buildbot or Jenkins start? My take is that we want to work alongside and enhance build automation systems, not replace them.

[DAVE] What is the size of the build variable list? It's hard to figure from the conf/local.conf file. If it is a small list, like 10, should we provide better and variable specific UI widgets rather than just edit boxes, depending on the variable's "type" (eg browse button with any path variable like SSTATE_DIR)?

[BELEN] This is one of the questions we should start working on straight away. In informal chats we have spoken about a split between "build variables" (the ones that affect the build and therefore you can edit via Toaster) and "infrastructure variables" (to call them something, that affect the build infrastructure and you cannot edit via Toaster. Things like PARALLEL_MAKE for example). We should start getting into details about that split as soon as possible, and determine which variables we are going to expose via Toaster to be edited. Once again, Paul's feedback here would be good (RP's probably as well).

Regarding variable "types", I had a brief conversation with RP recently about this issue. It is unlikely that the meta data will provide us with information about the type of input expected by a variable. If and how the Toaster interface should fill that gap is up for discussion. From my perspective, we should do as much as we can to help users assign valid values to their variables.

[DAVE] I don't recall a user entry on your videos that specify the project directory for the build. This must be part of the initial release. I don't recommend making it an edited data file option, eg not via in settings.py.

[BELEN] When you run Toaster locally you can configure the location of your build directory as you wish, but when Toaster is running remotely, do we want to let people select the location of their build directory?

[DAVE] Can we make any selection on any page 'stick' with cookies or 'unstick' with a 'reset to defaults' button, in addition to propagating laborious configuration setups inputs via 'clone configuration'?

[BELEN] We will need to overcome my personal hate of 'reset to defaults' options :) They are very dangerous if we don't provide an 'undo' option as well: they might wipe your configuration and revert it to the rather useless default one, losing a bunch of work you did beforehand. Any configuration changes you make are saved: in that sense, they are 'sticky'. If I create project A, then I add 2 layers to the project configuration, select a machine and 3 targets, all those options are saved to the project configuration. You don't need to add the 2 layers, select the machine and the 3 targets every time you want to run a build.

[RAVI] I am wondering about the purpose of a project. In the design, project has a machine and owner attributes. If we define a project as a collection of builds for a machine, then we should not allow to change the machine after a project is created. Then, if a project is just a collection of builds for a particular machine, then we can modify the existing all builds view to group builds by machine. Based on this argument, I do not see a need for project and build hierarchy.

[DAVID] My group understands that a "project" is a specific build directory. A project will most likely to have multiple builds, and things like the (a) machine, (b) layers, (c) conf variables, and so forth would be presumably set before the first target is selected and executed. It is also the case that the developer may decide to change any of those settings, including even the machine type, and then carry on with more builds. The use case would be for example is arch coverage testing of a particular set of kernel or user space packages.

A host can then have multiple project directories (I most often work that way), for example when a developer truly wants to keep their development and/or debugging from not polluting other projects.

Also, separate users should do their work in separate project directories, now that we are extending the model to cover that.

There can then be multiple hosts, with all of their multiple projects, all channeling their Toaster data into a shared database. And with that you have a build farm.

[RAVI] I see that the uniqueness of a project is the <hostname>://<build directory path>. The contents of the build directory can be changed and multiple builds can be kicked off from the same project with different configurations. If this the case, the project creation should also setup a build directory on the host.

Looking at this use case, we should have toaster running on the host machine with a shared database. Or if we run a single toaster instance serving multiple projects (hosts and build directories), we should have a mechanism for the toaster machine to talk to the project machines. Am I missing something?

[PAUL] Yes, but the project need not be tied to a specific host or location on disk. To my mind, the project is simply a configuration set - if you had a build farm the actual build could be run on any server within that farm and that side of it would be largely hidden from the user (until it needed to be visible, e.g. when you need to diagnose a host contamination issue.) This then means that the management of build directories can be independent, i.e. old build directories can be automatically deleted after a time period or when space runs low, but the project remains in the database for later use when needed.

[DAVID] I am completely willing to accept the above definition of a "Toaster Project". The part that convinces me is that (a) while under active build and development it does map to a physical location, but (b) it can hold the data and presumably the "snapshot" artifacts of a physical build directory long after that actual directory has been deleted. That last part is a highly desired feature from the Wind River team, and was also brought up as a specific question at my Toaster presentation at the Yocto Project Summit.

This implies that given the abstract nature of a Toaster "Project", it makes sense that not only can the developer share that "project" with another developer and Toaster instance (as per Belén design), but it makes sense that the original developer can make their own clones of that project when they for example want use as a reference basis for new projects with additional adjustments and/or for binding and running on a different host/directory.

Here is what I would propose is the workflow.

1) When a Toaster Project is created,

* The "user" is the preset to the last "user", else if this is the first time it is defaulted to $USER. This is a required field.

* The "email" is preset to the last email that matches "user", else it is empty. QUESTION: Is this a required field?

* The host:dir values are preset just as "poky/oe-init-build-env" would preset these values, in other words to the current machine and to "poky/../build" (unless redirected with the appropriate environment variables). If that directory (a) already exists and has a "conf/local.conf" file and (b) that directory exists for an existing Project, then that is an error. This would allow us to re-connect to a deleted or unregistered YP build directories.

* We either have an explicit "Project Create" button, or we allow for the first build target command to insure that for new projects the project directory is created and the content preset by "poky/oe-init-build-env" before continuing.

2) For external hosts mounted via NFS:

* If an external host is mounted via a YP compatible NFS system, then that would be covered by the normal workflow above, using the local host and the respective NFS path. QUESTION: do we wish to identify the actual remove HOST ID in these circumstances, or just let the NFS path speak for itself?

3) For fully external hosts

This is the situation where a host remote to the Toaster, but is intended to be managed by this Toaster and use this Toaster's database.

* This implies using some sort of rpc, probably via SSH, to execute the creation and build targets.

* There would presumably be an installation of YP on that remote host, and that location would need to be known to Toaster. The installation could be as simple as an NFS mount that points to the local YP installation, which would provide a single source at the cost of NFS traffic.

=== Hide builds from a project ===

This option simply removes or hides selected builds from the Toaster display, for example builds quickly halted because the owner realized that they missed a setting. The database records and the project on the disk is untouched. This un-clutters the display, yet requires little programing (and development time).

===== Discussion =====

[BELEN] I have no strong feelings against this one, but we need to bear in mind that:

* It will probably be hard to allow hiding in bulk, since the reasons why someone might want to hide a build could be wildly different. So this will have to be done on a build by build basis.

* This also involves providing a way to reverse the hiding action, i.e. see hidden builds and show them again.

[DAVID] In the Wind River version of Wiki, we have a "Manage" button at the bottom. This brings up a page of all of the entries, with the ability to change the entry's hidden and description attributes. This does allow a bulk update easily.

I propose that we do the same, where a "Manage" button brings up all of the builds and/or projects, each highlighted according to if it is currently hidden, also each with a text entry for the description (see the next design proposal) and a check box to toggle the hidden state.

We then need to decide if this meta date is stored in the database or in the page's cookie.

=== Ability to add a description to a build or project ===

User's may not recall the difference and purpose of a given build or project, so it would be good for the Toaster interface to allow the user to add this and update this as needed.

[DAVID] See the discussion above about "hidden" builds for a proposed implementation.

=== Delete builds from a project ===

This option is where a selected build and all its related database records are removed from the database. This removes the database overhead for this unwanted builds. The project on the disk is untouched. Since the database should be normalized, this operation should be clean and not interfere with other builds. I believe we have a command line version of this operation with Toaster-1.6.

===== Discussion =====

[Alex] Technically, this is straightforward. In terms of policy access, we need to define one; I propose that only the project owner is allowed to delete builds. Impersonation is trivial right now, but once we implement authentication, bricks will fall in the right place in terms of access control.

=== Ability to add / remove layers ===

Add and delete from the project layer directories. These layers may be within repositories or in plain directories on the disk.

===== Discussion =====

[FARRELL] How would I upload my specialized device driver/library/system call handler/etc that necessarily needs to be part of
the build? Would this be done by importing a layer?

[BELEN] Yes. For the 1.7 release, all customisation must be done via layers.

[DAVE] Are you constraining 'import layer' to a git repo path? Why not just a local directory that is not a git repo? In that case should we have a browse button as well as an edit box?

[BELEN] This is a really good question. I need the technical crowd to establish what's possible / what we want to do. We can then design accordingly.

[DAVID] It is the case that we recommend all of our customers to add their content via layers, and specifically layer directories since they are most likely to be under continual and direct change and would not be placed into a formal repo, or more likely they would be managed by their SCM system as is and certainly not packed into a base git clone.

=== Ability to identify / set project owner / user ===

A build farm manager (and their shared Toaster instance) may accept build sets from clients, in which case it is important to know who initiates and owns each specific build, for completion notifications and project space recycling.

===== Discussion =====

[FARRELL] You presented the notion of a user - in the movie it was 'Belen'. I think it may be necessary to formalize this
idea with a login screen and at least rudimentary permission and protection. Here is a use case: Imagine a vendor
deploys Toaster on a server which serves all of its customers. The customers are very diverse and are often competitors
of one another and do not want their configuration/information/ideas shared with other vendor customers.

[BELEN] Very true: access control is a big deal and very important. However, we are not sure we can tackle this in the 1.7 release. That's why in the video I speak of "owners" instead of "users". Our "owner" concept derives from the need to identify who created a certain project, but they are not really users since, for example, they don't involve authentication (there is no password).

Here is a possible description of these "owners". Each client accessing a Toaster instance might have an owner linked to it. You don't explicitly create these owners: you just set them in the process of creating your first project. Then, we remember them. Owners involve only 3 pieces of information: a name, an optional email address and the client they are associated with. You can change the details at any time. So "owners" don't involve access control: they just tell me who created (and therefore owns) a certain project. You can have the same owner details in several clients: I might have a laptop and a builds machine, I access the same Toaster instance from both, and I can use the same owner name and email address in both.

The above opens some questions:

* can others edit the configuration of a project I own?
* can others start builds against a project I own?
* can others download artifacts of a project I own?
* can others even see the projects I own?

The simplest thing would be either:

* 'yes' to all of the above, but that means that people creating Toaster instances that can be accessed by more than one person have to be mindful of the fact that everything in them is open.

* 'no' to all of the above, and channel all collaboration through the export / import projects functionality, which is safer but also stifles co-operation.

Just to clarify, ownership as above is established per project, not per build.

[FARRELL] As much as I hate to suggest this...It may be necessary to introduce the notion of group access - Owner/Group/Others ala unix/linux. I think it may be necessary for users to be able to share their results while excluding others. I can imagine a number of use cases. Here is one:

Two customers/users, Fu and Bar, work for the same company C. Each wants to
be able to have personal 'experimental' builds but may need to share a 'production'
build - react to errors and download results, etc. Of course, company C doesn't want
company B to see their results, configurations, custom drivers, etc. But, Fu needs to
see and have access to some of Bar's results. These problems can be solved with the
introduction of the group access.

I realize this adds a layer of complexity which may be necessary. An alternative would be access control lists (acl). These can be even more flexible but a lot more complicated to implement, manage and maintain.

I think we should decide on an access control model soon and work out an implementation. Even with the current implementation, retrofitting will be difficult and somewhat tedious which will become even more so as we add more features.

[DAVID] The intermediate near term solution is to consider the 1.7 as working with only trusted users, in that they are all already within the same intranet. Under this limitation, if user Foo needs their work fully separate from user Bar, they could just have separate instances and databases of Toaster running, and use existing Linux security to accomplish this.

That said, we could provide for the default build/project view to default to the current "owner", where they would have to go out of their way to other people's work.

We could also consider an "audit log" that captures project creation (and especially project deletion), so that at least you know if someone stomped on your stuff :-)

[BELEN] I am with David on this one: I think we can find temporary alternatives. It's not that I disagree with Farrell, mind: he is completely right that we will need this. Problem is: we need to be mindful of what we can deliver with the resources we have.

=== Ability to identify build host and build directory ===

To support distributed build hosts sharing a common Toaster database, there needs to be a way in the database to identify which host (IP address and or host name) and local directory the project was built in.

===== Discussion =====

[ALEX] I am not sure I follow the use case / user story here. Can we get more details on why this is important ?

=== Ability to download files (YP #6096) ===

While developers would presumably have access to all build files within the intranet for the build logs and generated target image files, there are cases where that is not so. For example, the owner may be using a Windows host or a portable device to access Toaster, and may wish to download and view the error logs to determine their next course of action. Also, it may be the case that the specific build host is accessible only to the build farm manager and not the individual project owner, and having Toaster serve the file(s) would resolve this barrier.

===== Discussion =====

[FARRELL] The ability to download the result of the build should be somewhere - maybe a column on the All Builds page?

[DAVID] As for things like file downloads, between Ravi and myself I think we will find time to get that one done since we both really want it!

[BELEN] Absolutely: in fact, this is something I might start designing straight away so that it can be implemented in 1.7M1. However, in order to do that, we need to reach agreement as to what a build artifact is:

* Is it just the rootfs files? Or do they include
* the cooker and task logs?
* shared state objects used?
* recipe files / patches?
* [yours here]

=== Share projects via configuration (export / import) ===

This is the capture and sharing of the minimal project configuration data, such that a remote developer can replicate the same project against their own Yocto Project installation.

===== Discussion =====

[DAVE] The video ended at the export project discussion. Is export just sending email, the conf file (or files) to someone, or is there be more to it?

[BELEN] I am not sure what the answer is: we need input from Paul and Alex here. In my head, the project configuration (not the builds) is exported to some kind of file(s) that I can then share with others in whichever way I see fit (email, file sharing, etc).

[ALEX] I'm thinking that exporting the project will produce a file that can be imported on another Toaster instance and replicate the original settings of the project that was exported. The concrete means of achieving this may be as simple as exporting a JSON snapshot of the relevant database entries.

=== Definition of a Project ===

The proposed design adds the new concept of the "Project".

===== Discussion =====

[PAUL] To my mind, the project is simply a configuration set - if you had a build farm the actual build could be run on any server within that farm and that side of it would be largely hidden from the user (until it needed to be visible, e.g. when you need to diagnose a host contamination issue.) This then means that the management of build directories can be independent, i.e. old build directories can be automatically deleted after a time period or when space runs low, but the project remains in the database for later use when needed.

[DAVID] I am completely willing to accept your definition of a "Toaster Project". The part that convinces me is that (a) while under active build and development it does map to a physical location, but (b) it can hold the data and presumably the "snapshot" artifacts of a physical build directory long after that actual directory has been deleted. That last part is a highly desired feature from the Wind River team, and was also brought up as a specific question at my Toaster presentation at the Yocto Project Summit.

This implies that given the abstract nature of a Toaster "Project", it makes sense that not only can the developer share that "project" with another developer and Toaster instance (as per Belén design), but it makes sense that the original developer can make their own clones of that project when they for example want use as a reference basis for new projects with additional adjustments and/or for binding and running on a different host/directory.

===== Workflow around creating a Project =====

[DAVID] Here is what I would propose as a workflow.

1) When a Toaster Project is created,

* The "user" is the preset to the last "user", else if this is the first time it is defaulted to $USER. This is a required field.

* The "email" is preset to the last email that matches "user", else it is empty. QUESTION: Is this a required field?

* The host:dir values are preset just as "poky/oe-init-build-env" would preset these values, in other words to the current machine and to "poky/../build" (unless redirected with the appropriate environment variables). If that directory (a) already exists and has a "conf/local.conf" file and (b) that directory is already registered with an existing Project, then that is an error. This would provide some sanity check, but would also allow us to re-connect to an unregistered YP build directory (and/or ones previous registered with a subsequently deleted project).

* We either have an explicit "Project Create" button, or we allow for the first build target command to insure that for new projects the project directory is created and the content preset by "poky/oe-init-build-env" before continuing.

2) For external hosts mounted via NFS:

* If an external host is mounted via a YP compatible NFS system, then that would be covered by the normal workflow above, using the local host and the respective NFS path. QUESTION: do we wish to identify the actual remove HOST ID in these circumstances, or just let the NFS path speak for itsel

3) For fully external hosts

This is the situation where a host remote to the Toaster, but is intended to be managed by this Toaster and use this Toaster's database.

* This implies using some sort of rpc, probably via SSH, to execute the creation and build targets.

* There would presumably be an installation of YP on that remote host, and that location would need to be known to Toaster. The installation could be as simple as an NFS mount that points to the local YP installation, which would provide a single source at the cost of NFS traffic.

== Toaster System Architectures ==

This is an iteration of the possible supported system architectures of a Toaster implementation. We should validate how the proposed design would work with each of these options.

=== Minimal Local Project Implementation ===

Configuration: Local host, local project, database in project, toaster and web server started from project

This is the default implementation, using the "source toaster [start|stop]" support.

=== Multiple Local Project Implementation ===

Configuration: Local host, Multiple local projects, shared local database, toaster and web server started separately

This is currently supported by (a) edits to the local.conf file to point to the shared database location, and (b) using special commands to start the shared Toaster and webserver instance.

=== Multiple Distributed Host Implementation ===

Configuration: Multiple hosts, Multiple local projects per host, shared network database, toaster and web server started with network database instance

Each host would either have its own installation of YP, or have a centralized copy shared over NFS. This implementation is currently not yet documented nor supported.

== Open Bugzilla features / defects ==

[http://wiki.yoctoproject.org/wiki/Bugzilla_feature_list This is what's open in Bugzilla right now]. Apart from what's already assigned to 1.6.1, there are 2 issues that I am keen on getting fixed:

[https://bugzilla.yoctoproject.org/show_bug.cgi?id=5187 5187] - Same cooker log file name used for several builds 
[https://bugzilla.yoctoproject.org/show_bug.cgi?id=6095 6095] - Retrieve full build information independently of task execution

Feel free to add to this list.

== Future features ==

A list of features that have come up in discussion, but not for the 1.7 release.

=== Delete projects ===

This option deletes the project all the way down to the project's instance on the disk. This should be a safe operation unless a separate project is directly sharing its sstate cache. That would be an recommended usage, where a proper usage would a standalone shared sstate cache. There may be artifacts in that shared sstate cache that are specific and unique to the deleted project, but it is out of scope for this option to try to delete those as well.

=== Ability to add/remove recipes ===

This is the ability to add or remove recipes from the project's configuration.

=== Ability to import source (tarballs, repos) ===

This is the ability to bring external sources into the project and include them with wrapper recipes.

=== Ability to add/remove packages from image ===

This is the ability to add or remove specific binary packages from the respective image(s) of a project.

=== Share projects via Copy (all but "./tmp") ===

Project directories (all except for their local "tmp" directory) should be portable between hosts, where the receiving developer needs only to adjust the local.conf file for the local installation and sstate directories.

=== Ability to get notification of build status ===

Since builds can take a long time, it would be very useful to get for example email notification when a build stops, either with success or with failure, rather than needing to continually and manually poll the Toaster interface.

===== Discussion =====

[DAVE] I agree, but can we generalize this post-build processing to enable post-build script execution triggered by either success, failure, or either? We could provide a few parameterized scripts to, for example:

* send email to a user or list with build status,
* (on success) start an emulation session with regression tests on a build,
* (on success) download the built file system or SDK to a user's host or target,
* provide hooks for the user to substitute their alternative post-build triggers.

=== Ability to access Toaster from portable device ===

Since builds generally take a long time, the user may be away from their desk (or even work hours) and wish to track the status of their pending builds. Supporting a basic Toaster interface for such limited devices would be advantageous.

=== Ability to compare projects/builds ===

A user may not recall the difference between one build and another, or one project and another. The data is in the database, so it would not be hard to generate a report itemizing the observed differences. Here are some the potential changes that can be extracted:

* Changed variable definitions (less base path differences). This would also cover changes in things like the machine, layers, extra added or excluded recipes, and multi-lib support.
* Changes in the included packages list
* Perhaps changes in the build/caching statistics

=== Snapshots of builds for long term archiving (project dir lifecycle) ===

A build manager may wish to keep the pertinent facts of builds for an extended period of time. This could be for a few days to allow time for the build's owner to reconcile the results, or if maybe over a year for tracking long-term trends and regressions. In each case, there should be a way to capture a minimal set of information from the build directory (e.g. the conf files and any build failure logs) and preserve it so that together with the database there is enough information to study that project even when the full build directory is removed and recycled.

Toaster 1.7 design

2014-04-25T09:42:42Z

Ddalex: /* Ability to identify build host and build directory */

== A couple of videos to kick off things ==

The following videos illustrate some of the features we are thinking of adding to Toaster during the 1.7 release cycle. When watching them, keep in mind that this is all very rough: more like sketches to help us start a conversation about scope than proper designs. Chances are the final designs will be completely different.

About projects - [[media:Building-1.7.mov]] 
Starting a build - [[media:How-to-build.mov]]

== Feature list ==

=== Ability to create new projects ===

This is the ability to use the Toaster interface to create new project directories, manage the content, and initiate the builds of selectable target images.

===== Summary of discussion =====

* A project is a set of layers and configuration variables values that describes how to do a build.
* Creating a project means writing up values for the above-mentioned set, and store these values in a persistent manner.
* A build for a project means effecting (instantiating) the build process for the current set of values.
* Build directory location (host/machine) is not relevant to Toaster.
* Saving the build artifacts and providing access to the build artifacts is critical for Toaster.

===== Discussion =====

[DAVE] What is a typical workflow that gets to the build project part? I see the configure screens but not the actually build launch.

[DAVE] Does clicking 'new build' fire off a build? If so, which configuration is built?

[BELEN] Heh, I completely forgot to show that in the first video. [http://wiki.yoctoproject.org/wiki/images/b/b1/How-to-build.mov Here is part 2] showing how to start a build and answering (I hope) both questions.

[DAVE] Can projects be configured, primed for builds, but not built?

[BELEN] Yes.

[DAVE] Will the user be able to configure several builds, then queue them all to build?

[BELEN] Users might create more than one project then start a build for each of them by clicking the "build" button in each project page, but nothing more sophisticated than that for the time being. Here is where things can get tricky. Where does Toaster stop and something like Buildbot or Jenkins start? My take is that we want to work alongside and enhance build automation systems, not replace them.

[DAVE] What is the size of the build variable list? It's hard to figure from the conf/local.conf file. If it is a small list, like 10, should we provide better and variable specific UI widgets rather than just edit boxes, depending on the variable's "type" (eg browse button with any path variable like SSTATE_DIR)?

[BELEN] This is one of the questions we should start working on straight away. In informal chats we have spoken about a split between "build variables" (the ones that affect the build and therefore you can edit via Toaster) and "infrastructure variables" (to call them something, that affect the build infrastructure and you cannot edit via Toaster. Things like PARALLEL_MAKE for example). We should start getting into details about that split as soon as possible, and determine which variables we are going to expose via Toaster to be edited. Once again, Paul's feedback here would be good (RP's probably as well).

Regarding variable "types", I had a brief conversation with RP recently about this issue. It is unlikely that the meta data will provide us with information about the type of input expected by a variable. If and how the Toaster interface should fill that gap is up for discussion. From my perspective, we should do as much as we can to help users assign valid values to their variables.

[DAVE] I don't recall a user entry on your videos that specify the project directory for the build. This must be part of the initial release. I don't recommend making it an edited data file option, eg not via in settings.py.

[BELEN] When you run Toaster locally you can configure the location of your build directory as you wish, but when Toaster is running remotely, do we want to let people select the location of their build directory?

[DAVE] Can we make any selection on any page 'stick' with cookies or 'unstick' with a 'reset to defaults' button, in addition to propagating laborious configuration setups inputs via 'clone configuration'?

[BELEN] We will need to overcome my personal hate of 'reset to defaults' options :) They are very dangerous if we don't provide an 'undo' option as well: they might wipe your configuration and revert it to the rather useless default one, losing a bunch of work you did beforehand. Any configuration changes you make are saved: in that sense, they are 'sticky'. If I create project A, then I add 2 layers to the project configuration, select a machine and 3 targets, all those options are saved to the project configuration. You don't need to add the 2 layers, select the machine and the 3 targets every time you want to run a build.

[RAVI] I am wondering about the purpose of a project. In the design, project has a machine and owner attributes. If we define a project as a collection of builds for a machine, then we should not allow to change the machine after a project is created. Then, if a project is just a collection of builds for a particular machine, then we can modify the existing all builds view to group builds by machine. Based on this argument, I do not see a need for project and build hierarchy.

[DAVID] My group understands that a "project" is a specific build directory. A project will most likely to have multiple builds, and things like the (a) machine, (b) layers, (c) conf variables, and so forth would be presumably set before the first target is selected and executed. It is also the case that the developer may decide to change any of those settings, including even the machine type, and then carry on with more builds. The use case would be for example is arch coverage testing of a particular set of kernel or user space packages.

A host can then have multiple project directories (I most often work that way), for example when a developer truly wants to keep their development and/or debugging from not polluting other projects.

Also, separate users should do their work in separate project directories, now that we are extending the model to cover that.

There can then be multiple hosts, with all of their multiple projects, all channeling their Toaster data into a shared database. And with that you have a build farm.

[RAVI] I see that the uniqueness of a project is the <hostname>://<build directory path>. The contents of the build directory can be changed and multiple builds can be kicked off from the same project with different configurations. If this the case, the project creation should also setup a build directory on the host.

Looking at this use case, we should have toaster running on the host machine with a shared database. Or if we run a single toaster instance serving multiple projects (hosts and build directories), we should have a mechanism for the toaster machine to talk to the project machines. Am I missing something?

[PAUL] Yes, but the project need not be tied to a specific host or location on disk. To my mind, the project is simply a configuration set - if you had a build farm the actual build could be run on any server within that farm and that side of it would be largely hidden from the user (until it needed to be visible, e.g. when you need to diagnose a host contamination issue.) This then means that the management of build directories can be independent, i.e. old build directories can be automatically deleted after a time period or when space runs low, but the project remains in the database for later use when needed.

[DAVID] I am completely willing to accept the above definition of a "Toaster Project". The part that convinces me is that (a) while under active build and development it does map to a physical location, but (b) it can hold the data and presumably the "snapshot" artifacts of a physical build directory long after that actual directory has been deleted. That last part is a highly desired feature from the Wind River team, and was also brought up as a specific question at my Toaster presentation at the Yocto Project Summit.

This implies that given the abstract nature of a Toaster "Project", it makes sense that not only can the developer share that "project" with another developer and Toaster instance (as per Belén design), but it makes sense that the original developer can make their own clones of that project when they for example want use as a reference basis for new projects with additional adjustments and/or for binding and running on a different host/directory.

Here is what I would propose is the workflow.

1) When a Toaster Project is created,

* The "user" is the preset to the last "user", else if this is the first time it is defaulted to $USER. This is a required field.

* The "email" is preset to the last email that matches "user", else it is empty. QUESTION: Is this a required field?

* The host:dir values are preset just as "poky/oe-init-build-env" would preset these values, in other words to the current machine and to "poky/../build" (unless redirected with the appropriate environment variables). If that directory (a) already exists and has a "conf/local.conf" file and (b) that directory exists for an existing Project, then that is an error. This would allow us to re-connect to a deleted or unregistered YP build directories.

* We either have an explicit "Project Create" button, or we allow for the first build target command to insure that for new projects the project directory is created and the content preset by "poky/oe-init-build-env" before continuing.

2) For external hosts mounted via NFS:

* If an external host is mounted via a YP compatible NFS system, then that would be covered by the normal workflow above, using the local host and the respective NFS path. QUESTION: do we wish to identify the actual remove HOST ID in these circumstances, or just let the NFS path speak for itself?

3) For fully external hosts

This is the situation where a host remote to the Toaster, but is intended to be managed by this Toaster and use this Toaster's database.

* This implies using some sort of rpc, probably via SSH, to execute the creation and build targets.

* There would presumably be an installation of YP on that remote host, and that location would need to be known to Toaster. The installation could be as simple as an NFS mount that points to the local YP installation, which would provide a single source at the cost of NFS traffic.

=== Hide builds from a project ===

This option simply removes or hides selected builds from the Toaster display, for example builds quickly halted because the owner realized that they missed a setting. The database records and the project on the disk is untouched. This un-clutters the display, yet requires little programing (and development time).

===== Discussion =====

[BELEN] I have no strong feelings against this one, but we need to bear in mind that:

* It will probably be hard to allow hiding in bulk, since the reasons why someone might want to hide a build could be wildly different. So this will have to be done on a build by build basis.

* This also involves providing a way to reverse the hiding action, i.e. see hidden builds and show them again.

[DAVID] In the Wind River version of Wiki, we have a "Manage" button at the bottom. This brings up a page of all of the entries, with the ability to change the entry's hidden and description attributes. This does allow a bulk update easily.

I propose that we do the same, where a "Manage" button brings up all of the builds and/or projects, each highlighted according to if it is currently hidden, also each with a text entry for the description (see the next design proposal) and a check box to toggle the hidden state.

We then need to decide if this meta date is stored in the database or in the page's cookie.

=== Ability to add a description to a build or project ===

User's may not recall the difference and purpose of a given build or project, so it would be good for the Toaster interface to allow the user to add this and update this as needed.

[DAVID] See the discussion above about "hidden" builds for a proposed implementation.

=== Delete builds from a project ===

This option is where a selected build and all its related database records are removed from the database. This removes the database overhead for this unwanted builds. The project on the disk is untouched. Since the database should be normalized, this operation should be clean and not interfere with other builds. I believe we have a command line version of this operation with Toaster-1.6.

===== Discussion =====

[Alex] Technically, this is straightforward. In terms of policy access, we need to define one; I propose that only the project owner is allowed to delete builds. Impersonation is trivial right now, but once we implement authentication, bricks will fall in the right place in terms of access control.

=== Ability to add / remove layers ===

Add and delete from the project layer directories. These layers may be within repositories or in plain directories on the disk.

===== Discussion =====

[FARRELL] How would I upload my specialized device driver/library/system call handler/etc that necessarily needs to be part of
the build? Would this be done by importing a layer?

[BELEN] Yes. For the 1.7 release, all customisation must be done via layers.

[DAVE] Are you constraining 'import layer' to a git repo path? Why not just a local directory that is not a git repo? In that case should we have a browse button as well as an edit box?

[BELEN] This is a really good question. I need the technical crowd to establish what's possible / what we want to do. We can then design accordingly.

[DAVID] It is the case that we recommend all of our customers to add their content via layers, and specifically layer directories since they are most likely to be under continual and direct change and would not be placed into a formal repo, or more likely they would be managed by their SCM system as is and certainly not packed into a base git clone.

=== Ability to identify / set project owner / user ===

A build farm manager (and their shared Toaster instance) may accept build sets from clients, in which case it is important to know who initiates and owns each specific build, for completion notifications and project space recycling.

===== Discussion =====

[FARRELL] You presented the notion of a user - in the movie it was 'Belen'. I think it may be necessary to formalize this
idea with a login screen and at least rudimentary permission and protection. Here is a use case: Imagine a vendor
deploys Toaster on a server which serves all of its customers. The customers are very diverse and are often competitors
of one another and do not want their configuration/information/ideas shared with other vendor customers.

[BELEN] Very true: access control is a big deal and very important. However, we are not sure we can tackle this in the 1.7 release. That's why in the video I speak of "owners" instead of "users". Our "owner" concept derives from the need to identify who created a certain project, but they are not really users since, for example, they don't involve authentication (there is no password).

Here is a possible description of these "owners". Each client accessing a Toaster instance might have an owner linked to it. You don't explicitly create these owners: you just set them in the process of creating your first project. Then, we remember them. Owners involve only 3 pieces of information: a name, an optional email address and the client they are associated with. You can change the details at any time. So "owners" don't involve access control: they just tell me who created (and therefore owns) a certain project. You can have the same owner details in several clients: I might have a laptop and a builds machine, I access the same Toaster instance from both, and I can use the same owner name and email address in both.

The above opens some questions:

* can others edit the configuration of a project I own?
* can others start builds against a project I own?
* can others download artifacts of a project I own?
* can others even see the projects I own?

The simplest thing would be either:

* 'yes' to all of the above, but that means that people creating Toaster instances that can be accessed by more than one person have to be mindful of the fact that everything in them is open.

* 'no' to all of the above, and channel all collaboration through the export / import projects functionality, which is safer but also stifles co-operation.

Just to clarify, ownership as above is established per project, not per build.

[FARRELL] As much as I hate to suggest this...It may be necessary to introduce the notion of group access - Owner/Group/Others ala unix/linux. I think it may be necessary for users to be able to share their results while excluding others. I can imagine a number of use cases. Here is one:

Two customers/users, Fu and Bar, work for the same company C. Each wants to
be able to have personal 'experimental' builds but may need to share a 'production'
build - react to errors and download results, etc. Of course, company C doesn't want
company B to see their results, configurations, custom drivers, etc. But, Fu needs to
see and have access to some of Bar's results. These problems can be solved with the
introduction of the group access.

I realize this adds a layer of complexity which may be necessary. An alternative would be access control lists (acl). These can be even more flexible but a lot more complicated to implement, manage and maintain.

I think we should decide on an access control model soon and work out an implementation. Even with the current implementation, retrofitting will be difficult and somewhat tedious which will become even more so as we add more features.

[DAVID] The intermediate near term solution is to consider the 1.7 as working with only trusted users, in that they are all already within the same intranet. Under this limitation, if user Foo needs their work fully separate from user Bar, they could just have separate instances and databases of Toaster running, and use existing Linux security to accomplish this.

That said, we could provide for the default build/project view to default to the current "owner", where they would have to go out of their way to other people's work.

We could also consider an "audit log" that captures project creation (and especially project deletion), so that at least you know if someone stomped on your stuff :-)

[BELEN] I am with David on this one: I think we can find temporary alternatives. It's not that I disagree with Farrell, mind: he is completely right that we will need this. Problem is: we need to be mindful of what we can deliver with the resources we have.

=== Ability to identify build host and build directory ===

To support distributed build hosts sharing a common Toaster database, there needs to be a way in the database to identify which host (IP address and or host name) and local directory the project was built in.

===== Discussion =====

[ALEX] I am not sure I follow the use case / user story here. Can we get more details on why this is important ?

=== Ability to download files (YP #6096) ===

While developers would presumably have access to all build files within the intranet for the build logs and generated target image files, there are cases where that is not so. For example, the owner may be using a Windows host or a portable device to access Toaster, and may wish to download and view the error logs to determine their next course of action. Also, it may be the case that the specific build host is accessible only to the build farm manager and not the individual project owner, and having Toaster serve the file(s) would resolve this barrier.

===== Discussion =====

[FARRELL] The ability to download the result of the build should be somewhere - maybe a column on the All Builds page?

[DAVID] As for things like file downloads, between Ravi and myself I think we will find time to get that one done since we both really want it!

[BELEN] Absolutely: in fact, this is something I might start designing straight away so that it can be implemented in 1.7M1. However, in order to do that, we need to reach agreement as to what a build artifact is:

* Is it just the rootfs files? Or do they include
* the cooker and task logs?
* shared state objects used?
* recipe files / patches?
* [yours here]

=== Share projects via configuration (export / import) ===

This is the capture and sharing of the minimal project configuration data, such that a remote developer can replicate the same project against their own Yocto Project installation.

===== Discussion =====

[DAVE] The video ended at the export project discussion. Is export just sending email, the conf file (or files) to someone, or is there be more to it?

[BELEN] I am not sure what the answer is: we need input from Paul and Alex here. In my head, the project configuration (not the builds) is exported to some kind of file(s) that I can then share with others in whichever way I see fit (email, file sharing, etc).

=== Definition of a Project ===

The proposed design adds the new concept of the "Project".

===== Discussion =====

[PAUL] To my mind, the project is simply a configuration set - if you had a build farm the actual build could be run on any server within that farm and that side of it would be largely hidden from the user (until it needed to be visible, e.g. when you need to diagnose a host contamination issue.) This then means that the management of build directories can be independent, i.e. old build directories can be automatically deleted after a time period or when space runs low, but the project remains in the database for later use when needed.

[DAVID] I am completely willing to accept your definition of a "Toaster Project". The part that convinces me is that (a) while under active build and development it does map to a physical location, but (b) it can hold the data and presumably the "snapshot" artifacts of a physical build directory long after that actual directory has been deleted. That last part is a highly desired feature from the Wind River team, and was also brought up as a specific question at my Toaster presentation at the Yocto Project Summit.

This implies that given the abstract nature of a Toaster "Project", it makes sense that not only can the developer share that "project" with another developer and Toaster instance (as per Belén design), but it makes sense that the original developer can make their own clones of that project when they for example want use as a reference basis for new projects with additional adjustments and/or for binding and running on a different host/directory.

===== Workflow around creating a Project =====

[DAVID] Here is what I would propose as a workflow.

1) When a Toaster Project is created,

* The "user" is the preset to the last "user", else if this is the first time it is defaulted to $USER. This is a required field.

* The "email" is preset to the last email that matches "user", else it is empty. QUESTION: Is this a required field?

* The host:dir values are preset just as "poky/oe-init-build-env" would preset these values, in other words to the current machine and to "poky/../build" (unless redirected with the appropriate environment variables). If that directory (a) already exists and has a "conf/local.conf" file and (b) that directory is already registered with an existing Project, then that is an error. This would provide some sanity check, but would also allow us to re-connect to an unregistered YP build directory (and/or ones previous registered with a subsequently deleted project).

* We either have an explicit "Project Create" button, or we allow for the first build target command to insure that for new projects the project directory is created and the content preset by "poky/oe-init-build-env" before continuing.

2) For external hosts mounted via NFS:

* If an external host is mounted via a YP compatible NFS system, then that would be covered by the normal workflow above, using the local host and the respective NFS path. QUESTION: do we wish to identify the actual remove HOST ID in these circumstances, or just let the NFS path speak for itsel

3) For fully external hosts

This is the situation where a host remote to the Toaster, but is intended to be managed by this Toaster and use this Toaster's database.

* This implies using some sort of rpc, probably via SSH, to execute the creation and build targets.

* There would presumably be an installation of YP on that remote host, and that location would need to be known to Toaster. The installation could be as simple as an NFS mount that points to the local YP installation, which would provide a single source at the cost of NFS traffic.

== Toaster System Architectures ==

This is an iteration of the possible supported system architectures of a Toaster implementation. We should validate how the proposed design would work with each of these options.

=== Minimal Local Project Implementation ===

Configuration: Local host, local project, database in project, toaster and web server started from project

This is the default implementation, using the "source toaster [start|stop]" support.

=== Multiple Local Project Implementation ===

Configuration: Local host, Multiple local projects, shared local database, toaster and web server started separately

This is currently supported by (a) edits to the local.conf file to point to the shared database location, and (b) using special commands to start the shared Toaster and webserver instance.

=== Multiple Distributed Host Implementation ===

Configuration: Multiple hosts, Multiple local projects per host, shared network database, toaster and web server started with network database instance

Each host would either have its own installation of YP, or have a centralized copy shared over NFS. This implementation is currently not yet documented nor supported.

== Open Bugzilla features / defects ==

[http://wiki.yoctoproject.org/wiki/Bugzilla_feature_list This is what's open in Bugzilla right now]. Apart from what's already assigned to 1.6.1, there are 2 issues that I am keen on getting fixed:

[https://bugzilla.yoctoproject.org/show_bug.cgi?id=5187 5187] - Same cooker log file name used for several builds 
[https://bugzilla.yoctoproject.org/show_bug.cgi?id=6095 6095] - Retrieve full build information independently of task execution

Feel free to add to this list.

== Future features ==

A list of features that have come up in discussion, but not for the 1.7 release.

=== Delete projects ===

This option deletes the project all the way down to the project's instance on the disk. This should be a safe operation unless a separate project is directly sharing its sstate cache. That would be an recommended usage, where a proper usage would a standalone shared sstate cache. There may be artifacts in that shared sstate cache that are specific and unique to the deleted project, but it is out of scope for this option to try to delete those as well.

=== Ability to add/remove recipes ===

This is the ability to add or remove recipes from the project's configuration.

=== Ability to import source (tarballs, repos) ===

This is the ability to bring external sources into the project and include them with wrapper recipes.

=== Ability to add/remove packages from image ===

This is the ability to add or remove specific binary packages from the respective image(s) of a project.

=== Share projects via Copy (all but "./tmp") ===

Project directories (all except for their local "tmp" directory) should be portable between hosts, where the receiving developer needs only to adjust the local.conf file for the local installation and sstate directories.

=== Ability to get notification of build status ===

Since builds can take a long time, it would be very useful to get for example email notification when a build stops, either with success or with failure, rather than needing to continually and manually poll the Toaster interface.

===== Discussion =====

[DAVE] I agree, but can we generalize this post-build processing to enable post-build script execution triggered by either success, failure, or either? We could provide a few parameterized scripts to, for example:

* send email to a user or list with build status,
* (on success) start an emulation session with regression tests on a build,
* (on success) download the built file system or SDK to a user's host or target,
* provide hooks for the user to substitute their alternative post-build triggers.

=== Ability to access Toaster from portable device ===

Since builds generally take a long time, the user may be away from their desk (or even work hours) and wish to track the status of their pending builds. Supporting a basic Toaster interface for such limited devices would be advantageous.

=== Ability to compare projects/builds ===

A user may not recall the difference between one build and another, or one project and another. The data is in the database, so it would not be hard to generate a report itemizing the observed differences. Here are some the potential changes that can be extracted:

* Changed variable definitions (less base path differences). This would also cover changes in things like the machine, layers, extra added or excluded recipes, and multi-lib support.
* Changes in the included packages list
* Perhaps changes in the build/caching statistics

=== Snapshots of builds for long term archiving (project dir lifecycle) ===

A build manager may wish to keep the pertinent facts of builds for an extended period of time. This could be for a few days to allow time for the build's owner to reconcile the results, or if maybe over a year for tracking long-term trends and regressions. In each case, there should be a way to capture a minimal set of information from the build directory (e.g. the conf files and any build failure logs) and preserve it so that together with the database there is enough information to study that project even when the full build directory is removed and recycled.

Toaster 1.7 design

2014-04-25T09:30:53Z

Ddalex: /* Feature list */

== A couple of videos to kick off things ==

The following videos illustrate some of the features we are thinking of adding to Toaster during the 1.7 release cycle. When watching them, keep in mind that this is all very rough: more like sketches to help us start a conversation about scope than proper designs. Chances are the final designs will be completely different.

About projects - [[media:Building-1.7.mov]] 
Starting a build - [[media:How-to-build.mov]]

== Feature list ==

=== Ability to create new projects ===

This is the ability to use the Toaster interface to create new project directories, manage the content, and initiate the builds of selectable target images.

===== Summary of discussion =====

* A project is a set of layers and configuration variables values that describes how to do a build.
* Creating a project means writing up values for the above-mentioned set, and store these values in a persistent manner.
* A build for a project means effecting (instantiating) the build process for the current set of values.
* Build directory location (host/machine) is not relevant to Toaster.
* Saving the build artifacts and providing access to the build artifacts is critical for Toaster.

===== Discussion =====

[DAVE] What is a typical workflow that gets to the build project part? I see the configure screens but not the actually build launch.

[DAVE] Does clicking 'new build' fire off a build? If so, which configuration is built?

[BELEN] Heh, I completely forgot to show that in the first video. [http://wiki.yoctoproject.org/wiki/images/b/b1/How-to-build.mov Here is part 2] showing how to start a build and answering (I hope) both questions.

[DAVE] Can projects be configured, primed for builds, but not built?

[BELEN] Yes.

[DAVE] Will the user be able to configure several builds, then queue them all to build?

[BELEN] Users might create more than one project then start a build for each of them by clicking the "build" button in each project page, but nothing more sophisticated than that for the time being. Here is where things can get tricky. Where does Toaster stop and something like Buildbot or Jenkins start? My take is that we want to work alongside and enhance build automation systems, not replace them.

[DAVE] What is the size of the build variable list? It's hard to figure from the conf/local.conf file. If it is a small list, like 10, should we provide better and variable specific UI widgets rather than just edit boxes, depending on the variable's "type" (eg browse button with any path variable like SSTATE_DIR)?

[BELEN] This is one of the questions we should start working on straight away. In informal chats we have spoken about a split between "build variables" (the ones that affect the build and therefore you can edit via Toaster) and "infrastructure variables" (to call them something, that affect the build infrastructure and you cannot edit via Toaster. Things like PARALLEL_MAKE for example). We should start getting into details about that split as soon as possible, and determine which variables we are going to expose via Toaster to be edited. Once again, Paul's feedback here would be good (RP's probably as well).

Regarding variable "types", I had a brief conversation with RP recently about this issue. It is unlikely that the meta data will provide us with information about the type of input expected by a variable. If and how the Toaster interface should fill that gap is up for discussion. From my perspective, we should do as much as we can to help users assign valid values to their variables.

[DAVE] I don't recall a user entry on your videos that specify the project directory for the build. This must be part of the initial release. I don't recommend making it an edited data file option, eg not via in settings.py.

[BELEN] When you run Toaster locally you can configure the location of your build directory as you wish, but when Toaster is running remotely, do we want to let people select the location of their build directory?

[DAVE] Can we make any selection on any page 'stick' with cookies or 'unstick' with a 'reset to defaults' button, in addition to propagating laborious configuration setups inputs via 'clone configuration'?

[BELEN] We will need to overcome my personal hate of 'reset to defaults' options :) They are very dangerous if we don't provide an 'undo' option as well: they might wipe your configuration and revert it to the rather useless default one, losing a bunch of work you did beforehand. Any configuration changes you make are saved: in that sense, they are 'sticky'. If I create project A, then I add 2 layers to the project configuration, select a machine and 3 targets, all those options are saved to the project configuration. You don't need to add the 2 layers, select the machine and the 3 targets every time you want to run a build.

[RAVI] I am wondering about the purpose of a project. In the design, project has a machine and owner attributes. If we define a project as a collection of builds for a machine, then we should not allow to change the machine after a project is created. Then, if a project is just a collection of builds for a particular machine, then we can modify the existing all builds view to group builds by machine. Based on this argument, I do not see a need for project and build hierarchy.

[DAVID] My group understands that a "project" is a specific build directory. A project will most likely to have multiple builds, and things like the (a) machine, (b) layers, (c) conf variables, and so forth would be presumably set before the first target is selected and executed. It is also the case that the developer may decide to change any of those settings, including even the machine type, and then carry on with more builds. The use case would be for example is arch coverage testing of a particular set of kernel or user space packages.

A host can then have multiple project directories (I most often work that way), for example when a developer truly wants to keep their development and/or debugging from not polluting other projects.

Also, separate users should do their work in separate project directories, now that we are extending the model to cover that.

There can then be multiple hosts, with all of their multiple projects, all channeling their Toaster data into a shared database. And with that you have a build farm.

[RAVI] I see that the uniqueness of a project is the <hostname>://<build directory path>. The contents of the build directory can be changed and multiple builds can be kicked off from the same project with different configurations. If this the case, the project creation should also setup a build directory on the host.

Looking at this use case, we should have toaster running on the host machine with a shared database. Or if we run a single toaster instance serving multiple projects (hosts and build directories), we should have a mechanism for the toaster machine to talk to the project machines. Am I missing something?

[PAUL] Yes, but the project need not be tied to a specific host or location on disk. To my mind, the project is simply a configuration set - if you had a build farm the actual build could be run on any server within that farm and that side of it would be largely hidden from the user (until it needed to be visible, e.g. when you need to diagnose a host contamination issue.) This then means that the management of build directories can be independent, i.e. old build directories can be automatically deleted after a time period or when space runs low, but the project remains in the database for later use when needed.

[DAVID] I am completely willing to accept the above definition of a "Toaster Project". The part that convinces me is that (a) while under active build and development it does map to a physical location, but (b) it can hold the data and presumably the "snapshot" artifacts of a physical build directory long after that actual directory has been deleted. That last part is a highly desired feature from the Wind River team, and was also brought up as a specific question at my Toaster presentation at the Yocto Project Summit.

This implies that given the abstract nature of a Toaster "Project", it makes sense that not only can the developer share that "project" with another developer and Toaster instance (as per Belén design), but it makes sense that the original developer can make their own clones of that project when they for example want use as a reference basis for new projects with additional adjustments and/or for binding and running on a different host/directory.

Here is what I would propose is the workflow.

1) When a Toaster Project is created,

* The "user" is the preset to the last "user", else if this is the first time it is defaulted to $USER. This is a required field.

* The "email" is preset to the last email that matches "user", else it is empty. QUESTION: Is this a required field?

* The host:dir values are preset just as "poky/oe-init-build-env" would preset these values, in other words to the current machine and to "poky/../build" (unless redirected with the appropriate environment variables). If that directory (a) already exists and has a "conf/local.conf" file and (b) that directory exists for an existing Project, then that is an error. This would allow us to re-connect to a deleted or unregistered YP build directories.

* We either have an explicit "Project Create" button, or we allow for the first build target command to insure that for new projects the project directory is created and the content preset by "poky/oe-init-build-env" before continuing.

2) For external hosts mounted via NFS:

* If an external host is mounted via a YP compatible NFS system, then that would be covered by the normal workflow above, using the local host and the respective NFS path. QUESTION: do we wish to identify the actual remove HOST ID in these circumstances, or just let the NFS path speak for itself?

3) For fully external hosts

This is the situation where a host remote to the Toaster, but is intended to be managed by this Toaster and use this Toaster's database.

* This implies using some sort of rpc, probably via SSH, to execute the creation and build targets.

* There would presumably be an installation of YP on that remote host, and that location would need to be known to Toaster. The installation could be as simple as an NFS mount that points to the local YP installation, which would provide a single source at the cost of NFS traffic.

=== Hide builds from a project ===

This option simply removes or hides selected builds from the Toaster display, for example builds quickly halted because the owner realized that they missed a setting. The database records and the project on the disk is untouched. This un-clutters the display, yet requires little programing (and development time).

===== Discussion =====

[BELEN] I have no strong feelings against this one, but we need to bear in mind that:

* It will probably be hard to allow hiding in bulk, since the reasons why someone might want to hide a build could be wildly different. So this will have to be done on a build by build basis.

* This also involves providing a way to reverse the hiding action, i.e. see hidden builds and show them again.

[DAVID] In the Wind River version of Wiki, we have a "Manage" button at the bottom. This brings up a page of all of the entries, with the ability to change the entry's hidden and description attributes. This does allow a bulk update easily.

I propose that we do the same, where a "Manage" button brings up all of the builds and/or projects, each highlighted according to if it is currently hidden, also each with a text entry for the description (see the next design proposal) and a check box to toggle the hidden state.

We then need to decide if this meta date is stored in the database or in the page's cookie.

=== Ability to add a description to a build or project ===

User's may not recall the difference and purpose of a given build or project, so it would be good for the Toaster interface to allow the user to add this and update this as needed.

[DAVID] See the discussion above about "hidden" builds for a proposed implementation.

=== Delete builds from a project ===

This option is where a selected build and all its related database records are removed from the database. This removes the database overhead for this unwanted builds. The project on the disk is untouched. Since the database should be normalized, this operation should be clean and not interfere with other builds. I believe we have a command line version of this operation with Toaster-1.6.

===== Discussion =====

[Alex] Technically, this is straightforward. In terms of policy access, we need to define one; I propose that only the project owner is allowed to delete builds. Impersonation is trivial right now, but once we implement authentication, bricks will fall in the right place in terms of access control.

=== Ability to add / remove layers ===

Add and delete from the project layer directories. These layers may be within repositories or in plain directories on the disk.

===== Discussion =====

[FARRELL] How would I upload my specialized device driver/library/system call handler/etc that necessarily needs to be part of
the build? Would this be done by importing a layer?

[BELEN] Yes. For the 1.7 release, all customisation must be done via layers.

[DAVE] Are you constraining 'import layer' to a git repo path? Why not just a local directory that is not a git repo? In that case should we have a browse button as well as an edit box?

[BELEN] This is a really good question. I need the technical crowd to establish what's possible / what we want to do. We can then design accordingly.

[DAVID] It is the case that we recommend all of our customers to add their content via layers, and specifically layer directories since they are most likely to be under continual and direct change and would not be placed into a formal repo, or more likely they would be managed by their SCM system as is and certainly not packed into a base git clone.

=== Ability to identify / set project owner / user ===

A build farm manager (and their shared Toaster instance) may accept build sets from clients, in which case it is important to know who initiates and owns each specific build, for completion notifications and project space recycling.

===== Discussion =====

[FARRELL] You presented the notion of a user - in the movie it was 'Belen'. I think it may be necessary to formalize this
idea with a login screen and at least rudimentary permission and protection. Here is a use case: Imagine a vendor
deploys Toaster on a server which serves all of its customers. The customers are very diverse and are often competitors
of one another and do not want their configuration/information/ideas shared with other vendor customers.

[BELEN] Very true: access control is a big deal and very important. However, we are not sure we can tackle this in the 1.7 release. That's why in the video I speak of "owners" instead of "users". Our "owner" concept derives from the need to identify who created a certain project, but they are not really users since, for example, they don't involve authentication (there is no password).

Here is a possible description of these "owners". Each client accessing a Toaster instance might have an owner linked to it. You don't explicitly create these owners: you just set them in the process of creating your first project. Then, we remember them. Owners involve only 3 pieces of information: a name, an optional email address and the client they are associated with. You can change the details at any time. So "owners" don't involve access control: they just tell me who created (and therefore owns) a certain project. You can have the same owner details in several clients: I might have a laptop and a builds machine, I access the same Toaster instance from both, and I can use the same owner name and email address in both.

The above opens some questions:

* can others edit the configuration of a project I own?
* can others start builds against a project I own?
* can others download artifacts of a project I own?
* can others even see the projects I own?

The simplest thing would be either:

* 'yes' to all of the above, but that means that people creating Toaster instances that can be accessed by more than one person have to be mindful of the fact that everything in them is open.

* 'no' to all of the above, and channel all collaboration through the export / import projects functionality, which is safer but also stifles co-operation.

Just to clarify, ownership as above is established per project, not per build.

[FARRELL] As much as I hate to suggest this...It may be necessary to introduce the notion of group access - Owner/Group/Others ala unix/linux. I think it may be necessary for users to be able to share their results while excluding others. I can imagine a number of use cases. Here is one:

Two customers/users, Fu and Bar, work for the same company C. Each wants to
be able to have personal 'experimental' builds but may need to share a 'production'
build - react to errors and download results, etc. Of course, company C doesn't want
company B to see their results, configurations, custom drivers, etc. But, Fu needs to
see and have access to some of Bar's results. These problems can be solved with the
introduction of the group access.

I realize this adds a layer of complexity which may be necessary. An alternative would be access control lists (acl). These can be even more flexible but a lot more complicated to implement, manage and maintain.

I think we should decide on an access control model soon and work out an implementation. Even with the current implementation, retrofitting will be difficult and somewhat tedious which will become even more so as we add more features.

[DAVID] The intermediate near term solution is to consider the 1.7 as working with only trusted users, in that they are all already within the same intranet. Under this limitation, if user Foo needs their work fully separate from user Bar, they could just have separate instances and databases of Toaster running, and use existing Linux security to accomplish this.

That said, we could provide for the default build/project view to default to the current "owner", where they would have to go out of their way to other people's work.

We could also consider an "audit log" that captures project creation (and especially project deletion), so that at least you know if someone stomped on your stuff :-)

[BELEN] I am with David on this one: I think we can find temporary alternatives. It's not that I disagree with Farrell, mind: he is completely right that we will need this. Problem is: we need to be mindful of what we can deliver with the resources we have.

=== Ability to identify build host and build directory ===

To support distributed build hosts sharing a common Toaster database, there needs to be a way in the database to identify which host (IP address and or host name) and local directory the project was built in.

=== Ability to download files (YP #6096) ===

While developers would presumably have access to all build files within the intranet for the build logs and generated target image files, there are cases where that is not so. For example, the owner may be using a Windows host or a portable device to access Toaster, and may wish to download and view the error logs to determine their next course of action. Also, it may be the case that the specific build host is accessible only to the build farm manager and not the individual project owner, and having Toaster serve the file(s) would resolve this barrier.

===== Discussion =====

[FARRELL] The ability to download the result of the build should be somewhere - maybe a column on the All Builds page?

[DAVID] As for things like file downloads, between Ravi and myself I think we will find time to get that one done since we both really want it!

[BELEN] Absolutely: in fact, this is something I might start designing straight away so that it can be implemented in 1.7M1. However, in order to do that, we need to reach agreement as to what a build artifact is:

* Is it just the rootfs files? Or do they include
* the cooker and task logs?
* shared state objects used?
* recipe files / patches?
* [yours here]

=== Share projects via configuration (export / import) ===

This is the capture and sharing of the minimal project configuration data, such that a remote developer can replicate the same project against their own Yocto Project installation.

===== Discussion =====

[DAVE] The video ended at the export project discussion. Is export just sending email, the conf file (or files) to someone, or is there be more to it?

[BELEN] I am not sure what the answer is: we need input from Paul and Alex here. In my head, the project configuration (not the builds) is exported to some kind of file(s) that I can then share with others in whichever way I see fit (email, file sharing, etc).

=== Definition of a Project ===

The proposed design adds the new concept of the "Project".

===== Discussion =====

[PAUL] To my mind, the project is simply a configuration set - if you had a build farm the actual build could be run on any server within that farm and that side of it would be largely hidden from the user (until it needed to be visible, e.g. when you need to diagnose a host contamination issue.) This then means that the management of build directories can be independent, i.e. old build directories can be automatically deleted after a time period or when space runs low, but the project remains in the database for later use when needed.

[DAVID] I am completely willing to accept your definition of a "Toaster Project". The part that convinces me is that (a) while under active build and development it does map to a physical location, but (b) it can hold the data and presumably the "snapshot" artifacts of a physical build directory long after that actual directory has been deleted. That last part is a highly desired feature from the Wind River team, and was also brought up as a specific question at my Toaster presentation at the Yocto Project Summit.

This implies that given the abstract nature of a Toaster "Project", it makes sense that not only can the developer share that "project" with another developer and Toaster instance (as per Belén design), but it makes sense that the original developer can make their own clones of that project when they for example want use as a reference basis for new projects with additional adjustments and/or for binding and running on a different host/directory.

===== Workflow around creating a Project =====

[DAVID] Here is what I would propose as a workflow.

1) When a Toaster Project is created,

* The "user" is the preset to the last "user", else if this is the first time it is defaulted to $USER. This is a required field.

* The "email" is preset to the last email that matches "user", else it is empty. QUESTION: Is this a required field?

* The host:dir values are preset just as "poky/oe-init-build-env" would preset these values, in other words to the current machine and to "poky/../build" (unless redirected with the appropriate environment variables). If that directory (a) already exists and has a "conf/local.conf" file and (b) that directory is already registered with an existing Project, then that is an error. This would provide some sanity check, but would also allow us to re-connect to an unregistered YP build directory (and/or ones previous registered with a subsequently deleted project).

* We either have an explicit "Project Create" button, or we allow for the first build target command to insure that for new projects the project directory is created and the content preset by "poky/oe-init-build-env" before continuing.

2) For external hosts mounted via NFS:

* If an external host is mounted via a YP compatible NFS system, then that would be covered by the normal workflow above, using the local host and the respective NFS path. QUESTION: do we wish to identify the actual remove HOST ID in these circumstances, or just let the NFS path speak for itsel

3) For fully external hosts

This is the situation where a host remote to the Toaster, but is intended to be managed by this Toaster and use this Toaster's database.

* This implies using some sort of rpc, probably via SSH, to execute the creation and build targets.

* There would presumably be an installation of YP on that remote host, and that location would need to be known to Toaster. The installation could be as simple as an NFS mount that points to the local YP installation, which would provide a single source at the cost of NFS traffic.

== Toaster System Architectures ==

This is an iteration of the possible supported system architectures of a Toaster implementation. We should validate how the proposed design would work with each of these options.

=== Minimal Local Project Implementation ===

Configuration: Local host, local project, database in project, toaster and web server started from project

This is the default implementation, using the "source toaster [start|stop]" support.

=== Multiple Local Project Implementation ===

Configuration: Local host, Multiple local projects, shared local database, toaster and web server started separately

This is currently supported by (a) edits to the local.conf file to point to the shared database location, and (b) using special commands to start the shared Toaster and webserver instance.

=== Multiple Distributed Host Implementation ===

Configuration: Multiple hosts, Multiple local projects per host, shared network database, toaster and web server started with network database instance

Each host would either have its own installation of YP, or have a centralized copy shared over NFS. This implementation is currently not yet documented nor supported.

== Open Bugzilla features / defects ==

[http://wiki.yoctoproject.org/wiki/Bugzilla_feature_list This is what's open in Bugzilla right now]. Apart from what's already assigned to 1.6.1, there are 2 issues that I am keen on getting fixed:

[https://bugzilla.yoctoproject.org/show_bug.cgi?id=5187 5187] - Same cooker log file name used for several builds 
[https://bugzilla.yoctoproject.org/show_bug.cgi?id=6095 6095] - Retrieve full build information independently of task execution

Feel free to add to this list.

== Future features ==

A list of features that have come up in discussion, but not for the 1.7 release.

=== Delete projects ===

This option deletes the project all the way down to the project's instance on the disk. This should be a safe operation unless a separate project is directly sharing its sstate cache. That would be an recommended usage, where a proper usage would a standalone shared sstate cache. There may be artifacts in that shared sstate cache that are specific and unique to the deleted project, but it is out of scope for this option to try to delete those as well.

=== Ability to add/remove recipes ===

This is the ability to add or remove recipes from the project's configuration.

=== Ability to import source (tarballs, repos) ===

This is the ability to bring external sources into the project and include them with wrapper recipes.

=== Ability to add/remove packages from image ===

This is the ability to add or remove specific binary packages from the respective image(s) of a project.

=== Share projects via Copy (all but "./tmp") ===

Project directories (all except for their local "tmp" directory) should be portable between hosts, where the receiving developer needs only to adjust the local.conf file for the local installation and sstate directories.

=== Ability to get notification of build status ===

Since builds can take a long time, it would be very useful to get for example email notification when a build stops, either with success or with failure, rather than needing to continually and manually poll the Toaster interface.

===== Discussion =====

[DAVE] I agree, but can we generalize this post-build processing to enable post-build script execution triggered by either success, failure, or either? We could provide a few parameterized scripts to, for example:

* send email to a user or list with build status,
* (on success) start an emulation session with regression tests on a build,
* (on success) download the built file system or SDK to a user's host or target,
* provide hooks for the user to substitute their alternative post-build triggers.

=== Ability to access Toaster from portable device ===

Since builds generally take a long time, the user may be away from their desk (or even work hours) and wish to track the status of their pending builds. Supporting a basic Toaster interface for such limited devices would be advantageous.

=== Ability to compare projects/builds ===

A user may not recall the difference between one build and another, or one project and another. The data is in the database, so it would not be hard to generate a report itemizing the observed differences. Here are some the potential changes that can be extracted:

* Changed variable definitions (less base path differences). This would also cover changes in things like the machine, layers, extra added or excluded recipes, and multi-lib support.
* Changes in the included packages list
* Perhaps changes in the build/caching statistics

=== Snapshots of builds for long term archiving (project dir lifecycle) ===

A build manager may wish to keep the pertinent facts of builds for an extended period of time. This could be for a few days to allow time for the build's owner to reconcile the results, or if maybe over a year for tracking long-term trends and regressions. In each case, there should be a way to capture a minimal set of information from the build directory (e.g. the conf files and any build failure logs) and preserve it so that together with the database there is enough information to study that project even when the full build directory is removed and recycled.

Toaster 1.7 design

2014-04-16T17:50:30Z

Ddalex: /* Delete builds from a project */

Toaster 1.7 design

2014-04-16T17:48:56Z

Ddalex: /* Delete builds from a project */

Contribute to Toaster

2014-01-23T15:05:07Z

Ddalex:

This page summarises the Toaster development process. We hope this will help you start contributing to the project.

== Set up the local repository ==

* Select a [http://www.yoctoproject.org/docs/1.5.1/ref-manual/ref-manual.html#detailed-supported-distros Yocto Project 1.5 compatible host], [https://www.djangoproject.com/download/ install Django-1.5] and South 0.8.4. The "pip" application is recommended to manage the install process.

<code>
$ sudo apt-get install pip
$ sudo pip uninstall django
$ sudo pip install django==1.5
$ sudo pip install South==0.8.4
</code>

* Setup a local repository for the development branch

<code>
$ cd <installdir>
$ git clone git://git.yoctoproject.org/poky
$ cd poky
$ git remote add contrib http://git.yoctoproject.org/git/poky-contrib
$ git fetch contrib
$ git checkout contrib/toaster/master -b toaster-master
</code>

* Setup up your branch for pushes to the Yocto Project [http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/ poky-contrib repository]

<code>
$ git remote set-url contrib git@git.yoctoproject.org:poky-contrib
</code>

== Set up the project and Toaster interface ==

* Run a build, with Toaster database capture enabled

<code>
$ cd <installdir>
$ source poky/oe-init-build-env
$ source toaster start
$ bitbake core-image-minimal
</code>

NOTE: Toaster MUST be started before you start your build, else no data will be captured. You can recover a working (if sparse) database if you do this to execute a quick re-build.

<code>
$ source toaster start
$ bitbake -c cleansstate base-files
$ bitbake core-image-minimal
</code>

* Run the Toaster interface

<code>
$ xdg-open http://localhost:8000/
</code>

NOTE: You can alternatively open your browser manually to <code>http://localhost:8000/</code>

== Edit and submit content for review ==

* Create a local branch. The branch name is generally of the form <code><username>/<a_name_for_the_branch></code>. For example: <code>dreyna/recipe-detail-view</code>. You can choose any user name and send it to Michael Halstead (mhalstead at linuxfoundation dot org), together with your SSL public key to enable your pushes to the Yocto Project [http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/ poky-contrib repository]. For example:

<code>
$ git checkout -b dreyna/recipe-detail-view
</code>

* Edit and test your content. You mind find [https://getfirebug.com/ Firebug] useful for web development purposes. All rendered pages should be validated for HTML format compliance. There are lots of HTML validators you can use: [http://users.skynet.be/mgueury/mozilla/ HtmlValidator] is one of them.

* Set up your commit(s). The same push can have several partitioned commits.

<code>
$ cd <installdir>/poky
$ git add bitbake/lib/toaster/...
$ git commit -s
</code>

Use <code>$ git commit -s</code> so that you don't need to add the <code>Signed-off-by</code> manually to your patches.

You might also find <code>$ git add -p [filename]</code> helpful. It will allow you to review multiple changes to a single file one by one.

NOTE: The format of the commit should be like this

<code>
vvvvvvvvvvvvvvvvvvvvvvvvv
<short one line summary>
 
<long(er) description, can be multi-line, should break at around 60 chars>
 
[YOCTO #0000] # OPTIONAL LINE: replace with the real bugzilla issue number
 
Signed-off-by: First Last <name@company.com>
^^^^^^^^^^^^^^^^^^^^^^^^^
</code>

If your patch is directly addressing a Bugzilla issue, you should reference the issue number by adding the <code>YOCTO #0000</code> line just above the <code>Signed-off</code> line.

A comprehensive document about commit messages is available at:

http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines

* Rebase your branch on the latest toaster/master. This will lower the probability that your code will conflict when trying to merge the patch upstream by making sure it's based on the latest upstream.

* Push your branch for review. For example, if you branch name is <code>dreyna/recipe-detail-view</code>:

<code>
$ git push poky-contrib dreyna/recipe-detail-view
</code>

* Send an email to the [https://lists.yoctoproject.org/listinfo/webhob Toaster mailing list] with the following content:

* Once the email comes back that the patches are merged, please abandon the branch, and start new development on a new branch.

** A brief description of the review request together with the branch name
** Any technical details to call out to reviewers
** Any limitations, assumptions, dependencies, and/or differed work
** Ideally, a test plan that demonstrates how the feature was tested with sufficient detail for general testers and documentation writers. [https://lists.yoctoproject.org/pipermail/toaster/2014-January/000280.html This is a test plan example] you can use as a reference.

* Update the Bugzilla entry. If your patch is directly addressing a Bugzilla issue, please mark the Bugzilla entry as "In Progress Review", and when the patch is merged into upstream <code>bitbake/master</code> (not <code>poky-contrib/toaster/master</code>), please mark the Bugzilla entry as "Resolved / Fixed". This will let the QA know what happens to the code base and the Bugzilla entries.

== Rebase your repository from master ==

To update your repository to the latest content, rebase it (as opposed to attempted a merge).

<code>
$ cd <installdir>/poky
$ git fetch
$ git rebase [-i] poky-contrib/toaster/master
</code>

Analysis REST API Contracts

2013-11-18T12:01:50Z

Ddalex:

[[Category:Toaster]]

* Implementation is released on http://git.yoctoproject.org/cgit/cgit.cgi/poky/log/?h=dora-toaster and http://git.yoctoproject.org/cgit/cgit.cgi/poky/log/?h=master

This page gives you information on the Toaster Analysis REST API. This is a read-only API which provides information about a build that was recorded by Toaster.

It documents the search operation endpoints, parameters, and responses. You might also be interested in the [[Toaster#Installation and Running|Toaster installation and running instructions]].

Toaster is accessible through REST APIs, which make the backend available to any client.
The data collected during the build is available via an unrestricted, anonymous, read-only API running on the local host machine.

== General ==

The data interface is available as a search query returning a list of typed objects.
The request method must be GET. The answers are always returned as JSON.

=== API version ===

The REST API is versioned as to allow further upgrades without having to maintain backward compatibility through changing the major version number.
Minor version numbers are intended to signify a capability level while maintaining compatibility.

This is the API version 1.0. All API requests are grouped under URL:

<code>GET /api/1.0/</code>

=== Object types ===

The API can return [[REST API Contracts#Build|Build]], [[REST API Contracts#Target|Target]], [[REST API Contracts#Target installed packages|Target installed packages]], [[REST API Contracts#Layer|Layer]], [[REST API Contracts#Layer version|Layer version]], [[REST API Contracts#Task|Task]], [[REST API Contracts#Task dependencies|Task dependencies]], [[REST API Contracts#Recipe|Recipe]], [[REST API Contracts#Recipe dependencies|Recipe dependencies]], [[REST API Contracts#Build packages|Build packages]], [[REST API Contracts#Package dependencies|Package dependencies]], [[REST API Contracts#Package files|Package files]], [[REST API Contracts#Variables|Variables]] and [[REST API Contracts#Log messages|Log messages]]. Each object is described below.

=== Search operation ===

The primary operation of the API is to retrieve a set of objects from the backend based on a search operation. The result will also contain all the data for the objects being returned, as to avoid separate connections to further bring the objects of the type queried.

The search operation can also specify the ordering of the results by key.

The search parameters are the same for all object types.

==== URI ====

All the search endpoints follow the same URI pattern

<code>
GET /api/1.0/{on}s/?{parameters}
</code>


* The on is the object name, one the endpoints described below.
* The parameters are described in the next section, and are the same for all search queries

==== Parameters ====

{| class="wikitable"
|-
! Name !! Type !! Description !! Comments
|-
| limit || number || The number of objects to be returned || -
|-
| offset || number || Offset into the search result index. || -
|-
| search || string || Search string for all fields || The argument is the string to be searched in the database. The backend MUST return only records that match the string.
|-
| filter || string || String used to specify result filter function || The general form: "<FIELD>:<VALUE>[,<FIELD>:<VALUE>]*". The backend MUST only return records that have FIELD matching to the VALUE specified. The <FIELD> names for each object type are the property names of the object.
|-
| orderby || string || The name of the field that determines the sorting of the results. || "<FIELD>:<ORDER_DIRECTION>". The results are ordered by the <FIELD> values in the <ORDER_DIRECTION> order. <ORDER_DIRECTION> values may be "asc" or "desc".
|}

==== Response ====

All the responses are in JSON format. The response contains a set of fields describing the content of the data, and an array with the object data inside it.

== Build ==

Returns information about the builds recorded by Toaster.

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/builds || JSON || Returns page size-limited and search criteria-filtered builds from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of builds to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.outcome || number || 2 || YES || - || Signals successful or failed build || 0 Failed build 1 Successful build
|-
| [Object Root].list[].fields.build_name || string || - || YES || - || Build name|| Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5187 5187]
|-
| [Object Root].list[].fields.machine || string || - || YES || - || The selected hardware || -
|-
| [Object Root].list[].fields.started_on || number representation of date || - || YES || - || Marks the moment the process is started ||
|-
| [Object Root].list[].fields.completed_on || number representation of date || - || YES || - || Marks the moment the process is completed ||
|-
| [Object Root].list[].fields.errors_no || number || - || YES || - || Number of errors thrown by the build || -
|-
| [Object Root].list[].fields.warnings_no || number || - || YES || - || Number of warnings thrown by the build || -
|-
| [Object Root].list[].fields.image_fstypes || string || - || YES || - || The file types create by this build || Space-separated file name suffixes. Known issues: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5453 5453]
|-
| [Object Root].list[].fields.cooker_log_path || string || - || YES || - || Path to log file || Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5187 5187]
|-
| [Object Root].list[].fields.distro || string || - || YES || - || Distro name || -
|-
| [Object Root].list[].fields.distro_version || string || - || YES || - || Distro version || -
|-
| [Object Root].list[].fields.bitbake_version || string || - || YES || - || BitBake version || -
|}

== Target ==

Returns information about a build target(s).

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/targets || JSON || Returns page size-limited and search criteria-filtered targets from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of targets to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.build || number || - || YES || - || The ID of the build that created this target || -
|-
| [Object Root].list[].fields.target || string || - || YES || - || The name of the build target || -
|-
| [Object Root].list[].fields.file_name || string || - || YES || - || Full path(s) to the root file system file(s) produced by the build (if any) || Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5189 5189]
|-
| [Object Root].list[].fields.file_size || number || - || YES || - || The file size of the build target || Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5228 5228]
|-
| [Object Root].list[].fields.is_image || boolean || - || YES || False || True if one of the build targets is an image recipe || -
|}

== Target installed packages ==

Returns information about the packages installed in a target, when such target is an image recipe.

NOTE: [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#maintaining-build-output-quality Build history] must be enabled to collect this information.

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/target_packages || JSON || Returns page size-limited and search criteria-filtered installed packages from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of packages to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.target || number || - || YES || - || The pk of the target that has this package || -
|-
| [Object Root].list[].fields.recipe || number || - || YES || - || The pk of the recipe that generated this package || Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5276 5276]
|-
| [Object Root].list[].fields.name || string || - || YES || - || Package name || -
|-
| [Object Root].list[].fields.version || string || - || YES || - || Package version || Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5276 5276]
|-
| [Object Root].list[].fields.size || number || - || YES || - || The size of the package, in bytes || -
|-
|}

== Layer ==

Returns information about the layers used in the builds recorded by Toaster.

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/layers || JSON || Returns page size-limited and search criteria-filtered layers from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of layers to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.name || string || - || YES || - || The local name under which the layer is known || -
|-
| [Object Root].list[].fields.local_path || string || - || YES || - || Path to the layer on local machine || -
|-
| [Object Root].list[].fields.layer_index_url || URL || - || NO || - || URL to the layer index application || Possibly not available. Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5192 5192]
|-
|}

== Layer version ==

Returns information about the version (branch:commit in Git) of a layer used in a certain build.

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/layerversions || JSON || Returns page size-limited and search criteria-filtered layers from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of layers to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.layer || number || - || YES || - || PK of the layer object || -
|-
| [Object Root].list[].fields.branch || string || - || YES || - || The branch name of the layer || -
|-
| [Object Root].list[].fields.commit || string || - || YES || - || The current commit of the layer || -
|-
| [Object Root].list[].fields.priority || number || - || NO || - || Priority of the layer || Known issues: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5218 5218]
|-
|}

== Task ==

Each build consists of a series of tasks. This is the endpoint for searching the task table.

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/tasks || JSON || Returns page size-limited and search criteria-filtered tasks from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of tasks to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.build || number || - || YES || - || Identifies the build in which the task occurred || -
|-
| [Object Root].list[].fields.order || number || - || YES || - || The sequence ID of the task in launch order ||
|-
| [Object Root].list[].fields.task_executed || boolean || - || YES || - || True if task actually executed || -
|-
| [Object Root].list[].fields.outcome || number || 6 || YES || 6 || 0 Executed successfully 1 Covered by another task 2 Restored from sstate 3 Artifacts already existing 4 Fail during execution 5 Not available || Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5216 5216]
|-
| [Object Root].list[].fields.sstate_checksum || string || - || YES || - || Sstate checksum of the task || -
|-
| [Object Root].list[].fields.path_to_sstate_obj || string || - || NO || - || The path to the sstate file, if applicable || Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5252 5252]
|-
| [Object Root].list[].fields.recipe || number || - || YES || - || ID of the recipe generating the task || -
|-
| [Object Root].list[].fields.task_name || string || - || YES || - || Name of the task || e.g. "do_fetch"
|-
| [Object Root].list[].fields.source_url || string || - || NO || - || Path to script source file (yes, it's misnamed) || Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5255 5255]
|-
| [Object Root].list[].fields.logfile || string || - || YES || - || Path to log file || -
|-
| [Object Root].list[].fields.work_directory || string || - || YES || - || Cwd during task execution || Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5253 5253]
|-
| [Object Root].list[].fields.script_type || number || 3 || NO || 0 || See if we executed a shell or python script, or no script was executed || 0 noexec 1 python 2 shell Known issues: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5327 5327]
|-
| [Object Root].list[].fields.line_number || number || - || NO || - || Starting number of the line in the source file || Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5254 5254]
|-
| [Object Root].list[].fields.disk_io || number || - || NO || - || Time spent by the task in I/O operations (in ms) || Known issues: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5073 5073], [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5485 5485]
|-
| [Object Root].list[].fields.cpu_usage || number || - || NO || - || Percent of the CPU used during the task || Known issues: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5073 5073], [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5485 5485]
|-
| [Object Root].list[].fields.sstate_result || number || 4 || NO || 0 || Outcome of the sstate task || 0 N/A 1 Missing 2 Failed 3 Restored successfully Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5220 5220]
|-
| [Object Root].list[].message.message || string || - || NO || - || Any message from the system regarding the task || -
|-
| [Object Root].list[].message.elapsed_time || string || - || NO || - || Time taken by the task to complete (in seconds) || Known issues: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5300 5300]
|}

== Task dependencies ==

Returns the tasks dependency data.

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/task_dependencies || JSON || Returns page size-limited and search criteria-filtered tasks from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of tasks to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.task || number || - || YES || - || The PK of the target task || -
|-
| [Object Root].list[].fields.depends_on || number || - || YES || - || The PK of the task that the target task depends on || -
|-
|}

== Recipe ==

Returns information about recipes in a certain build.

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/recipes || JSON || Returns page size-limited and search criteria-filtered recipes from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of recipes to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.name || string || - || YES || - || The recipe name || -
|-
| [Object Root].list[].fields.version || string || - || YES || - || The recipe version || Known issues: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5459 5459]
|-
| [Object Root].list[].fields.layer_version || number || - || YES || - || The version of the layer in which the recipe existed at the task building time || -
|-
| [Object Root].list[].fields.summary || string || - || YES || - || The content of the SUMMARY variable || -
|-
| [Object Root].list[].fields.description || string || - || YES || - || The content of the DESCRIPTION variable || -
|-
| [Object Root].list[].fields.section || string || - || YES || - || The section to which the recipe belongs || -
|-
| [Object Root].list[].fields.license || string || - || YES || - || The license of the recipe || -
|-
| [Object Root].list[].fields.licensing_info || string || - || YES || - || The directory containing the recipe license files || Known issues: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5194 5194]
|-
| [Object Root].list[].fields.homepage || string || - || YES || - || The website for the software provided by the recipe || -
|-
| [Object Root].list[].fields.bugtracker || string || - || YES || - || The bug tracking website for the software provided by the recipe || -
|-
| [Object Root].list[].fields.author || string || - || YES || - || The author of the recipe || Know issues: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5449 5449]
|-
| [Object Root].list[].fields.file_path || string || - || YES || - || Path to the recipe .bb file || -
|-
|}

== Recipe dependencies ==

Returns the recipe dependency data.

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/recipe_dependencies || JSON || Returns page size-limited and search criteria-filtered recipes from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of recipes to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.recipe || number || - || YES || - || The PK of the target recipe || -
|-
| [Object Root].list[].fields.depends_on || number || - || YES || - || The PK of the recipe that the target recipe depends on || -
|-
| [Object Root].list[].fields.dep_type || number || 2 || YES || - || Dependency type || 0 [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-DEPENDS DEPENDS] 1 [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-RDEPENDS RDEPENDS]
|-
|}

== Build packages ==

Returns data about packages built. This information is collected at build time, while [[#Target installed packages | target installed packages]] information is collected from [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#maintaining-build-output-quality build history].

Known issues: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5269 5269]

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/packages || JSON || Returns page size-limited and search criteria-filtered packages from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of packages to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.build || number || - || YES || - || The PK of the build that generated this package || -
|-
| [Object Root].list[].fields.recipe || number || - || YES || - || The PK of the recipe that generated this package || -
|-
| [Object Root].list[].fields.name || string || - || YES || - || The package name || -
|-
| [Object Root].list[].fields.version || string || - || YES || - || The package version || -
|-
| [Object Root].list[].fields.revision || string || - || YES || - || The package revision || -
|-
| [Object Root].list[].fields.size || number || - || YES || - || The size of the package, in kilobytes || Known issues: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5503 5503]
|-
| [Object Root].list[].fields.section || string || - || YES || - || The section to which this package belongs || -
|-
| [Object Root].list[].fields.license || string || - || YES || - || The license of the package || -
|-
| [Object Root].list[].fields.summary || string || - || YES || - || The content of the recipe SUMMARY || -
|-
| [Object Root].list[].fields.description || string || - || YES || - || The content of the recipe DESCRIPTION || -
|-
|}

== Package dependencies ==

Returns the package dependency data.

Known issues: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5269 5269]

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/package_dependencies || JSON || Returns page size-limited and search criteria-filtered packages from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of packages to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.package || number || - || YES || - || The PK of the target package || -
|-
| [Object Root].list[].fields.depends_on || number || - || YES || - || The PK of the package that the target package depends on || Known issues: [http://bugzilla.yoctoproject.org/show_bug.cgi?id=5225 5225]
|-
| [Object Root].list[].fields.dep_type || number || 6 || YES || - || Dependency type || 0 [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-RDEPENDS RDEPENDS] 1 [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-RPROVIDES RPROVIDES] 2 [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-RRECOMMENDS RRECOMMENDS] 5 [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-RSUGGESTS RSUGGESTS] 4 [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-RREPLACES RREPLACES] 5 [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-RCONFLICTS RCONFLICTS] 
|-
|}

== Package files ==

Returns data about package-generated files.

Known issues: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=5269 5269]

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/package_files || JSON || Returns page size-limited and search criteria-filtered files for a package from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of files to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.bpackage || number || - || YES || - || The PK of the package generating the file || -
|-
| [Object Root].list[].fields.path || String || - || YES || - || Full file path || -
|-
| [Object Root].list[].fields.size || number || - || YES || - || File size, in bytes || -
|-
|}

== Variables ==

Returns data about key-value pairs applied to a certain build.

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/variables || JSON || Returns page size-limited and search criteria-filtered variables from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of variables to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.build || number || - || YES || - || The PK of the build that generated this variable || -
|-
| [Object Root].list[].fields.variable_name || string || - || YES || - || The key in a key-value pair || -
|-
| [Object Root].list[].fields.variable_value || string || - || YES || - || The value in a key-value pair || -
|-
| [Object Root].list[].fields.changed || boolean || - || YES || - || Not defined || Not currently available
|-
| [Object Root].list[].fields.human_readable_name || string || - || YES || - || Not defined || Not currently available
|-
| [Object Root].list[].fields.description || string || - || YES || - || Variable description in documentation.conf || -
|-
|}

== Variable History ==

Returns information on how the variable reached its current value.

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/variablehistory || JSON || Returns page size-limited and search criteria-filtered variables from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of entries to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.variable || number || - || YES || - || The PK of the variable this operation changed || -
|-
| [Object Root].list[].fields.file_name || string || - || YES || - || The file name where the variable value change was specified || -
|-
| [Object Root].list[].fields.line_number || integer || - || YES || - || The line number in the file where the variable value change was specified || -
|-
| [Object Root].list[].fields.operation || string || - || YES || - || The change that took place, uninterpreted value || -
|-
|}

== Log messages ==

Returns data about errors / warnings thrown by the build.

=== Endpoints ===

{| class="wikitable"
|-
! Method !! Endpoint !! Body !! Description !! Comments
|-
| GET || /api/1.0/logmessages || JSON || Returns page size-limited and search criteria-filtered messages from the database || -
|}

=== Response ===

{| class="wikitable"
|-
! Name !! Type !! Dimension !! Required !! Default value(s) !! Description !! Comments
|-
| [Object Root] || object || - || YES || - || - || -
|-
| [Object Root].count || number || - || YES || - || Total amount of messages to be displayed (according to filter parameter) || -
|-
| [Object Root].list || object array || - || YES || - || - || To describe a generic object element of the array, the reference to its root is "list[]"
|-
| [Object Root].list[].pk || number || - || YES || - || - || -
|-
| [Object Root].list[].model || string || - || YES || - || - || -
|-
| [Object Root].list[].fields || object || - || YES || - || - || -
|-
| [Object Root].list[].fields.build || number || - || YES || - || The PK of the build that generated this message || -
|-
| [Object Root].list[].fields.level || number || 3 || YES || - || Message type || 0 - INFO 1 - WARNING 2 - ERROR 
|-
| [Object Root].list[].fields.message || string || - || YES || - || The message content || -
|-
| [Object Root].list[].fields.pathname || string || - || YES || - || Path to the file generating the message || -
|-
| [Object Root].list[].fields.lineno || number || - || YES || - || Line number generating the message || -
|-
|}

REST API Contracts

2013-11-18T11:59:02Z

Ddalex:

Toaster offers multiple REST APIs for interacting with the system.
The REST APIs are classified in modules, each module providing a different set of functionality.

== Analysis Module ==

Currently released on dora-toaster release, and ready to be used
* [[Analysis REST API Contracts]]

== Build Module ==

This is in planning phases