Contribute to Toaster

This page summarises the Toaster development process. We hope this will help you start contributing to the project. We were previously using a process that involved the toaster-next branch on poky-contrib. This workflow was especially useful in an environment where Toaster was undergoing a lot of change and needed the ability to layer patch sets upon each other before they had been taken up by bitbake. Since that is not currently the case, toaster-next just adds complication; so it has been removed from this workflow. See Contribute to Toaster (toaster-next version) for the old toaster-next based workflow.

What can I do?

The Yocto Project Bugzilla instance lists all the things that need to be done:

• If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the Toaster mailing list
• If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the Toaster mailing list to find out why the design work is not done yet

Set up the local repository

For development of Toaster we recommend setting up a local install of Toaster. Installation instructions are available in the main Toaster documentation

Submitting patches

Publishing your patches to Toaster is a two step process.

1. Sending patches to the Toaster mailing list for review
2. Submitting the patches that you reviewed to the upstream repository

By submitting your patches first to the Toaster mailing list, you can be sure the patches are reviewed by the people in the community who are familiar with the Toaster source code, and who have experience developing web applications.

That also means that, by the time the patches are submitted to the upstream mailing lists, they are in pretty good shape. That helps the project maintainers, and hopefully also helps you.

Toaster code lives in Bitbake repository at [1]. All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

Workflow

To contribute to toaster you will also need authorization to write to the upstream yocto project repository. Contact a member of the toaster team for details.

1) Download master branch of the yocto project

  git clone git://git.yoctoproject.org/poky && cd poky 

2) Add poky-contrib to the local repository you set up above

  git remote add poky-contrib ssh://git@git.yoctoproject.org/poky-contrib 

3) Fetch the poky-contrib branches

  git fetch --all 

4) Start your feature branch off of master

  git checkout -b the/target/branch origin/master 

5) Do Work

6) Test the changes. Run the Django unit tests. People put effort into these so we should make sure we use them. This assumes you have phantomjs installed. This can usually be done from the distribution apt-get install phantomjs, for example. If you want to test against Chrome or Firefox, see the README in bitbake/lib/toaster/tests/browser.

  pip3 install selenium 
 TOASTER_TESTS_BROWSER=phantomjs bitbake/lib/toaster/manage.py test orm toastergui tests.browser 

Note: If you would like to run the tests in a container so they are repeatable and do not continually break due to host upgrades see Running Toaster Tests with Containers

7) Rebase on master. It has probably changed while you were working (unless you are really really fast!)

  git rebase origin/master 

8) Send to the toaster-mailing list using one of the methods outlined below.

Sending patches to Toaster Project

NOTE: The format of the commit message should be like this

    toaster: <module> <short one line summary>

    long(er) description

    [YOCTO #0000]

    Signed-off-by: First Last <name@domain.com>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the toaster mailing list ( toaster@yoctoproject.org ) by "git send-email".

e.g.

   $ git send-email HEAD^ 

Alternatively, you can use the utilities in the script directory to prepare your patch

1) Use the create-pull-request script (from poky) to create a pull request while on your feature branch

  ./scripts/create-pull-request -s "toaster: Fixes and clean ups" -u poky-contrib -r poky-contrib/toaster-next

2) Review their content, especially the summary mail:

  edit ./pull-<pid>/0000-cover-letter.patch

3)When you are satisfied, you can send them with:

  ./scripts/send-pull-request -a -p ./pull-<pid> -t toaster@yoctoproject.org

A comprehensive document about commit messages is available on the openembedded wiki

More help learning git is available on github and the official documentation

Sending branches to Toaster Project

If you wish to submit whole branches please use the poky-contrib repository see Poky Contributions#Poky_Contrib_Branch for setup guide.

Once you have pushed a branch please then send an email to the toaster mailing list with the subject in the following format:

[review-request] my_branch_name

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

Submitting patch sets for integration into Bitbake

Toaster patches are normally submitted upstream to the BitBake repository by the reviewer (not the author). This tells the upstream maintainers that the patches have been reviewed by the people who are familiar with the Toaster source code, and makes their busy lives a bit easier.

Since development happens on the poky-contrib repository, but the patches need to be merged to the Bitbake repository, the following process should be executed.

1) Bring toaster-next up to date with master

  git fetch poky-contrib toaster-next&& git fetch origin master
  git checkout -b toaster-next poky-contrib/toaster-next 
  git rebase origin/master 

2) Checkout the target branch

  git checkout the/target/branch

3) Create a new branch for submission

   git checkout -b yourname/submit/the/target/branch 

4) Make sure the branch is rebased on current poky-contrib toaster-next.

  git rebase poky-contrib/toaster-next

5) Test the changes. Run the Django unit tests. People put effort into these so we should make sure we use them. his assumes you have phantomjs installed. This can usually be done from the distribution apt-get install phantomjs, for example. If you want to test against Chrome or Firefox, see the README in bitbake/lib/toaster/tests/browser.

  pip3 install selenium 
 TOASTER_TESTS_BROWSER=phantomjs bitbake/lib/toaster/manage.py test orm toastergui tests.browser 

6) Add signed off by to the commit messages

  git filter-branch -f --msg-filter 'cat && echo "Signed-off-by: $(git config --get user.name) <$(git config --get user.email)>"' toaster-next..HEAD

7) Push the modified commit messages and rebased version to poky-contrib

  git push -u poky-contrib yourname/submit/the/target/branch 

8) Use the create-pull-request script (from poky) to create a pull request

  ./scripts/create-pull-request -d bitbake -s "toaster: Fixes and clean ups" -u poky-contrib -r poky-contrib/toaster-next

Note: If the patch creates any NEW files, the integration scripts that pull it into bitbake will fail. So, if new files are created as part of this patch set, you need to explicitly point that out in the body of the email for the patch set or do it as a PR rather than as a patch set.

9) Review their content, especially the summary mail:

  edit ./pull-<pid>/0000-cover-letter.patch

10) Push the branch you just signed off on and sent upstream to toaster-next

  git push -f -u  poky-contrib yourname/submit/the/target/branch:toaster-next 

When you are satisfied, you can send them with:

  ./scripts/send-pull-request -a -p ./pull-<pid> -t bitbake-devel@lists.openembedded.org

Submitting patches for prior releases

The procedure is the same, but using the prior release as the base branch instead of the "master" branch in bitbake.

Also, make sure that you add the name of the prior release for which the patchset is intended in the prefix of the patchset, as parameter to the "create-pull-request" command, e.g. -p 1.26 for the 1.26 branch.

Gotchas

Too Big

Sometimes the mailer will refuse to send patches, especially on binary or long-line files. The proper way to go around that is to reply to the patchset you've submitted to the mailing list, asking for a git pull directly from the poky-contrib branch.

Different Lists

The toasterconf.json files live in the meta and meta-yocto layer. Patches to the meta-yocto/conf/toasterconf.json go to poky@yoctoproject.org. Patches to meta/conf/toasterjson.conf go to openembedded-core@lists.openembedded.org. Patches for toaster.bbclass also go to the openembedded--core list. If you have a branch that spans both (let's say you made coupled changes to meta/classes/toaster.bbclass and bitbake/lib/bb/ui/buildinfohelper.py. then your branch needs to be split into 2 different patch set submissions. Let's assume there are 5 commits, 2 to meta and 3 to bitbake. Currently, you can do this suboptimal workaround:

 ./scripts/create-pull-request -d bitbake -s "toaster: Fixes and clean ups" -u poky-contrib -r poky-contrib/toaster-next
 ./scripts/create-pull-request -d meta -s "toaster: Fixes and clean ups" -u poky-contrib -r poky-contrib/toaster-next

This will create 2 pull-XXXX directories, one for openembedded and one for bitbake-devel. Unfortunately, the meta pull-XXXX dir will have patches numbered 1-5 with 3 0 byte files and the bitbake pull-XXXX dir will have patches numbered 1-5 with 2 0 byte files. You delete the 0 byte files and use a text editor to do the renumbering. At some point it would be nice to have the create-pull-request do this for us but this level of file modification would probably mean moving it out of a shell script and into python. Once the files have been renumbered (in the subject line and the cover letter) the send-pull-request can be used as referenced above.

A different workaround that may work better for you is to create two branches which lets you use the create-pull-request and send-pull-request scripts normally on each branch. Just make sure to target them correctly

 git checkout -b yourname/submit/the/target/branch-ForBitbake yourname/submit/the/target/branch
 git rebase -i
 # delete all the commits that are OE specific in the set you wish to submit
 git push -u poky-contrib yourname/submit/the/target/branchForBitbake:yourname/submit/the/target/branchForBitbake
 ./scripts/create-pull-request -d bitbake -s "toaster: Fixes and clean ups" -u poky-contrib -r poky-contrib/toaster-next
 git checkout -b yourname/submit/the/target/branch-ForOE yourname/submit/the/target/branch
 git rebase -i
 # delete all the commits that are bitbake specific in the set you wish to submit
 git push -u poky-contrib yourname/submit/the/target/branchForOE:yourname/submit/the/target/branchForOE
 ./scripts/create-pull-request -d meta -s "toaster: Fixes and clean ups" -u poky-contrib -r poky-contrib/toaster-next

One Patch of a Long Patch Set Needs Resubmission

Suppose you upstream a 10 commit patch set to the bitbake-devel list and someone finds an issue with patch #3. Regenerating the whole series is silly so how do you address this? First, follow the bitbake submission steps until you end up on the yourname/submit/the/target/branch branch. Then you can (note the reset --hard will wipe any local changes in your working dir so commit or stash first):

 git checkout -b yourname/submit/the/target/branch-newHEAD
 git reset --hard <commit of resubmission issue> 
 git commit --amend --signoff 
 git send-email --in-reply-to="longNumber.git.me@mycomp.com" --subject-prefix="bitbake-devel] [PATCHVX 03/10"  --to=bitbake-devel@lists.openembedded.org --no-chain-reply-to --suppress-cc=all -M -1 --relative=bitbake

The longNumber.git.me@mycomp.com comes from the email message id for the particular patch #3 so that the email threading works. In gmail, you can click on the dropdown button on the right side of the screen and choose "Show Original". This will have a field in the header like Message-Id: <4551b56f132497c055f39567946a5d3be347d770.1468363530.git.myemailusername@mycompany.com> The entire string except the '<>' are used. for example:

 --in-reply-to="4551b56f132497c055f39567946a5d3be347d770.1468363530.git.myemailusername@mycompany.com"

Don't forget to switch back to the yourname/submit/the/target/branch and push that to poky-contrib/toaster-next! This is a little harder since you will have previously pushed a different version of the patch to poky-contrib/toaster-next. You can do a rebase/merge by hand but that can be prone to error. Here's the working sequence assuming it's been a few days between the first poky-contrib/toaster-next push and the one you are working on now (meaning both poky-contrib/toaster-next and origin/master have changes).
The first issue you are likely to face is that the git filter-branch command in the standard instructions may sign off on too many commits. If you know you just want to sign off on the last 7 commits on the yourname/submit/the/target/branch you can:

 git filter-branch -f --msg-filter 'cat && echo "Signed-off-by: $(git config --get user.name) <$(git config --get user.email)>"' HEAD~7..HEAD

Once your signoff is copacetic, updating poky-contrib/toaster-next can be done as follows:

 git fetch poky-contrib toaster-next&& git fetch origin master
 git checkout -b toaster-next poky-contrib/toaster-next 
 git rebase origin/master 
 git checkout yourname/submit/the/target/branch 
 git rebase --strategy recursive -X theirs -X patience toaster-next
 git push -fu poky-contrib poky-contrib yourname/submit/the/target/branch:toaster-next 

Submitting patches for documentation

Documentation patches should be sent to Yocto mailing list with [yocto-docs] in the subject, CC Scott Rifenbark (and make sure you send it to his gmail, not his defunct Intel address). For his email address, look at this post.

Code syle guide

Templates

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

Yes please:

{{var}}

<div>
  {# Maintaining indentation #}
  {% if %}
   <p>this</p>
  {% else %}
   <p>that</p>
  {% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
 {{layer}}
{% endfor %}

No thank you:

{{var}}
<div>
{# Maintaining indentation #}
{%if%}<p>this</p>{%else%}<p>that</p>{%endif%}
</div>


{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}

Note:

• Maintain indentation as you would with other languages
• White space after '%'
• Comment blocks for longer comments

Javascript

Yes please:

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
  cheddar : 1,
  stilton : 2,
  emmental : 3, 
};

function doThingsHere(){
  return 1;
}

/* If one equals two do some other things and make sure that
 * if the the click handler is setup correctly.
 */
if (one === two) {
  var cheese = "cheddar";
  oneVar = doThingsHere();

  $(this).click(function (event){
    alert("Hello");
  });
}

$("#little-mouse").focusout(function(){
  alert("bye")
});

if (oneVar)
  noThingHere();
else
  doThingHere();

No thank you:

// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2,  emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

    $(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
  alert("bye")
});

if (oneVar)
{
  noThingHere();
} else {  doThingHere(); }

Note:

• Variables should be marked with "var"
• Semicolons should be used
• Keep as close to 80 cols as possible
• Use 2 space per indentation
• Open curly braces after parenthesis for functions and close on a new line
• Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:

$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js

HTML

Yes please:

<div id="something-area">
  <p class="important">This is some text</p>
</div>

<div>
  <p id="important-text>This is some text</p>
</div>

No thank you:

<div id="somethingarea">
    <p class="Important">This is some text</p></div>
<div id="somethingarea">
<p id="ImportantText">This is
some text
</p>

</div>

Note:

• 2 space indentation
• Lower case, ids hyphenated when multiple words
• No duplicate ids

• Run your HTML through a HTML validator available for local install. The w3c validator it's self doesn't currently validate html5, it uses as a back end Nu Html Checker which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:

 $ mkdir html5-validator && cd html5-validator
 $ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
 $ git clone https://github.com/validator/validator.git
 $ python build/build.py all
 $ python build/build.py all

HTML can be indented quickly using tidy, for example:

 tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes  ./index.html 

Python

Lenient pep8 Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and error code list

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.

$ pep8 ./toastergui/urls.py

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

$ pylint --load-plugins pylint_django toastergui/tests.py

Working with design

Yes, the Yocto Project is one of those lucky projects with designers around to help in UI matters. We have a document explaining how to work with the design contributors: File:Working with design.pdf