Yocto Project - User contributions [en]

File:Toaster.sqlite

2025-11-30T04:43:26Z

David Reyna: David Reyna uploaded a new version of File:Toaster.sqlite

Sample Toaster data for development purposes.

File:Toaster.sqlite

2025-11-29T23:12:31Z

David Reyna: David Reyna uploaded a new version of File:Toaster.sqlite

Sample Toaster data for development purposes.

Contribute to Toaster

2025-11-15T06:17:44Z

David Reyna:

[[Category:Toaster]]
This page summarises the Toaster development process. We hope this will help you start contributing to the project. We were previously using a process that involved the toaster-next branch on poky-contrib. This workflow was especially useful in an environment where Toaster was undergoing a lot of change and needed the ability to layer patch sets upon each other before they had been taken up by bitbake. Since that is not currently the case, toaster-next just adds complication; so it has been removed from this workflow. See [[Contribute to Toaster (toaster-next version)]] for the old toaster-next based workflow.

== What can I do? ==

The [https://bugzilla.yoctoproject.org/buglist.cgi?product=Toaster Yocto Project Bugzilla instance] lists all the things that need to be done:

* If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list]
* If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list] to find out why the design work is not done yet

== Set up the local repository ==

For development of Toaster we recommend setting up a local install of Toaster. Installation instructions are available in the main [https://docs.yoctoproject.org/toaster-manual/index.html Toaster documentation]

== Submitting patches (bitbake-setup) ==

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].
All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

=== Workflow ===
 
1) Download master branch of the yocto project

<pre>
$ git clone https://git.openembedded.org/bitbake/
$ ./bitbake/bin/bitbake-setup settings set global top-dir-prefix $PWD
$ ./bitbake/bin/bitbake-setup init --non-interactive poky-master poky distro/poky machine/qemux86-64
$ cd bitbake-builds/poky-master-poky-distro_poky-machine_qemux86-64
</pre>

 
2) Start your feature branch off of master, name style of branch is convention, but suggested.

<pre>
$ cd layers/bitbake
$ git checkout -b username/toaster/FeatureOrBug origin/master
</pre>

 
3) Do Work

 
4) Rebase on master. It has probably changed while you were working (unless you are really really fast!)

<pre>
$ git rebase origin/master
</pre>

 
5) Create your commit

NOTE: The format of the commit message should be like this

<pre>
toaster: <module> <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

 
6) Prepare your patch

<code>
$ git format-patch -1
</code>

 

=== Submitting patch for integration into Bitbake ===

Email list: bitbake-devel@lists.openembedded.org
 
NOTE: For the email's subject line, prepend "[bitbake-devel]" to the patch's subject line

For example:

<pre>
TO : bitbake-devel@lists.openembedded.org
SUBJECT: [bitbake-devel][PATCH] toaster: support bitbake-setup
CONTENT:
From c063178ab0971fe297a9e9bca6e6b04eafc5149c Mon Sep 17 00:00:00 2001
From: David Reyna <David.Reyna@windriver.com>
Date: Thu, 30 Oct 2025 14:03:35 -0700
Subject: [PATCH] toaster: support bitbake-setup

This adds support for the new bitbake-setup
<<<CUT>>>
[YOCTO #16012]

Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
bin/toaster | 21 ++--
.../bldcontrol/localhostbecontroller.py | 99 +++++++++++
<<<CUT>>>
</pre>

== Submitting patches (Poky Classic, OBSOLETE) ==

Publishing your patches to Toaster is a two step process.
# Sending patches to the [https://lists.yoctoproject.org/listinfo/toaster/| Toaster mailing list] for review
# Submitting the patches that you reviewed to the upstream repository

By submitting your patches first to the Toaster mailing list, you can be sure the patches are reviewed by the people in the community who are familiar with the Toaster source code, and who have experience developing web applications.

That also means that, by the time the patches are submitted to the upstream mailing lists, they are in pretty good shape. That helps the project maintainers, and hopefully also helps you.

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].
All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

=== Workflow ===
To contribute to toaster you will also need authorization to write to the upstream yocto project repository. Contact a member of the toaster team for details.

1) Download master branch of the yocto project
<code> git clone git://git.yoctoproject.org/poky && cd poky </code>

2) Add poky-contrib to the local repository you set up above
<code> git remote add poky-contrib ssh://git@push.yoctoproject.org/poky-contrib </code>

3) Fetch the poky-contrib branches
<code> git fetch --all </code>

4) Start your feature branch off of master, name style of branch is convention, but suggested.
<code> git checkout -b username/toaster/FeatureOrBug origin/master </code>

5) Do Work

6) Test the changes. Run the Django unit tests. People put effort into these so we should make sure we use them. This assumes you have phantomjs installed. This can usually be done from the distribution <code> apt-get install phantomjs</code>, for example. If you want to test against Chrome or Firefox, see the README in bitbake/lib/toaster/tests/browser.
<code> pip3 install selenium </code>
<code>TOASTER_TESTS_BROWSER=phantomjs bitbake/lib/toaster/manage.py test orm toastergui tests.browser </code>
 
Note: If you would like to run the tests in a container so they are repeatable and do not continually break due to host upgrades see [[Running Toaster Tests with Containers]]

7) Rebase on master. It has probably changed while you were working (unless you are really really fast!)
<code> git rebase origin/master </code>

8) Push your feature branch to poky-contrib
<code> git push -u poky-contrib username/toaster/FeatureOrBug:username/toaster/FeatureOrBug</code>

9) Send to the toaster-mailing list using one of the methods outlined below.

10) NOTE: when the patch has been accepted upstream, you can clean up your poy-contrib branch with:

<code> git push -u poky-contrib :username/toaster/FeatureOrBug </code>

=== Sending patches to Toaster Project ===

NOTE: The format of the commit message should be like this

<pre>
toaster: <module> <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] ( toaster@yoctoproject.org ) by "git send-email".

e.g.
<code>
$ git send-email HEAD^
</code>

Alternatively, you can use the utilities in the script directory to prepare your patch

1) Use the create-pull-request script (from poky) to create a pull request while on your feature branch
<code>./scripts/create-pull-request -s "toaster: Fixes and clean ups" -u poky-contrib -r origin/master</code>

2) Review their content, especially the summary mail:
<code>edit ./pull-<pid>/0000-cover-letter.patch</code>

3)When you are satisfied, you can send them with:
<code>./scripts/send-pull-request -a -p ./pull-<pid></code> -t toaster@yoctoproject.org

A comprehensive document about commit messages is available on the [http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines openembedded wiki]

More help learning git is available on [https://try.github.io github] and [http://git-scm.com/documentation/ the official documentation]

=== Sending branches to Toaster Project ===

If you wish to submit whole branches please use the poky-contrib repository see [[Poky Contributions#Poky_Contrib_Branch]] for setup guide.

Once you have pushed a branch please then send an email to the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] with the subject in the following format:

<code>
[review-request] my_branch_name
</code>

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

=== Submitting patch sets for integration into Bitbake ===

Toaster patches are normally submitted upstream to the BitBake repository by the reviewer (not the author). This tells the upstream maintainers that the patches have been reviewed by the people who are familiar with the Toaster source code, and makes their busy lives a bit easier.

Since development happens on the poky-contrib repository, but the patches need to be merged to the Bitbake repository, the following process should be executed.

1) Bring master up to date
<code> git fetch origin master</code>
<code> git checkout master </code>
<code> git pull [master] </code>

2) Checkout the target branch you wish to upstream

(a) If you are working with a single branch do this:

<code>git checkout username/toaster/FeatureOrBug</code>

(b) If you are including several branches into this one submission, gather the commit IDs for each branch and do this:

<code>git cherry-pick commit-id-a </code>
<code>git cherry-pick commit-id-b </code>
<code>... </code>

3) Create a new branch for submission
<code> git checkout -b yourname/submit/username/toaster/FeatureOrBug </code>
This will look something like
<code> git checkout -b bavery/submit/bob/toaster/FixBug1234 </code>
where bavery is signing off on and upstreaming bob's fix to the Yocto Bugzilla bug 1234.
 
----
----
* Alternatively, you can use the patchworks web site:
2) Create a new branch for submission
<code> git checkout -b yourname/submit/username/toaster/FeatureOrBug </code>
This will look something like
<code> git checkout -b bavery/submit/bob/toaster/FixBug1234 </code>
where bavery is signing off on and upstreaming bob's fix to the Yocto Bugzilla bug 1234.
3) Download and apply the mbox patch from the website: https://patchwork.openembedded.org/project/toaster/patches/

<code> git am -s the-downloaded-patch.mbox </code>
----
----
 
4) Make sure the branch is rebased on current master.
<code>git rebase origin/master</code>

5) Test the changes. Run the Django unit tests. People put effort into these so we should make sure we use them. his assumes you have phantomjs installed. This can usually be done from the distribution <code> apt-get install phantomjs</code>, for example. If you want to test against Chrome or Firefox, see the README in bitbake/lib/toaster/tests/browser.
<code> pip3 install selenium </code>
<code>TOASTER_TESTS_BROWSER=phantomjs bitbake/lib/toaster/manage.py test orm toastergui tests.browser </code>

6) Add signed off by to the commit messages (Note: If you applied the mbox patch with <code> git am -s foo.mbox</code> You do not need to sign off again.)
<code>git filter-branch -f --msg-filter 'cat && echo "Signed-off-by: $(git config --get user.name) <$(git config --get user.email)>"' master..HEAD</code>

7) Push the modified commit messages and rebased version to poky-contrib
<code>git push -u poky-contrib yourname/submit/username/toaster/FeatureOrBug </code>

8) Use the create-pull-request script (from poky) to create a pull request for the appropriate tree/mailing list

(a) For the 'bitbake' tree (e.g. bitbake/.../toasterui.py, bitbake/.../toastergui/*):

<code>./scripts/create-pull-request -d bitbake -s "toaster: Fixes and clean ups" -u poky-contrib -r origin/master</code>

(b) for the 'meta' tree (e.g. meta/classes/toaster.class):

<code>./scripts/create-pull-request -d meta -s "toaster: Fixes and clean ups" -u poky-contrib -r origin/master</code>
 
Note: If the patch creates any NEW files, the integration scripts that pull it into bitbake will fail. So, if new files are created as part of this patch set, you need to explicitly point that out in the body of the email for the patch set or do it as a PR rather than as a patch set.


9) Review their content, especially the summary mail:
<code>edit ./pull-<pid>/0000-cover-letter.patch</code>

10) When you are satisfied, you can send them with:
--- for the 'bitbake' tree:
<code>./scripts/send-pull-request -a -p ./pull-<pid></code> -t bitbake-devel@lists.openembedded.org
--- for the 'meta' tree:
<code>./scripts/send-pull-request -a -p ./pull-<pid></code> -t openembedded-core@lists.openembedded.org

==== Submitting patches for prior releases ====

The procedure is the same, but using the prior release as the base branch instead of the "master" branch in bitbake.

Also, make sure that you add the name of the prior release for which the patchset is intended in the prefix of the patchset, as parameter to the "create-pull-request" command, e.g. '''-p 1.26''' for the 1.26 branch.

==== Gotchas ====
===== Too Big =====
Sometimes the mailer will refuse to send patches, especially on binary or long-line files. The proper way to go around that is to reply to the patchset you've submitted to the mailing list, asking for a git pull directly from the poky-contrib branch.
===== One Patch of a Long Patch Set Needs Resubmission =====
Suppose you upstream a 10 commit patch set to the bitbake-devel list and someone finds an issue with patch #3. Regenerating the whole series is silly so how do you address this? First, follow the bitbake submission steps until you end up on the yourname/submit/the/target/branch branch. Then you can (note the reset --hard will wipe any local changes in your working dir so commit or stash first): 
<code>
git checkout -b yourname/submit/the/target/branch-newHEAD
git reset --hard <commit of resubmission issue>
git commit --amend --signoff
git send-email --in-reply-to="longNumber.git.me@mycomp.com" --subject-prefix="bitbake-devel] [PATCHVX 03/10" --to=bitbake-devel@lists.openembedded.org --no-chain-reply-to --suppress-cc=all -M -1 --relative=bitbake
</code> 
The longNumber.git.me@mycomp.com comes from the email message id for the particular patch #3 so that the email threading works. In gmail, you can click on the dropdown button on the right side of the screen and choose "Show Original". This will have a field in the header like Message-Id: <4551b56f132497c055f39567946a5d3be347d770.1468363530.git.myemailusername@mycompany.com> The entire string except the '<>' are used. for example: 
<code>
--in-reply-to="4551b56f132497c055f39567946a5d3be347d770.1468363530.git.myemailusername@mycompany.com"
</code> 

The first issue you are likely to face is that the git filter-branch command in the standard instructions may sign off on too many commits. If you know you just want to sign off on the last 7 commits on the yourname/submit/the/target/branch you can:
<code>
git filter-branch -f --msg-filter 'cat && echo "Signed-off-by: $(git config --get user.name) <$(git config --get user.email)>"' HEAD~7..HEAD
</code>

=== Submitting patches for documentation ===

Documentation patches should be sent to [https://lists.yoctoproject.org/listinfo/yocto Yocto mailing list] with [yocto-docs] in the subject, CC Scott Rifenbark (and make sure you send it to his gmail, not his defunct Intel address). For his email address, look at [http://lists.openembedded.org/pipermail/bitbake-devel/2015-October/006632.html this post].

== Code syle guide ==

=== Templates ===

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

'''Yes please:'''

<pre>
{{var}}

<div>
{# Maintaining indentation #}
{% if %}
this
{% else %}
that
{% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
{{layer}}
{% endfor %}

</pre>

'''No thank you:'''
<pre>
{{var}}
<div>
{# Maintaining indentation #}
{%if%}this{%else%}that{%endif%}
</div>

{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}
</pre>

Note:
* Maintain indentation as you would with other languages
* White space after '%'
* Comment blocks for longer comments

=== Javascript ===

'''Yes please:'''
<pre>

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
cheddar : 1,
stilton : 2,
emmental : 3,
};

function doThingsHere(){
return 1;
}

/* If one equals two do some other things and make sure that
* if the the click handler is setup correctly.
*/
if (one === two) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function (event){
alert("Hello");
});
}

$("#little-mouse").focusout(function(){
alert("bye")
});

if (oneVar)
noThingHere();
else
doThingHere();
</pre>

'''No thank you:'''
<pre>
// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2, emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
alert("bye")
});

if (oneVar)
{
noThingHere();
} else { doThingHere(); }
</pre>

Note:
* Variables should be marked with "var"
* Semicolons should be used
* Keep as close to 80 cols as possible
* Use 2 space per indentation
* Open curly braces after parenthesis for functions and close on a new line
* Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:
<code>
$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js
</code>

=== HTML ===

'''Yes please:'''
<pre>
<div id="something-area">
This is some text
</div>

<div>
This is some text
</div>

</pre>

'''No thank you:'''
<pre>
<div id="somethingarea">
This is some text</div>
<div id="somethingarea">
This is
some text


</div>
</pre>

Note:
* 2 space indentation
* Lower case, ids hyphenated when multiple words
* No duplicate ids

* Run your HTML through a [http://validator.w3.org/#validate_by_input HTML validator] available for [http://validator.w3.org/source/ local install]. The w3c validator it's self doesn't currently validate html5, it uses as a back end [https://validator.github.io/validator/ Nu Html Checker] which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:
<pre>
$ mkdir html5-validator && cd html5-validator
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$ git clone https://github.com/validator/validator.git
$ python build/build.py all
$ python build/build.py all
</pre>

HTML can be indented quickly using tidy, for example:
<pre>
tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes ./index.html
</pre>

=== Python ===

Lenient [https://www.python.org/dev/peps/pep-0008 pep8]
Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and [http://pep8.readthedocs.org/en/latest/intro.html#error-codes error code list]

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.
<code>
$ pep8 ./toastergui/urls.py
</code>

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

<code>
$ pylint --load-plugins pylint_django toastergui/tests.py
</code>

== Working with design ==

Yes, the Yocto Project is one of those lucky projects with designers around to help in UI matters. We have a document explaining how to work with the design contributors: [[File:Working_with_design.pdf]]

== Debugging Toaster ==

Toaster can be quite complex given that it includes a wide range of technologies and languages from bitbake, events, python, Django, Sqlite, Javescript, CCS, HTML, and more. Here are hints and tips for debugging all of these aspects of Toaster.

=== General debugging tips ===

* The logger is your best friend
* When possible, build from command line in the Toaster environment to see log messages and errors directly
* Use "quilt-native" as your quick sanity build target
* For python code you can use pudb (https://wiki.yoctoproject.org/wiki/TipsAndTricks/DebuggingBitbakeInPudb)
* For bbclass code (toaster.bbclass) find or set up a python section to use pudb (see above link)
* Brute force find is your friend for mysterious error messages and tracking all places a variable is used
** find bitbake/lib/toaster -exec grep -l "SEARCH_TEXT" {} \; 2> /dev/null
* Look in https://wiki.yoctoproject.org/wiki/TipsAndTricks

=== How to debug a ... ===

* A stalled build:
** Cancel the build and look at one of the below logs to find the exception. The error message should give a clue on the user resolution or indicate a bug to file
** Failures in git and shell calls are (as of YP-2.4 Rocko) captured as visible build errors in the GUI
* Python file of a build management script (localhostbecontroller.py,...):
** Use log statements, and watch "build/toaster_runbuilds.log" (where you started Toaster)
* Python file of a build runtime scripts:
** Use log statements, and watch "build-toaster-x/toaster_ui.log"
* Python file of a Toaster management script (projects, tables):
** Use log statements, and watch "build/toaster_web.log" (where you started Toaster)
* Template file
** Usually the error will appear in the browser via Django, with a trace
** Add random display text to insure the page your editing is the one that shows
** Display internal values via page template to see what gets passed
** Add internal values into the context and add to the page template
** Use the Firefox/Chrome developer "inspector" to examine the DOM
* Javascript file
** Use the Firefox/Chrome developer "inspector" to do inline debugging
** Watch the Firefox/Chrome developer "console" for error messages
* The Toaster Database
** Use an application like "sqlitebrowser" to examine the tables and records
** Use a Python script to find and dump desired database records (see http://events.linuxfoundation.org/sites/events/files/slides/BitbakeAnalytics_ELC_Portland.pdf)

=== Specific error scenarios ... ===

* The build progress gets stuck at "Loading..."
** You have a syntax error, either in 'localhostbecontroller.py', the helper classes in 'models.py', or in the related javascript. Look at the tail of "toaster_runbuilds.log" file for pythons errors, and in your browser's "Console" for javascript errors.
* The message "Sorry, An error has occurred loading this page"
** The included javascript has a syntax error. The HTML page being rendered has local JS code at the top that caches these errors and displays the observed message (in order to be less ugly to the customer). That code also tries to add the error message to the browser console but that does not always work, and you are left with no clue
** You can start bisecting the recent edits you made to the respective JS files to locate the problem change
** You can also add a breakpoint in the mentioned catch code to see if you can see the error directly

=== How to update from a fix in a ... ===

* Template file: save your text edit and refresh page
* Javascript file: save your text edit and refresh page (or leave page and come back)
* Builder scripts: save your text edit and restart Toaster ("localhostbecontroller.py", "buildinfohelper.py", "toasterui.py", "bbcontroller.py",...)
* Views and URLs: save your text edit and refresh page
* Model class members: save your text edit and restart Toaster. You may also need a migration file.
* Model method: save your text edit and refresh page, else restart Toaster
* Fixture file (settings.xml, poky.xml, custom.xml): delete Toaster database and restart fresh

=== Where to debug ... ===

* Toaster start and stop, database init
** bin/toaster
* Project Defaults (Default machine, distro, layers, ...)
** bldcontrol/management/commands/checksettings.py
** orm/fixtures/*.xml
* Layer Index content (Available machines, distros, layers, ...)
** orm/management/commands/lsupdates.py
* Build cloning
** bldcontrol/localhostbecontroller.py
* Build start
** bldcontrol/localhostbecontroller.py
* URL mappings
** toastergui/urls.py
* Views
** toastergui/views.py
** toastergui/templates/*
* Toaster Tables (Layers, Distros, Machines, ...)
** toastergui/tables.py
** toastergui/static/js/libtoaster.js
* Events
** meta/classes/toaster.bbclass ->
** lib/bb/ui/toasterui.py ->
** lib/bb/ui/buildinfohelper.py ->
** orm/models.py et. al.

Contribute to Toaster

2025-11-15T06:08:38Z

David Reyna:

[[Category:Toaster]]
This page summarises the Toaster development process. We hope this will help you start contributing to the project. We were previously using a process that involved the toaster-next branch on poky-contrib. This workflow was especially useful in an environment where Toaster was undergoing a lot of change and needed the ability to layer patch sets upon each other before they had been taken up by bitbake. Since that is not currently the case, toaster-next just adds complication; so it has been removed from this workflow. See [[Contribute to Toaster (toaster-next version)]] for the old toaster-next based workflow.

== What can I do? ==

The [https://bugzilla.yoctoproject.org/buglist.cgi?product=Toaster Yocto Project Bugzilla instance] lists all the things that need to be done:

* If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list]
* If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list] to find out why the design work is not done yet

== Set up the local repository ==

For development of Toaster we recommend setting up a local install of Toaster. Installation instructions are available in the main [https://docs.yoctoproject.org/toaster-manual/index.html Toaster documentation]

== Submitting patches ==

Publishing your patches to Toaster is a two step process.
# Sending patches to the [https://lists.yoctoproject.org/listinfo/toaster/| Toaster mailing list] for review
# Submitting the patches that you reviewed to the upstream repository

By submitting your patches first to the Toaster mailing list, you can be sure the patches are reviewed by the people in the community who are familiar with the Toaster source code, and who have experience developing web applications.

That also means that, by the time the patches are submitted to the upstream mailing lists, they are in pretty good shape. That helps the project maintainers, and hopefully also helps you.

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].
All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

=== Workflow (bitbake-setup) ===
 
1) Download master branch of the yocto project

<pre>
$ git clone https://git.openembedded.org/bitbake/
$ ./bitbake/bin/bitbake-setup settings set global top-dir-prefix $PWD
$ ./bitbake/bin/bitbake-setup init --non-interactive poky-master poky distro/poky machine/qemux86-64
$ cd bitbake-builds/poky-master-poky-distro_poky-machine_qemux86-64
</pre>

 
2) Start your feature branch off of master, name style of branch is convention, but suggested.

<pre>
$ cd layers/bitbake
$ git checkout -b username/toaster/FeatureOrBug origin/master
</pre>

 
3) Do Work

 
4) Rebase on master. It has probably changed while you were working (unless you are really really fast!)

<pre>
$ git rebase origin/master
</pre>

 
5) Create your commit

NOTE: The format of the commit message should be like this

<pre>
toaster: <module> <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

 
6) Prepare your patch

<code>
$ git format-patch -1
</code>

 
7) Send your patch to the bitbake mailing list

Email list: bitbake-devel@lists.openembedded.org
 
NOTE: For the email's subject line, prepend "[bitbake-devel]" to the patch's subject line

For example:

<pre>
TO: bitbake-devel@lists.openembedded.org
SUBJECT: [bitbake-devel][PATCH] toaster: support bitbake-setup
CONTENT:
From c063178ab0971fe297a9e9bca6e6b04eafc5149c Mon Sep 17 00:00:00 2001
From: David Reyna <David.Reyna@windriver.com>
Date: Thu, 30 Oct 2025 14:03:35 -0700
Subject: [PATCH] toaster: support bitbake-setup

This adds support for the new bitbake-setup
<<<CUT>>>
[YOCTO #16012]

Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
bin/toaster | 21 ++--
.../bldcontrol/localhostbecontroller.py | 99 +++++++++++
<<<CUT>>>
</pre>

== Workflow (Poky Classic, OBSOLETE) ==
To contribute to toaster you will also need authorization to write to the upstream yocto project repository. Contact a member of the toaster team for details.

1) Download master branch of the yocto project
<code> git clone git://git.yoctoproject.org/poky && cd poky </code>

2) Add poky-contrib to the local repository you set up above
<code> git remote add poky-contrib ssh://git@push.yoctoproject.org/poky-contrib </code>

3) Fetch the poky-contrib branches
<code> git fetch --all </code>

4) Start your feature branch off of master, name style of branch is convention, but suggested.
<code> git checkout -b username/toaster/FeatureOrBug origin/master </code>

5) Do Work

6) Test the changes. Run the Django unit tests. People put effort into these so we should make sure we use them. This assumes you have phantomjs installed. This can usually be done from the distribution <code> apt-get install phantomjs</code>, for example. If you want to test against Chrome or Firefox, see the README in bitbake/lib/toaster/tests/browser.
<code> pip3 install selenium </code>
<code>TOASTER_TESTS_BROWSER=phantomjs bitbake/lib/toaster/manage.py test orm toastergui tests.browser </code>
 
Note: If you would like to run the tests in a container so they are repeatable and do not continually break due to host upgrades see [[Running Toaster Tests with Containers]]

7) Rebase on master. It has probably changed while you were working (unless you are really really fast!)
<code> git rebase origin/master </code>

8) Push your feature branch to poky-contrib
<code> git push -u poky-contrib username/toaster/FeatureOrBug:username/toaster/FeatureOrBug</code>

9) Send to the toaster-mailing list using one of the methods outlined below.

10) NOTE: when the patch has been accepted upstream, you can clean up your poy-contrib branch with:

<code> git push -u poky-contrib :username/toaster/FeatureOrBug </code>

=== Sending patches to Toaster Project ===

NOTE: The format of the commit message should be like this

<pre>
toaster: <module> <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] ( toaster@yoctoproject.org ) by "git send-email".

e.g.
<code>
$ git send-email HEAD^
</code>

Alternatively, you can use the utilities in the script directory to prepare your patch

1) Use the create-pull-request script (from poky) to create a pull request while on your feature branch
<code>./scripts/create-pull-request -s "toaster: Fixes and clean ups" -u poky-contrib -r origin/master</code>

2) Review their content, especially the summary mail:
<code>edit ./pull-<pid>/0000-cover-letter.patch</code>

3)When you are satisfied, you can send them with:
<code>./scripts/send-pull-request -a -p ./pull-<pid></code> -t toaster@yoctoproject.org

A comprehensive document about commit messages is available on the [http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines openembedded wiki]

More help learning git is available on [https://try.github.io github] and [http://git-scm.com/documentation/ the official documentation]

=== Sending branches to Toaster Project ===

If you wish to submit whole branches please use the poky-contrib repository see [[Poky Contributions#Poky_Contrib_Branch]] for setup guide.

Once you have pushed a branch please then send an email to the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] with the subject in the following format:

<code>
[review-request] my_branch_name
</code>

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

=== Submitting patch sets for integration into Bitbake ===

Toaster patches are normally submitted upstream to the BitBake repository by the reviewer (not the author). This tells the upstream maintainers that the patches have been reviewed by the people who are familiar with the Toaster source code, and makes their busy lives a bit easier.

Since development happens on the poky-contrib repository, but the patches need to be merged to the Bitbake repository, the following process should be executed.

1) Bring master up to date
<code> git fetch origin master</code>
<code> git checkout master </code>
<code> git pull [master] </code>

2) Checkout the target branch you wish to upstream

(a) If you are working with a single branch do this:

<code>git checkout username/toaster/FeatureOrBug</code>

(b) If you are including several branches into this one submission, gather the commit IDs for each branch and do this:

<code>git cherry-pick commit-id-a </code>
<code>git cherry-pick commit-id-b </code>
<code>... </code>

3) Create a new branch for submission
<code> git checkout -b yourname/submit/username/toaster/FeatureOrBug </code>
This will look something like
<code> git checkout -b bavery/submit/bob/toaster/FixBug1234 </code>
where bavery is signing off on and upstreaming bob's fix to the Yocto Bugzilla bug 1234.
 
----
----
* Alternatively, you can use the patchworks web site:
2) Create a new branch for submission
<code> git checkout -b yourname/submit/username/toaster/FeatureOrBug </code>
This will look something like
<code> git checkout -b bavery/submit/bob/toaster/FixBug1234 </code>
where bavery is signing off on and upstreaming bob's fix to the Yocto Bugzilla bug 1234.
3) Download and apply the mbox patch from the website: https://patchwork.openembedded.org/project/toaster/patches/

<code> git am -s the-downloaded-patch.mbox </code>
----
----
 
4) Make sure the branch is rebased on current master.
<code>git rebase origin/master</code>

5) Test the changes. Run the Django unit tests. People put effort into these so we should make sure we use them. his assumes you have phantomjs installed. This can usually be done from the distribution <code> apt-get install phantomjs</code>, for example. If you want to test against Chrome or Firefox, see the README in bitbake/lib/toaster/tests/browser.
<code> pip3 install selenium </code>
<code>TOASTER_TESTS_BROWSER=phantomjs bitbake/lib/toaster/manage.py test orm toastergui tests.browser </code>

6) Add signed off by to the commit messages (Note: If you applied the mbox patch with <code> git am -s foo.mbox</code> You do not need to sign off again.)
<code>git filter-branch -f --msg-filter 'cat && echo "Signed-off-by: $(git config --get user.name) <$(git config --get user.email)>"' master..HEAD</code>

7) Push the modified commit messages and rebased version to poky-contrib
<code>git push -u poky-contrib yourname/submit/username/toaster/FeatureOrBug </code>

8) Use the create-pull-request script (from poky) to create a pull request for the appropriate tree/mailing list

(a) For the 'bitbake' tree (e.g. bitbake/.../toasterui.py, bitbake/.../toastergui/*):

<code>./scripts/create-pull-request -d bitbake -s "toaster: Fixes and clean ups" -u poky-contrib -r origin/master</code>

(b) for the 'meta' tree (e.g. meta/classes/toaster.class):

<code>./scripts/create-pull-request -d meta -s "toaster: Fixes and clean ups" -u poky-contrib -r origin/master</code>
 
Note: If the patch creates any NEW files, the integration scripts that pull it into bitbake will fail. So, if new files are created as part of this patch set, you need to explicitly point that out in the body of the email for the patch set or do it as a PR rather than as a patch set.


9) Review their content, especially the summary mail:
<code>edit ./pull-<pid>/0000-cover-letter.patch</code>

10) When you are satisfied, you can send them with:
--- for the 'bitbake' tree:
<code>./scripts/send-pull-request -a -p ./pull-<pid></code> -t bitbake-devel@lists.openembedded.org
--- for the 'meta' tree:
<code>./scripts/send-pull-request -a -p ./pull-<pid></code> -t openembedded-core@lists.openembedded.org

==== Submitting patches for prior releases ====

The procedure is the same, but using the prior release as the base branch instead of the "master" branch in bitbake.

Also, make sure that you add the name of the prior release for which the patchset is intended in the prefix of the patchset, as parameter to the "create-pull-request" command, e.g. '''-p 1.26''' for the 1.26 branch.

==== Gotchas ====
===== Too Big =====
Sometimes the mailer will refuse to send patches, especially on binary or long-line files. The proper way to go around that is to reply to the patchset you've submitted to the mailing list, asking for a git pull directly from the poky-contrib branch.
===== One Patch of a Long Patch Set Needs Resubmission =====
Suppose you upstream a 10 commit patch set to the bitbake-devel list and someone finds an issue with patch #3. Regenerating the whole series is silly so how do you address this? First, follow the bitbake submission steps until you end up on the yourname/submit/the/target/branch branch. Then you can (note the reset --hard will wipe any local changes in your working dir so commit or stash first): 
<code>
git checkout -b yourname/submit/the/target/branch-newHEAD
git reset --hard <commit of resubmission issue>
git commit --amend --signoff
git send-email --in-reply-to="longNumber.git.me@mycomp.com" --subject-prefix="bitbake-devel] [PATCHVX 03/10" --to=bitbake-devel@lists.openembedded.org --no-chain-reply-to --suppress-cc=all -M -1 --relative=bitbake
</code> 
The longNumber.git.me@mycomp.com comes from the email message id for the particular patch #3 so that the email threading works. In gmail, you can click on the dropdown button on the right side of the screen and choose "Show Original". This will have a field in the header like Message-Id: <4551b56f132497c055f39567946a5d3be347d770.1468363530.git.myemailusername@mycompany.com> The entire string except the '<>' are used. for example: 
<code>
--in-reply-to="4551b56f132497c055f39567946a5d3be347d770.1468363530.git.myemailusername@mycompany.com"
</code> 

The first issue you are likely to face is that the git filter-branch command in the standard instructions may sign off on too many commits. If you know you just want to sign off on the last 7 commits on the yourname/submit/the/target/branch you can:
<code>
git filter-branch -f --msg-filter 'cat && echo "Signed-off-by: $(git config --get user.name) <$(git config --get user.email)>"' HEAD~7..HEAD
</code>

=== Submitting patches for documentation ===

Documentation patches should be sent to [https://lists.yoctoproject.org/listinfo/yocto Yocto mailing list] with [yocto-docs] in the subject, CC Scott Rifenbark (and make sure you send it to his gmail, not his defunct Intel address). For his email address, look at [http://lists.openembedded.org/pipermail/bitbake-devel/2015-October/006632.html this post].

== Code syle guide ==

=== Templates ===

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

'''Yes please:'''

<pre>
{{var}}

<div>
{# Maintaining indentation #}
{% if %}
this
{% else %}
that
{% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
{{layer}}
{% endfor %}

</pre>

'''No thank you:'''
<pre>
{{var}}
<div>
{# Maintaining indentation #}
{%if%}this{%else%}that{%endif%}
</div>

{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}
</pre>

Note:
* Maintain indentation as you would with other languages
* White space after '%'
* Comment blocks for longer comments

=== Javascript ===

'''Yes please:'''
<pre>

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
cheddar : 1,
stilton : 2,
emmental : 3,
};

function doThingsHere(){
return 1;
}

/* If one equals two do some other things and make sure that
* if the the click handler is setup correctly.
*/
if (one === two) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function (event){
alert("Hello");
});
}

$("#little-mouse").focusout(function(){
alert("bye")
});

if (oneVar)
noThingHere();
else
doThingHere();
</pre>

'''No thank you:'''
<pre>
// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2, emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
alert("bye")
});

if (oneVar)
{
noThingHere();
} else { doThingHere(); }
</pre>

Note:
* Variables should be marked with "var"
* Semicolons should be used
* Keep as close to 80 cols as possible
* Use 2 space per indentation
* Open curly braces after parenthesis for functions and close on a new line
* Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:
<code>
$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js
</code>

=== HTML ===

'''Yes please:'''
<pre>
<div id="something-area">
This is some text
</div>

<div>
This is some text
</div>

</pre>

'''No thank you:'''
<pre>
<div id="somethingarea">
This is some text</div>
<div id="somethingarea">
This is
some text


</div>
</pre>

Note:
* 2 space indentation
* Lower case, ids hyphenated when multiple words
* No duplicate ids

* Run your HTML through a [http://validator.w3.org/#validate_by_input HTML validator] available for [http://validator.w3.org/source/ local install]. The w3c validator it's self doesn't currently validate html5, it uses as a back end [https://validator.github.io/validator/ Nu Html Checker] which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:
<pre>
$ mkdir html5-validator && cd html5-validator
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$ git clone https://github.com/validator/validator.git
$ python build/build.py all
$ python build/build.py all
</pre>

HTML can be indented quickly using tidy, for example:
<pre>
tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes ./index.html
</pre>

=== Python ===

Lenient [https://www.python.org/dev/peps/pep-0008 pep8]
Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and [http://pep8.readthedocs.org/en/latest/intro.html#error-codes error code list]

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.
<code>
$ pep8 ./toastergui/urls.py
</code>

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

<code>
$ pylint --load-plugins pylint_django toastergui/tests.py
</code>

== Working with design ==

Yes, the Yocto Project is one of those lucky projects with designers around to help in UI matters. We have a document explaining how to work with the design contributors: [[File:Working_with_design.pdf]]

== Debugging Toaster ==

Toaster can be quite complex given that it includes a wide range of technologies and languages from bitbake, events, python, Django, Sqlite, Javescript, CCS, HTML, and more. Here are hints and tips for debugging all of these aspects of Toaster.

=== General debugging tips ===

* The logger is your best friend
* When possible, build from command line in the Toaster environment to see log messages and errors directly
* Use "quilt-native" as your quick sanity build target
* For python code you can use pudb (https://wiki.yoctoproject.org/wiki/TipsAndTricks/DebuggingBitbakeInPudb)
* For bbclass code (toaster.bbclass) find or set up a python section to use pudb (see above link)
* Brute force find is your friend for mysterious error messages and tracking all places a variable is used
** find bitbake/lib/toaster -exec grep -l "SEARCH_TEXT" {} \; 2> /dev/null
* Look in https://wiki.yoctoproject.org/wiki/TipsAndTricks

=== How to debug a ... ===

* A stalled build:
** Cancel the build and look at one of the below logs to find the exception. The error message should give a clue on the user resolution or indicate a bug to file
** Failures in git and shell calls are (as of YP-2.4 Rocko) captured as visible build errors in the GUI
* Python file of a build management script (localhostbecontroller.py,...):
** Use log statements, and watch "build/toaster_runbuilds.log" (where you started Toaster)
* Python file of a build runtime scripts:
** Use log statements, and watch "build-toaster-x/toaster_ui.log"
* Python file of a Toaster management script (projects, tables):
** Use log statements, and watch "build/toaster_web.log" (where you started Toaster)
* Template file
** Usually the error will appear in the browser via Django, with a trace
** Add random display text to insure the page your editing is the one that shows
** Display internal values via page template to see what gets passed
** Add internal values into the context and add to the page template
** Use the Firefox/Chrome developer "inspector" to examine the DOM
* Javascript file
** Use the Firefox/Chrome developer "inspector" to do inline debugging
** Watch the Firefox/Chrome developer "console" for error messages
* The Toaster Database
** Use an application like "sqlitebrowser" to examine the tables and records
** Use a Python script to find and dump desired database records (see http://events.linuxfoundation.org/sites/events/files/slides/BitbakeAnalytics_ELC_Portland.pdf)

=== Specific error scenarios ... ===

* The build progress gets stuck at "Loading..."
** You have a syntax error, either in 'localhostbecontroller.py', the helper classes in 'models.py', or in the related javascript. Look at the tail of "toaster_runbuilds.log" file for pythons errors, and in your browser's "Console" for javascript errors.
* The message "Sorry, An error has occurred loading this page"
** The included javascript has a syntax error. The HTML page being rendered has local JS code at the top that caches these errors and displays the observed message (in order to be less ugly to the customer). That code also tries to add the error message to the browser console but that does not always work, and you are left with no clue
** You can start bisecting the recent edits you made to the respective JS files to locate the problem change
** You can also add a breakpoint in the mentioned catch code to see if you can see the error directly

=== How to update from a fix in a ... ===

* Template file: save your text edit and refresh page
* Javascript file: save your text edit and refresh page (or leave page and come back)
* Builder scripts: save your text edit and restart Toaster ("localhostbecontroller.py", "buildinfohelper.py", "toasterui.py", "bbcontroller.py",...)
* Views and URLs: save your text edit and refresh page
* Model class members: save your text edit and restart Toaster. You may also need a migration file.
* Model method: save your text edit and refresh page, else restart Toaster
* Fixture file (settings.xml, poky.xml, custom.xml): delete Toaster database and restart fresh

=== Where to debug ... ===

* Toaster start and stop, database init
** bin/toaster
* Project Defaults (Default machine, distro, layers, ...)
** bldcontrol/management/commands/checksettings.py
** orm/fixtures/*.xml
* Layer Index content (Available machines, distros, layers, ...)
** orm/management/commands/lsupdates.py
* Build cloning
** bldcontrol/localhostbecontroller.py
* Build start
** bldcontrol/localhostbecontroller.py
* URL mappings
** toastergui/urls.py
* Views
** toastergui/views.py
** toastergui/templates/*
* Toaster Tables (Layers, Distros, Machines, ...)
** toastergui/tables.py
** toastergui/static/js/libtoaster.js
* Events
** meta/classes/toaster.bbclass ->
** lib/bb/ui/toasterui.py ->
** lib/bb/ui/buildinfohelper.py ->
** orm/models.py et. al.

Contribute to Toaster

2025-11-15T06:07:42Z

David Reyna: Add plain workflow for bitbake-setup model

[[Category:Toaster]]
This page summarises the Toaster development process. We hope this will help you start contributing to the project. We were previously using a process that involved the toaster-next branch on poky-contrib. This workflow was especially useful in an environment where Toaster was undergoing a lot of change and needed the ability to layer patch sets upon each other before they had been taken up by bitbake. Since that is not currently the case, toaster-next just adds complication; so it has been removed from this workflow. See [[Contribute to Toaster (toaster-next version)]] for the old toaster-next based workflow.

== What can I do? ==

The [https://bugzilla.yoctoproject.org/buglist.cgi?product=Toaster Yocto Project Bugzilla instance] lists all the things that need to be done:

* If the issue says GUI design available in the Whiteboard field, there is a design specification document attached to the issue that you should follow. Send questions / comments about it to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list]
* If the issue says GUI design pending in the Whiteboard field, there is some design work still to be done. Feel free to take the issue and send an email to the [https://lists.yoctoproject.org/listinfo/toaster Toaster mailing list] to find out why the design work is not done yet

== Set up the local repository ==

For development of Toaster we recommend setting up a local install of Toaster. Installation instructions are available in the main [https://docs.yoctoproject.org/toaster-manual/index.html Toaster documentation]

== Submitting patches ==

Publishing your patches to Toaster is a two step process.
# Sending patches to the [https://lists.yoctoproject.org/listinfo/toaster/| Toaster mailing list] for review
# Submitting the patches that you reviewed to the upstream repository

By submitting your patches first to the Toaster mailing list, you can be sure the patches are reviewed by the people in the community who are familiar with the Toaster source code, and who have experience developing web applications.

That also means that, by the time the patches are submitted to the upstream mailing lists, they are in pretty good shape. That helps the project maintainers, and hopefully also helps you.

Toaster code lives in Bitbake repository at [http://git.openembedded.org/bitbake/|http://git.openembedded.org/bitbake/].
All contributions must be upstreamed to the Bitbake repository in order to make it to the "master" branch of the poky/ repository.

=== Workflow (bitbake-setup) ===
 
1) Download master branch of the yocto project

<pre>
$ git clone https://git.openembedded.org/bitbake/
$ ./bitbake/bin/bitbake-setup settings set global top-dir-prefix $PWD
$ ./bitbake/bin/bitbake-setup init --non-interactive poky-master poky distro/poky machine/qemux86-64
$ cd bitbake-builds/poky-master-poky-distro_poky-machine_qemux86-64
</pre>

 
2) Start your feature branch off of master, name style of branch is convention, but suggested.

<pre>
$ cd layers/bitbake
$ git checkout -b username/toaster/FeatureOrBug origin/master
</pre>

 
3) Do Work

 
4) Rebase on master. It has probably changed while you were working (unless you are really really fast!)

<pre>
$ git rebase origin/master
</pre>

 
5) Create your commit

NOTE: The format of the commit message should be like this

<pre>
toaster: <module> <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

 
6) Prepare your patch

<code>
$ git format-patch -1
</code>

 
7) Send your patch to the bitbake mailing list

Email list: bitbake-devel@lists.openembedded.org
 
NOTE: For the email's subject line, prepend "[bitbake-devel]" to the patch's subject line

For example:

<pre>
TO: bitbake-devel@lists.openembedded.org
SUBJECT: [bitbake-devel][PATCH] toaster: support bitbake-setup
CONTENT:
From c063178ab0971fe297a9e9bca6e6b04eafc5149c Mon Sep 17 00:00:00 2001
From: David Reyna <David.Reyna@windriver.com>
Date: Thu, 30 Oct 2025 14:03:35 -0700
Subject: [PATCH] toaster: support bitbake-setup

This adds support for the new bitbake-setup
<<<CUT>>>
[YOCTO #16012]

Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
bin/toaster | 21 ++--
.../bldcontrol/localhostbecontroller.py | 99 +++++++++++
<<<CUT>>>
</pre>

=== Workflow (Poky Classic, OBSOLETE) ===
To contribute to toaster you will also need authorization to write to the upstream yocto project repository. Contact a member of the toaster team for details.

1) Download master branch of the yocto project
<code> git clone git://git.yoctoproject.org/poky && cd poky </code>

2) Add poky-contrib to the local repository you set up above
<code> git remote add poky-contrib ssh://git@push.yoctoproject.org/poky-contrib </code>

3) Fetch the poky-contrib branches
<code> git fetch --all </code>

4) Start your feature branch off of master, name style of branch is convention, but suggested.
<code> git checkout -b username/toaster/FeatureOrBug origin/master </code>

5) Do Work

6) Test the changes. Run the Django unit tests. People put effort into these so we should make sure we use them. This assumes you have phantomjs installed. This can usually be done from the distribution <code> apt-get install phantomjs</code>, for example. If you want to test against Chrome or Firefox, see the README in bitbake/lib/toaster/tests/browser.
<code> pip3 install selenium </code>
<code>TOASTER_TESTS_BROWSER=phantomjs bitbake/lib/toaster/manage.py test orm toastergui tests.browser </code>
 
Note: If you would like to run the tests in a container so they are repeatable and do not continually break due to host upgrades see [[Running Toaster Tests with Containers]]

7) Rebase on master. It has probably changed while you were working (unless you are really really fast!)
<code> git rebase origin/master </code>

8) Push your feature branch to poky-contrib
<code> git push -u poky-contrib username/toaster/FeatureOrBug:username/toaster/FeatureOrBug</code>

9) Send to the toaster-mailing list using one of the methods outlined below.

10) NOTE: when the patch has been accepted upstream, you can clean up your poy-contrib branch with:

<code> git push -u poky-contrib :username/toaster/FeatureOrBug </code>

=== Sending patches to Toaster Project ===

NOTE: The format of the commit message should be like this

<pre>
toaster: <module> <short one line summary>

long(er) description

[YOCTO #0000]

Signed-off-by: First Last <name@domain.com>
</pre>

Where YOCTO #0000 is the related bug number if there is one. Signed off by with your git commit -s credentials.

We accept patches on the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] ( toaster@yoctoproject.org ) by "git send-email".

e.g.
<code>
$ git send-email HEAD^
</code>

Alternatively, you can use the utilities in the script directory to prepare your patch

1) Use the create-pull-request script (from poky) to create a pull request while on your feature branch
<code>./scripts/create-pull-request -s "toaster: Fixes and clean ups" -u poky-contrib -r origin/master</code>

2) Review their content, especially the summary mail:
<code>edit ./pull-<pid>/0000-cover-letter.patch</code>

3)When you are satisfied, you can send them with:
<code>./scripts/send-pull-request -a -p ./pull-<pid></code> -t toaster@yoctoproject.org

A comprehensive document about commit messages is available on the [http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines openembedded wiki]

More help learning git is available on [https://try.github.io github] and [http://git-scm.com/documentation/ the official documentation]

=== Sending branches to Toaster Project ===

If you wish to submit whole branches please use the poky-contrib repository see [[Poky Contributions#Poky_Contrib_Branch]] for setup guide.

Once you have pushed a branch please then send an email to the [https://www.yoctoproject.org/tools-resources/community/mailing-lists toaster mailing list] with the subject in the following format:

<code>
[review-request] my_branch_name
</code>

In the body of the email it's useful to describe your branch's functionality, which commits and a link to the git web.

If you need any assistance please post on the mailing list.

=== Submitting patch sets for integration into Bitbake ===

Toaster patches are normally submitted upstream to the BitBake repository by the reviewer (not the author). This tells the upstream maintainers that the patches have been reviewed by the people who are familiar with the Toaster source code, and makes their busy lives a bit easier.

Since development happens on the poky-contrib repository, but the patches need to be merged to the Bitbake repository, the following process should be executed.

1) Bring master up to date
<code> git fetch origin master</code>
<code> git checkout master </code>
<code> git pull [master] </code>

2) Checkout the target branch you wish to upstream

(a) If you are working with a single branch do this:

<code>git checkout username/toaster/FeatureOrBug</code>

(b) If you are including several branches into this one submission, gather the commit IDs for each branch and do this:

<code>git cherry-pick commit-id-a </code>
<code>git cherry-pick commit-id-b </code>
<code>... </code>

3) Create a new branch for submission
<code> git checkout -b yourname/submit/username/toaster/FeatureOrBug </code>
This will look something like
<code> git checkout -b bavery/submit/bob/toaster/FixBug1234 </code>
where bavery is signing off on and upstreaming bob's fix to the Yocto Bugzilla bug 1234.
 
----
----
* Alternatively, you can use the patchworks web site:
2) Create a new branch for submission
<code> git checkout -b yourname/submit/username/toaster/FeatureOrBug </code>
This will look something like
<code> git checkout -b bavery/submit/bob/toaster/FixBug1234 </code>
where bavery is signing off on and upstreaming bob's fix to the Yocto Bugzilla bug 1234.
3) Download and apply the mbox patch from the website: https://patchwork.openembedded.org/project/toaster/patches/

<code> git am -s the-downloaded-patch.mbox </code>
----
----
 
4) Make sure the branch is rebased on current master.
<code>git rebase origin/master</code>

5) Test the changes. Run the Django unit tests. People put effort into these so we should make sure we use them. his assumes you have phantomjs installed. This can usually be done from the distribution <code> apt-get install phantomjs</code>, for example. If you want to test against Chrome or Firefox, see the README in bitbake/lib/toaster/tests/browser.
<code> pip3 install selenium </code>
<code>TOASTER_TESTS_BROWSER=phantomjs bitbake/lib/toaster/manage.py test orm toastergui tests.browser </code>

6) Add signed off by to the commit messages (Note: If you applied the mbox patch with <code> git am -s foo.mbox</code> You do not need to sign off again.)
<code>git filter-branch -f --msg-filter 'cat && echo "Signed-off-by: $(git config --get user.name) <$(git config --get user.email)>"' master..HEAD</code>

7) Push the modified commit messages and rebased version to poky-contrib
<code>git push -u poky-contrib yourname/submit/username/toaster/FeatureOrBug </code>

8) Use the create-pull-request script (from poky) to create a pull request for the appropriate tree/mailing list

(a) For the 'bitbake' tree (e.g. bitbake/.../toasterui.py, bitbake/.../toastergui/*):

<code>./scripts/create-pull-request -d bitbake -s "toaster: Fixes and clean ups" -u poky-contrib -r origin/master</code>

(b) for the 'meta' tree (e.g. meta/classes/toaster.class):

<code>./scripts/create-pull-request -d meta -s "toaster: Fixes and clean ups" -u poky-contrib -r origin/master</code>
 
Note: If the patch creates any NEW files, the integration scripts that pull it into bitbake will fail. So, if new files are created as part of this patch set, you need to explicitly point that out in the body of the email for the patch set or do it as a PR rather than as a patch set.


9) Review their content, especially the summary mail:
<code>edit ./pull-<pid>/0000-cover-letter.patch</code>

10) When you are satisfied, you can send them with:
--- for the 'bitbake' tree:
<code>./scripts/send-pull-request -a -p ./pull-<pid></code> -t bitbake-devel@lists.openembedded.org
--- for the 'meta' tree:
<code>./scripts/send-pull-request -a -p ./pull-<pid></code> -t openembedded-core@lists.openembedded.org

==== Submitting patches for prior releases ====

The procedure is the same, but using the prior release as the base branch instead of the "master" branch in bitbake.

Also, make sure that you add the name of the prior release for which the patchset is intended in the prefix of the patchset, as parameter to the "create-pull-request" command, e.g. '''-p 1.26''' for the 1.26 branch.

==== Gotchas ====
===== Too Big =====
Sometimes the mailer will refuse to send patches, especially on binary or long-line files. The proper way to go around that is to reply to the patchset you've submitted to the mailing list, asking for a git pull directly from the poky-contrib branch.
===== One Patch of a Long Patch Set Needs Resubmission =====
Suppose you upstream a 10 commit patch set to the bitbake-devel list and someone finds an issue with patch #3. Regenerating the whole series is silly so how do you address this? First, follow the bitbake submission steps until you end up on the yourname/submit/the/target/branch branch. Then you can (note the reset --hard will wipe any local changes in your working dir so commit or stash first): 
<code>
git checkout -b yourname/submit/the/target/branch-newHEAD
git reset --hard <commit of resubmission issue>
git commit --amend --signoff
git send-email --in-reply-to="longNumber.git.me@mycomp.com" --subject-prefix="bitbake-devel] [PATCHVX 03/10" --to=bitbake-devel@lists.openembedded.org --no-chain-reply-to --suppress-cc=all -M -1 --relative=bitbake
</code> 
The longNumber.git.me@mycomp.com comes from the email message id for the particular patch #3 so that the email threading works. In gmail, you can click on the dropdown button on the right side of the screen and choose "Show Original". This will have a field in the header like Message-Id: <4551b56f132497c055f39567946a5d3be347d770.1468363530.git.myemailusername@mycompany.com> The entire string except the '<>' are used. for example: 
<code>
--in-reply-to="4551b56f132497c055f39567946a5d3be347d770.1468363530.git.myemailusername@mycompany.com"
</code> 

The first issue you are likely to face is that the git filter-branch command in the standard instructions may sign off on too many commits. If you know you just want to sign off on the last 7 commits on the yourname/submit/the/target/branch you can:
<code>
git filter-branch -f --msg-filter 'cat && echo "Signed-off-by: $(git config --get user.name) <$(git config --get user.email)>"' HEAD~7..HEAD
</code>

=== Submitting patches for documentation ===

Documentation patches should be sent to [https://lists.yoctoproject.org/listinfo/yocto Yocto mailing list] with [yocto-docs] in the subject, CC Scott Rifenbark (and make sure you send it to his gmail, not his defunct Intel address). For his email address, look at [http://lists.openembedded.org/pipermail/bitbake-devel/2015-October/006632.html this post].

== Code syle guide ==

=== Templates ===

Django has a template language which allows us to render pages based on the data (context). We use the template language to setup the initial state of the page and to create re-usable components that can be included in other pages.

The recommend template code style is as follows

'''Yes please:'''

<pre>
{{var}}

<div>
{# Maintaining indentation #}
{% if %}
this
{% else %}
that
{% endif %}
</div>

{% comment %}
This is a longer comment that describes all the things
that are below in quite a bit of detail because they're
a little more difficult to understand.
{% endcomment %}

{% for layer in layers_list %}
{{layer}}
{% endfor %}

</pre>

'''No thank you:'''
<pre>
{{var}}
<div>
{# Maintaining indentation #}
{%if%}this{%else%}that{%endif%}
</div>

{#This is a longer comment that describes all the things that are below in quite a bit of detail because they're a little more difficult to understand. #}
{%for o in layers_list%}{{o}}{%endfor%}
</pre>

Note:
* Maintain indentation as you would with other languages
* White space after '%'
* Comment blocks for longer comments

=== Javascript ===

'''Yes please:'''
<pre>

"use strict";

/* These hold some numbers */
var oneVar = 1;
var twoVar = 2;

var cheesesTypes = {
cheddar : 1,
stilton : 2,
emmental : 3,
};

function doThingsHere(){
return 1;
}

/* If one equals two do some other things and make sure that
* if the the click handler is setup correctly.
*/
if (one === two) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function (event){
alert("Hello");
});
}

$("#little-mouse").focusout(function(){
alert("bye")
});

if (oneVar)
noThingHere();
else
doThingHere();
</pre>

'''No thank you:'''
<pre>
// These hold some numbers
oneVar = 1
twoVar = 2

var cheesesTypes = { cheddar : 1, stilton : 2, emmental : 3, }

function doThingsHere ()
{
return 1;
}

//If one equals two do some other things and make sure that if the the click handler is setup correctly.
if( one === two ) {
var cheese = "cheddar";
oneVar = doThingsHere();

$(this).click(function(event){ alert("Hello"); });
}

document.getElementById("little-mouse").addEventListener("focusout", function(){
alert("bye")
});

if (oneVar)
{
noThingHere();
} else { doThingHere(); }
</pre>

Note:
* Variables should be marked with "var"
* Semicolons should be used
* Keep as close to 80 cols as possible
* Use 2 space per indentation
* Open curly braces after parenthesis for functions and close on a new line
* Use camelCase for function names and variable names

Make use of running your Javascript through jshint we have a .jshint configuration file in that js directory (toastergui/static/js)

e.g. install jshint and add to your current PATH, then run:
<code>
$ npm install jshint; export PATH=$PATH:$PWD/node_modules/.bin/
$ jshint ./toastergui/static/js/base.js
</code>

=== HTML ===

'''Yes please:'''
<pre>
<div id="something-area">
This is some text
</div>

<div>
This is some text
</div>

</pre>

'''No thank you:'''
<pre>
<div id="somethingarea">
This is some text</div>
<div id="somethingarea">
This is
some text


</div>
</pre>

Note:
* 2 space indentation
* Lower case, ids hyphenated when multiple words
* No duplicate ids

* Run your HTML through a [http://validator.w3.org/#validate_by_input HTML validator] available for [http://validator.w3.org/source/ local install]. The w3c validator it's self doesn't currently validate html5, it uses as a back end [https://validator.github.io/validator/ Nu Html Checker] which can be installed as a standalone service, full instructions in the readme.

Quick install instructions:
<pre>
$ mkdir html5-validator && cd html5-validator
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$ git clone https://github.com/validator/validator.git
$ python build/build.py all
$ python build/build.py all
</pre>

HTML can be indented quickly using tidy, for example:
<pre>
tidy -xml --indent auto --indent-spaces 2 --quiet yes -w -1 --show-body-only yes ./index.html
</pre>

=== Python ===

Lenient [https://www.python.org/dev/peps/pep-0008 pep8]
Ignoring most of the whitespace around character issues (E124,E203,E201,E265,E303,E302,E231) see toaster/.pep8 and [http://pep8.readthedocs.org/en/latest/intro.html#error-codes error code list]

Fix all issues identified by running code through pep8. We have a fairly lenient config file (toaster/.pep8).

e.g.
<code>
$ pep8 ./toastergui/urls.py
</code>

Run code through pylint and fix identified issues - Some can be reasonably ignored such as doc strings for every function or star-args. No pylintrc config provided here as most issues identified are highly contextual and should be ignored on a case by case basis.

<code>
$ pylint --load-plugins pylint_django toastergui/tests.py
</code>

== Working with design ==

Yes, the Yocto Project is one of those lucky projects with designers around to help in UI matters. We have a document explaining how to work with the design contributors: [[File:Working_with_design.pdf]]

== Debugging Toaster ==

Toaster can be quite complex given that it includes a wide range of technologies and languages from bitbake, events, python, Django, Sqlite, Javescript, CCS, HTML, and more. Here are hints and tips for debugging all of these aspects of Toaster.

=== General debugging tips ===

* The logger is your best friend
* When possible, build from command line in the Toaster environment to see log messages and errors directly
* Use "quilt-native" as your quick sanity build target
* For python code you can use pudb (https://wiki.yoctoproject.org/wiki/TipsAndTricks/DebuggingBitbakeInPudb)
* For bbclass code (toaster.bbclass) find or set up a python section to use pudb (see above link)
* Brute force find is your friend for mysterious error messages and tracking all places a variable is used
** find bitbake/lib/toaster -exec grep -l "SEARCH_TEXT" {} \; 2> /dev/null
* Look in https://wiki.yoctoproject.org/wiki/TipsAndTricks

=== How to debug a ... ===

* A stalled build:
** Cancel the build and look at one of the below logs to find the exception. The error message should give a clue on the user resolution or indicate a bug to file
** Failures in git and shell calls are (as of YP-2.4 Rocko) captured as visible build errors in the GUI
* Python file of a build management script (localhostbecontroller.py,...):
** Use log statements, and watch "build/toaster_runbuilds.log" (where you started Toaster)
* Python file of a build runtime scripts:
** Use log statements, and watch "build-toaster-x/toaster_ui.log"
* Python file of a Toaster management script (projects, tables):
** Use log statements, and watch "build/toaster_web.log" (where you started Toaster)
* Template file
** Usually the error will appear in the browser via Django, with a trace
** Add random display text to insure the page your editing is the one that shows
** Display internal values via page template to see what gets passed
** Add internal values into the context and add to the page template
** Use the Firefox/Chrome developer "inspector" to examine the DOM
* Javascript file
** Use the Firefox/Chrome developer "inspector" to do inline debugging
** Watch the Firefox/Chrome developer "console" for error messages
* The Toaster Database
** Use an application like "sqlitebrowser" to examine the tables and records
** Use a Python script to find and dump desired database records (see http://events.linuxfoundation.org/sites/events/files/slides/BitbakeAnalytics_ELC_Portland.pdf)

=== Specific error scenarios ... ===

* The build progress gets stuck at "Loading..."
** You have a syntax error, either in 'localhostbecontroller.py', the helper classes in 'models.py', or in the related javascript. Look at the tail of "toaster_runbuilds.log" file for pythons errors, and in your browser's "Console" for javascript errors.
* The message "Sorry, An error has occurred loading this page"
** The included javascript has a syntax error. The HTML page being rendered has local JS code at the top that caches these errors and displays the observed message (in order to be less ugly to the customer). That code also tries to add the error message to the browser console but that does not always work, and you are left with no clue
** You can start bisecting the recent edits you made to the respective JS files to locate the problem change
** You can also add a breakpoint in the mentioned catch code to see if you can see the error directly

=== How to update from a fix in a ... ===

* Template file: save your text edit and refresh page
* Javascript file: save your text edit and refresh page (or leave page and come back)
* Builder scripts: save your text edit and restart Toaster ("localhostbecontroller.py", "buildinfohelper.py", "toasterui.py", "bbcontroller.py",...)
* Views and URLs: save your text edit and refresh page
* Model class members: save your text edit and restart Toaster. You may also need a migration file.
* Model method: save your text edit and refresh page, else restart Toaster
* Fixture file (settings.xml, poky.xml, custom.xml): delete Toaster database and restart fresh

=== Where to debug ... ===

* Toaster start and stop, database init
** bin/toaster
* Project Defaults (Default machine, distro, layers, ...)
** bldcontrol/management/commands/checksettings.py
** orm/fixtures/*.xml
* Layer Index content (Available machines, distros, layers, ...)
** orm/management/commands/lsupdates.py
* Build cloning
** bldcontrol/localhostbecontroller.py
* Build start
** bldcontrol/localhostbecontroller.py
* URL mappings
** toastergui/urls.py
* Views
** toastergui/views.py
** toastergui/templates/*
* Toaster Tables (Layers, Distros, Machines, ...)
** toastergui/tables.py
** toastergui/static/js/libtoaster.js
* Events
** meta/classes/toaster.bbclass ->
** lib/bb/ui/toasterui.py ->
** lib/bb/ui/buildinfohelper.py ->
** orm/models.py et. al.

File:YP Presentations Summary.xlsx

2024-12-04T04:16:45Z

David Reyna: David Reyna uploaded a new version of File:YP Presentations Summary.xlsx

== Summary ==
Spread sheet of the YP Presentation archive, under active updates.

File:YP Presentations Summary.xlsx

2024-12-03T06:12:17Z

David Reyna: David Reyna uploaded a new version of File:YP Presentations Summary.xlsx

== Summary ==
Spread sheet of the YP Presentation archive, under active updates.

YoctoProject Summit yps2024.12

2024-11-26T08:12:29Z

David Reyna: Make DO links in 'big' type

= The Yocto Project Summit 2024.12 =
The Yocto Project Summit, yps2024.12, is a virtual conference from December 3rd to 5th, 2024 to gather members of the Yocto Project community together to discuss ideas and present information related to The Yocto Project.

== Organizing Committee ==
The organizing committee consists of (alphabetical by first name):
* David Reyna
* Josef Holzmayr-Khosh Amoz (YP Community Manager)
* Megan Knight (Advocacy Manager)
* Philip Balister

== Registration ==

Registration Link: https://cvent.me/V5kARQ

== Platforms ==
In terms of platforms we're using:
* pretalx for the CfP and schedule
* the LF's cvent for registration
* Zoom for the conference video
* irc for the conference chat (channel #yocto-summit-2024.12 on Libera.Chat)
* Digital Ocean for the hands-on classes

== Hands-on Servers Information (Pre-event Preparation) ==
# If you already have a Digital Ocean account, skip the next 2 steps.
# To create a Digital Ocean account, please sign up @ <big>[https://do.co/yoctoproject DigitalOcean]</big> to receive a $200 credit for new users.
# Confirm your email and add a payment method to satisfy abuse requirements - you won't be charged unless you leave the training machine running long past the summit
# Visit <big>https://cloud.digitalocean.com/account/api/tokens</big> and generate a new token named '''YPSummit''', and copy the string that pops up.
# Please complete this <big>[https://forms.gle/rPtZ84gbSmKJo2cN9 FORM]</big> so your API key can be added to the provisioning scripts. It's unlikely we will be able to offer additional credit to existing account holders this year. Leave the API key blank if you'd like Yocto Project to cover training VM expenses.

After the form has been filled by all participants we will provision machines. When the machines are ready logins will be "ssh ilab01@AXAXAXAX-summit.yocto.io" where AXAXAXAX are the first 8 characters of your confirmation string. The password is the full 11 character confirmation string.

Once you have a verified working server you may delete your api key at <big>https://cloud.digitalocean.com/account/api/tokens</big>. Otherwise all resources will be removed after the training. Any retained servers will be billed to you after the promotional credit expires in 60 days.


Lab files will be available for download once the event begins.

== For more information ==
* https://www.yoctoproject.org/event/2024-12/
* https://pretalx.com/yocto-project-summit-2024-12/

== Slide template for presenters ==
* https://docs.google.com/presentation/d/1ttXgB8Z1gasL7ItCYZ-uXT_FmfnIgPoMK_JtjvBrj1g
** keep the first and last slides
** edit the first slide to include your name, presentation title, and (optionally) your company
** the rest of the slides are examples and can be duplicated, edited, and/or removed
** please upload your finished slides to your specific event in pretalx
** Pretalx has an upload limit of 10Mb. Please for example compress your PDF files when you "Save-as", or use a free service like "https://www.adobe.com/acrobat/online/compress-pdf.html"

File:YP Presentations Summary.xlsx

2024-11-22T03:35:33Z

David Reyna: David Reyna uploaded a new version of File:YP Presentations Summary.xlsx

== Summary ==
Spread sheet of the YP Presentation archive, under active updates.

File:YP Presentations Summary.xlsx

2024-11-21T09:16:52Z

David Reyna: Spread sheet of the YP Presentation archive, under active updates.

== Summary ==
Spread sheet of the YP Presentation archive, under active updates.

YoctoProject Summit yps2024.12

2024-11-20T22:20:54Z

David Reyna: Created page with "= The Yocto Project Summit 2024.12 = The Yocto Project Summit, yps2024.12, is a virtual conference from December 3rd to 5th, 2024 to gather members of the Yocto Project community together to discuss ideas and present information related to The Yocto Project. == Organizing Committee == The organizing committee consists of (alphabetical by first name): * David Reyna * Josef Holzmayr-Khosh Amoz (YP Community Manager) * Megan Knight (Advocacy Manager) * Philip Balister ==..."

= The Yocto Project Summit 2024.12 =
The Yocto Project Summit, yps2024.12, is a virtual conference from December 3rd to 5th, 2024 to gather members of the Yocto Project community together to discuss ideas and present information related to The Yocto Project.

== Organizing Committee ==
The organizing committee consists of (alphabetical by first name):
* David Reyna
* Josef Holzmayr-Khosh Amoz (YP Community Manager)
* Megan Knight (Advocacy Manager)
* Philip Balister

== Registration ==

Registration Link: https://cvent.me/V5kARQ

== Platforms ==
In terms of platforms we're using:
* pretalx for the CfP and schedule
* the LF's cvent for registration
* Zoom for the conference video
* irc for the conference chat (channel #yocto-summit-2024.12 on Libera.Chat)
* Digital Ocean for the hands-on classes

== Hands-on Servers Information (Pre-event Preparation) ==
# If you already have a Digital Ocean account, skip the next 2 steps
# To create a Digital Ocean account, please sign up @ [https://do.co/yoctoproject DigitalOcean]
# Confirm your email and add a payment method to satisfy abuse requirements - you won't be charged unless you leave the training machine running long past the summit
# Visit https://cloud.digitalocean.com/account/api/tokens and generate a new token named '''YPSummit''', and copy the string that pops up.
# Please complete this [https://forms.gle/53NFyCRaEpmRWuaa8 form] so your API key can be added to the provisioning scripts and we can top up your account credit.

After the form has been filled by all participants we will provision machines. When the machines are ready logins will be "ssh ilab01@AXAXAXA-summit.yocto.io" where AXAXAXA are the first 7 characters of your confirmation string. The password is the full 11 character confirmation string.

Once you have a verified working server you may delete your api key at https://cloud.digitalocean.com/account/api/tokens. Otherwise all resources will be removed after the training. Any retained servers will be billed to you after the promotional credit expires in 60 days.


Lab files will be available for download once the event begins.

== For more information ==
* https://www.yoctoproject.org/event/2024-12/
* https://pretalx.com/yocto-project-summit-2024-12/

== Slide template for presenters ==
* https://docs.google.com/presentation/d/1ttXgB8Z1gasL7ItCYZ-uXT_FmfnIgPoMK_JtjvBrj1g
** keep the first and last slides
** edit the first slide to include your name, presentation title, and (optionally) your company
** the rest of the slides are examples and can be duplicated, edited, and/or removed
** please upload your finished slides to your specific event in pretalx
** Pretalx has an upload limit of 10Mb. Please for example compress your PDF files when you "Save-as", or use a free service like "https://www.adobe.com/acrobat/online/compress-pdf.html"

Yocto Project EOSS 2024

2024-05-09T16:47:08Z

David Reyna:

== Yocto Project® EOSS Mini-Summit 2024 ==

The Yocto Project ran a mini-Summit at EOSS in 2024.

EOSS home page: https://www.eosconference.com/san-diego/agenda/

Mini-Summit: https://www.yoctoproject.org/event/yocto-project-at-embedded-open-source-summit/

* '''DATE''': April 16-18, 2024

* '''LOCATION''': Seattle, Washington

== Slides ==

Slides are here:

== Example Wiki page with slides attached ==

https://wiki.yoctoproject.org/wiki/DevDay_Portland_2018

[[Category:SRTool]]
This page summarizes the Security Response Tool (SRTool) development process. We hope this will help you start contributing to the project. The SRTool is based on the Yocto Project Toaster codebase (Django,Python,Javascript), so many of the process and debugging techniques apply.
__FORCETOC__

== Published content: General CVE management issues and the SRTool ==
* The SRTool paper for the Embedded Linux Conference, Edinburg 2018:
** https://events.linuxfoundation.org/wp-content/uploads/2017/12/Keeping-Up-With-The-Joneses-CVEs-David-Reyna-Wind-River-Systems.pdf�
* Wind River Blog about CVEs and the SRTool
** http://blogs.windriver.com/wind_river_blog/2018/11/security-response-management-risk-cost-and-best-practices-in-an-imperfect-world.html�

 
== Set up the local repository and SRTool instance ==

1) Host requirements

The SRTool should in principle work on any host that supports Yocto Project. It requires "Python3", "Django >= 1.11", and "Sqlite3". A SQL GUI tool like 'sqlitebrowser' is recommended.

The required host package installation instructions are the same as Toaster, and the instructions can be found here: [http://www.yoctoproject.org/docs/latest/toaster-manual/toaster-manual.html#toaster-manual-start Toaster documentation]

It was specifically developed and tested with:
{| class="wikitable" style="text-align: left;"
! Item
! Host 1
! Host 2
! Host 3
|-
|Host
|Ubuntu 16.04
|Ubuntu 16.04
|
|-
|Python3
|3.5.2
|3.5.2
|
|-
|Django
|1.11 LTS
|2.2
|
|-
|Sqlite3
|3.11
|3.11
|
|-
|}

2) Cloning the SRTool

<code> $ git clone git://git.yoctoproject.org/srtool && cd srtool </code>

Here is the command if your SSH key is registered:

<code> $ git clone ssh://git@push.yoctoproject.org/srtool && cd srtool </code>

3) Starting the SRTool

Option #1: Restrict to the local browser:
<code> $ ./bin/srt start webport=localhost:9000 </code>

Option #2: Enable remote browsers:
<code> $ ./bin/srt start webport=0.0.0.0:9000 </code>

NOTE: The first time you run the SRTool, there will be a delay (30 to 60 minutes) as the default CVE repositories (NIST, Mitre, ...) are scanned, scored, and loaded into the database. After that, the updates will be incremental.

4) Create a superuser

You will need to create at least one super user account to promote users to higher permission levels:

<code> $ ./bin/srt manage createsuperuser</code>

5) Open browser to <IPADDR>:9000

This will bring up the SRTool page.

6) Read the "Guided Tour" 

Click on the "Guided Tour" button on the home page to see the on-line tool and user information.

7) Create your guest account

Click on the "Login" button in the top bar, select "Request Account", fill in the information, Click "Sign Up", and then log in.

8) Promote your account to Admin level

Login as the superuser, click "Management > Manage Users", click the edit icon for your guest account row, change the "Group" to "Admin", and click "Submit Changes".

You can now log out of the superuser account and into your own account.

 

== Sanity Test Bring-up ==

You can use these steps to start a minimal instance of the SRTool to help validate that the host and the code base are working on your system. This will for example set the environment variable "SRTDBG_MINIMAL_DB=1" which will only fetch a few representative records from each data source.

<code>
$ git pull
$ # Copy the dev tools to the base directory
$ cp bin/dev_tools/* .
$ # Source the debug environment
$ . ./srt_env.sh debug
$ # Start the system from scratch
$ ./recreate.sh
$ # Run the sanity test
$ ./bin/common/srtool_sanity_test.py -i
$
</code>

Sample output:

<code>
$ ./bin/common/srtool_sanity_test.py -i
Uname = #35~16.04.1-Ubuntu SMP Thu Jan 25 10:13:43 UTC 2018 x86_64
Django = (1, 11, 8, 'final', 0)
Python3 = Python 3.5.2
Sqlite3 = 3.11.0 2016-02-15
Server PID = 10234
Server Port = [10234](/opt/srtool/lib/manage.py runserver --noreload 0.0.0.0:9123)
CVEs = 64
Users = 5
Data source = 22
$
</code>

To re-start as the full instance, do this:

<code>
$ # Source the standard environment
$ . ./srt_env.sh
$ # Start the system from scratch
$ ./recreate.sh
$
</code>

 
== First Time Bring-up ==

* When you first start, you will have the CVEs from 2015 to 2018
* You will not have any Vulnerabilities, Investigations, or Defects until you (a) create them, and/or (b) integrate your defect system or other sustaining data.
* All CVEs will by default get the status "Historical".
** The CVEs that were created in the previous 30 days will instead get the status "New"
** This will allow you to immediately test the CVE triage features
** The status settings are normally preset by your defect system or other sustaining data
* You can create a Vulnerability by:
** Selecting a CVE
** Click on "Create Vulnerability"
** Follow the new link to the new Vulnerability
* You can create an Investigation by:
** Open a Vulnerability record
** Click on "Add Product"
** Select one or more Products, and click "Submit"
** Observe that the products are added together with new respective Investigations
** Follow the new link to the new Investigations
* You can create a Defect by:
** Open an Investigation record
** Click on "Create Defect"
** With the sample defect integration script, a simulated defect will be created and attached to the investigation
 

== SRTool Primary Goals and Workflows ==

* Goals
** A common system to track and share security issues, combining community CVE's
** A simple yet flexible interface for reporting and exploring the security issues
** A place to upload and share attachments, including patches, fixes, emails, and documents
** The ability to generate accurate and up-to-date reports and exports
** A modular design for easy extension and adoption
 
* Primary Workflows
** Guide and manage the incoming CVE triage process
** Attach CVEs to specific company products and defects
** Automated updates from upstream CVEs and internal defect system
** Ability to generate reports
** Simple customization for organizations
** Correlate CVEs with products and release (IN PROGRESS)
** Correlate CVEs with specific customer builds (IN PROGRESS)

 

== SRTool CVE Lifecycle ==

* CVE are imported
** New CVEs are imported and triaged from upstream (NIST/MITRE, 1000+/month)
** Specific CVEs are requested by customers
* CVEs are managed
** CVEs are investigated
** Vulnerable CVEs are assigned to products via defects
** SRTool manager tracks the progress of open CVEs
** SRTool manager produces reports as required by customers, field, and management
* CVEs are resolved
** CVEs are either fixed or declared not vulnerable
** Releases are updated with the CVE fixes
** Resolved CVEs are reported to customers, public website

 
== Testing the Alternate Sample Organization 'ACME' ==

* An example alternate organization (master application) called "acme" is included.
* This code shows all the ways the SRTool can be easily customized to the needs of your organization
* To enable it and try it out, do the following:

<code>
$ # Include the quick development tools
$ cp bin/dev_tools/* .
$ # Reset 'acme' as the master app/organization
$ ./stop.sh
$ ./master_app.sh acme
$ ./start.sh
</code>

* When you browse the SRTool pages, will now observe several differences
** In the top bar the text "[ACME]" will appear in several places. This shows how files like "lib/acme/templates/base.html" can take precedence over the default template files in "lib/srtgui".
** In the top left corner the "ACME" logo will appear. This is from "SRTOOL_LOCAL_LOGO" being defined in "datasource.json".
** If you click on the ACME logo you will be taken to an ACME hello page. This shows that "lib/acme/urls.py" has extended the URL map, plus how you can implement basic HTML pages in ACME's "lib/acme/views.py" and "lib/acme/templates/acme_hello.html".
** In the ACME hello page, there are links to custom table pages for Products and Defects. This shows how to implement custom Toaster Tables via "lib/acme/urls.py" and "lib/acme/tables.py"
** In the Products page, when you click "Export" a new report type "ACME Products Table" is appended. This shows how to extend and customize existing or new reports.

* To restore the default "yp" master application, do the following:
<code>
$ ./stop.sh
$ ./master_app.sh yp
$ ./start.sh
</code>

== Adapting SRTool to your Organization ==

* You can use the sample alternate organization "acme" as a guide to adapt SRTool to your organization
* In this example, we will instantiate a new organization called "yoyodyne"
* You should be able to place all of your local content into these two directories:
** "./bin/yoyodyne": this is your master datasource directory
** "./lib/yoyodyne": this is your master Django App directory
* If you find you need to make changes in the core SRTool code, we recommend (in preferential order):
** See if you can make it a generic feature so that it can become part of the mainline codebase
** See if you can make an easy hook for the mainline code, so that your custom content can still reside in your directories
** Talk to us on how to easily have managed custom patches to the mainline code (we do this in Toaster)

1) Instantiate your Organization

You can create a starting customization of the SRTool for your organization with these simple steps.
* In this example, we will use the name "yoyo" as a shorthand for a mythical "Yoyodyne Corporation".

<code>
$ cd /path_to/srtool_dir
$ copy bin/dev_tools/* .
$ ./stop.sh
$ ./master_app.sh create yoyo
$ ./start.sh
</code>

* This script will perform the following actions:
** Copy the sample 'bin/acme' directory to 'bin/yoyo'
** Copy the sample 'lib/acme' directory to 'lib/yoyo'
** Rename the respective Acme/ACME/acme labels and filenames to Yoyo/YOYO/yoyo
** Set 'yoyo' as the new main application
* You now have access to all the preset custom features described above for the ACME sample organization, now for Yoyodyne.

2) Add your defect system integration

You will need to implement a defect integration backend script to scan your defect system and instantiate the CVE defects in SRTool database, and then keep them up to date.

* An example implementation template for Jira is provided here.
<code>
bin/common/srtool_jira_template.py
</code>
* You can use this to guide your integration to Jira or to other defect systems.
* To see how to extend this template to your organization and yet maintain compatibility with upstream, see this section: [https://wiki.yoctoproject.org/wiki/SRTool_Developer_Page#Extending_and_customizing_common.2Ftemplate_code_files Extending and customizing common/template code files]
* Please contact the SRTool developers for assistance in your defect system integration effort.

The main features of this example code, in addition to the Jira system access, is:

* The "--init" command
** This command that will initialize the SRTool database with your SRT related defects. Specifically, this command can used to bootstrap the SRTool database for your company's data
** It will create an "Investigation" for each defect, and attach it to the respective product
** It will create a "Vulnerability" to wrap the respective "Investigation"
** If should find the respective CVE record if it exists, and attach that to the created "Vulnerability"
** If a "Vulnerability" record already exists for the defect's CVE, then this script should attach the new "Investigation" to that "Vulnerability" record.
* The "--update" command
** This will scan your CVE-related defects
** If the matching defect record in the SRTool database does not exist, then treat it like the "--init" command.
** It will update the SRTool defect record with the current defect system state
** If the defect has changed priority (or any other important state information), it will create a "Notification" event to the respective manager so that they can know that they need to act on the change.
* The example code assumes that the CVE names are available via a text-based search, for example when the respective CVE name is in the defect title or comments. If your system uses a different marker, then you will need to code for that.

 

== High Level Development Plan ==

{| class="wikitable" style="text-align: left;"
! Release
! Features
! Status/date
|-
|0.80
|Multi-host testing, Django user model, Django 2.x support
|Released (December 2018)
|-
|0.90
|Mapping of CVE-CPE > Packages > Recipes > Product/Release CPEs, Add Releases to Products, modular report definitions
|Under development (January 2019)
|-
|1.00
|General Release
|Under development (February 2019)
|-
|1.10
|Improved CPE/package mapping hueristics, advanced reports
|Under development (March+ 2019)
|-
|}

 
== SRTool Bugzilla Page ==

Here is the link to the SRTool Bugzilla Page: [https://bugzilla.yoctoproject.org/buglist.cgi?list_id=609538&bug_status=__all__&query_format=specific&order=relevance%20desc&product=Security%20Response%20Tool SRTool Bugzilla Page]

 
== SRTool CGIT Page ==

Here is the link to the CGIT Page: [http://git.yoctoproject.org/cgit/cgit.cgi/srtool SRTool CGIT Page]

 
== SRTool User Page ==

Here is the link to the User's Documentation page: [https://wiki.yoctoproject.org/wiki/SRTool_User_Page SRTool UserPage]

 
== SRTool Developer Page ==

Here is the link to the Developer's page: [https://wiki.yoctoproject.org/wiki/SRTool_Developer_Page SRTool Developer Page]

YP Summit Europe 2020

2020-10-08T22:11:03Z

David Reyna: /* Slides */

This page contains slides for the talks in the Yocto Project Virtual Summit at ELCE 2020

Starting at 9:00am, 29-30 October 2020.

YP Summit home page is here: [https://wiki.yoctoproject.org/wiki/Yocto_Project_Summit_2020 https://wiki.yoctoproject.org/wiki/Yocto_Project_Summit_2020]

== Slides ==

* Presenter Slides Thursday October 29

* Presenter/Class Slides Friday October 30

* Class Code

* Slides template

[https://wiki.yoctoproject.org/wiki/File:Yocto_Project_DevDay_EU2020_Template.pptx Yocto_Project_DevDay_EU2020_Template.pptx]
[https://wiki.yoctoproject.org/wiki/File:Yocto_Project_DevDay_EU2020_Template.odp Yocto_Project_DevDay_EU2020_Template.odp]

File:Yocto Project DevDay EU2020 Template.odp

2020-10-08T22:09:58Z

David Reyna: Template slide deck for YP Summit Europe 2020 (ODP).

Template slide deck for YP Summit Europe 2020 (ODP).

YP Summit Europe 2020

2020-10-08T16:10:16Z

David Reyna: