Yocto Project - User contributions [en]

Building your own recipes from first principles

2022-07-20T12:35:28Z

Alen Sunnny Mathew: /* Overview */

= '''Yocto Project Quick Build''' =
== Kirkstone (4.0.2)==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [https://docs.yoctoproject.org/ref-manual/index.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the [https://docs.yoctoproject.org/ Yocto Quickstart]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files Locating Yocto Project Source Files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy Network Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory], which is build in this case and is located in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory Source Directory ]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache Shared State Cache] mirrors and enabling [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence Hash Equivalence ]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory ]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Start the Build==

Continue with the following command to build an OS image for the target, which is core-image-sato in this example:

$ bitbake core-image-sato
For information on using the bitbake command, see the [https://docs.yoctoproject.org/overview-manual/concepts.html#bitbake BitBake] section in the Yocto Project Overview and Concepts Manual, or see The [https://docs.yoctoproject.org/bitbake/2.0/bitbake-user-manual/bitbake-user-manual-intro.html#the-bitbake-command BitBake Command] in the BitBake User Manual.

== Simulate Your Image Using QEMU==
once the image is build , you can start QEMU with the following commands.
$ runqemu qemux86-64

[https://docs.yoctoproject.org/dev-manual/qemu.html#using-the-quick-emulator-qemu The Quick EMUlator (QEMU)] Yocto Project Development Tasks Manual will gives you more understanding about running QEMU.

== Exit QEMU:==
Exit QEMU by either clicking on the shutdown icon or by typing Ctrl-C in the QEMU transcript window from which you evoked QEMU.

= Build an example project on the host for testing (optional) =

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T12:33:43Z

Alen Sunnny Mathew: /* Overview */

= '''Yocto Project Quick Build''' =
== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [https://docs.yoctoproject.org/ref-manual/index.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the [https://docs.yoctoproject.org/ Yocto Quickstart]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files Locating Yocto Project Source Files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy Network Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory], which is build in this case and is located in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory Source Directory ]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache Shared State Cache] mirrors and enabling [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence Hash Equivalence ]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory ]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Start the Build==

Continue with the following command to build an OS image for the target, which is core-image-sato in this example:

$ bitbake core-image-sato
For information on using the bitbake command, see the [https://docs.yoctoproject.org/overview-manual/concepts.html#bitbake BitBake] section in the Yocto Project Overview and Concepts Manual, or see The [https://docs.yoctoproject.org/bitbake/2.0/bitbake-user-manual/bitbake-user-manual-intro.html#the-bitbake-command BitBake Command] in the BitBake User Manual.

== Simulate Your Image Using QEMU==
once the image is build , you can start QEMU with the following commands.
$ runqemu qemux86-64

[https://docs.yoctoproject.org/dev-manual/qemu.html#using-the-quick-emulator-qemu The Quick EMUlator (QEMU)] Yocto Project Development Tasks Manual will gives you more understanding about running QEMU.

== Exit QEMU:==
Exit QEMU by either clicking on the shutdown icon or by typing Ctrl-C in the QEMU transcript window from which you evoked QEMU.

= Build an example project on the host for testing (optional) =

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T12:31:42Z

Alen Sunnny Mathew: /* Build an example project on the host for testing (optional) */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [https://docs.yoctoproject.org/ref-manual/index.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the [https://docs.yoctoproject.org/ Yocto Quickstart]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files Locating Yocto Project Source Files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy Network Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory], which is build in this case and is located in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory Source Directory ]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache Shared State Cache] mirrors and enabling [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence Hash Equivalence ]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory ]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Start the Build==

Continue with the following command to build an OS image for the target, which is core-image-sato in this example:

$ bitbake core-image-sato
For information on using the bitbake command, see the [https://docs.yoctoproject.org/overview-manual/concepts.html#bitbake BitBake] section in the Yocto Project Overview and Concepts Manual, or see The [https://docs.yoctoproject.org/bitbake/2.0/bitbake-user-manual/bitbake-user-manual-intro.html#the-bitbake-command BitBake Command] in the BitBake User Manual.

== Simulate Your Image Using QEMU==
once the image is build , you can start QEMU with the following commands.
$ runqemu qemux86-64

[https://docs.yoctoproject.org/dev-manual/qemu.html#using-the-quick-emulator-qemu The Quick EMUlator (QEMU)] Yocto Project Development Tasks Manual will gives you more understanding about running QEMU.

== Exit QEMU:==
Exit QEMU by either clicking on the shutdown icon or by typing Ctrl-C in the QEMU transcript window from which you evoked QEMU.

= Build an example project on the host for testing (optional) =

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:53:44Z

Alen Sunnny Mathew: /* Exit QEMU: */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [https://docs.yoctoproject.org/ref-manual/index.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the [https://docs.yoctoproject.org/ Yocto Quickstart]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files Locating Yocto Project Source Files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy Network Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory], which is build in this case and is located in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory Source Directory ]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache Shared State Cache] mirrors and enabling [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence Hash Equivalence ]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory ]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Start the Build==

Continue with the following command to build an OS image for the target, which is core-image-sato in this example:

$ bitbake core-image-sato
For information on using the bitbake command, see the [https://docs.yoctoproject.org/overview-manual/concepts.html#bitbake BitBake] section in the Yocto Project Overview and Concepts Manual, or see The [https://docs.yoctoproject.org/bitbake/2.0/bitbake-user-manual/bitbake-user-manual-intro.html#the-bitbake-command BitBake Command] in the BitBake User Manual.

== Simulate Your Image Using QEMU==
once the image is build , you can start QEMU with the following commands.
$ runqemu qemux86-64

[https://docs.yoctoproject.org/dev-manual/qemu.html#using-the-quick-emulator-qemu The Quick EMUlator (QEMU)] Yocto Project Development Tasks Manual will gives you more understanding about running QEMU.

== Exit QEMU:==
Exit QEMU by either clicking on the shutdown icon or by typing Ctrl-C in the QEMU transcript window from which you evoked QEMU.

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:52:37Z

Alen Sunnny Mathew: /* Simulate Your Image Using QEMU */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [https://docs.yoctoproject.org/ref-manual/index.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the [https://docs.yoctoproject.org/ Yocto Quickstart]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files Locating Yocto Project Source Files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy Network Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory], which is build in this case and is located in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory Source Directory ]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache Shared State Cache] mirrors and enabling [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence Hash Equivalence ]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory ]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Start the Build==

Continue with the following command to build an OS image for the target, which is core-image-sato in this example:

$ bitbake core-image-sato
For information on using the bitbake command, see the [https://docs.yoctoproject.org/overview-manual/concepts.html#bitbake BitBake] section in the Yocto Project Overview and Concepts Manual, or see The [https://docs.yoctoproject.org/bitbake/2.0/bitbake-user-manual/bitbake-user-manual-intro.html#the-bitbake-command BitBake Command] in the BitBake User Manual.

== Simulate Your Image Using QEMU==
once the image is build , you can start QEMU with the following commands.
$ runqemu qemux86-64

[https://docs.yoctoproject.org/dev-manual/qemu.html#using-the-quick-emulator-qemu The Quick EMUlator (QEMU)] Yocto Project Development Tasks Manual will gives you more understanding about running QEMU.

= Exit QEMU: =
Exit QEMU by either clicking on the shutdown icon or by typing Ctrl-C in the QEMU transcript window from which you evoked QEMU.

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:51:20Z

Alen Sunnny Mathew: /* Simulate Your Image Using QEMU */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [https://docs.yoctoproject.org/ref-manual/index.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the [https://docs.yoctoproject.org/ Yocto Quickstart]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files Locating Yocto Project Source Files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy Network Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory], which is build in this case and is located in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory Source Directory ]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache Shared State Cache] mirrors and enabling [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence Hash Equivalence ]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory ]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Start the Build==

Continue with the following command to build an OS image for the target, which is core-image-sato in this example:

$ bitbake core-image-sato
For information on using the bitbake command, see the [https://docs.yoctoproject.org/overview-manual/concepts.html#bitbake BitBake] section in the Yocto Project Overview and Concepts Manual, or see The [https://docs.yoctoproject.org/bitbake/2.0/bitbake-user-manual/bitbake-user-manual-intro.html#the-bitbake-command BitBake Command] in the BitBake User Manual.

== Simulate Your Image Using QEMU==
once the image is build , you can start QEMU with the following commands.
$ runqemu qemux86-64

[https://docs.yoctoproject.org/dev-manual/qemu.html#using-the-quick-emulator-qemu The Quick EMUlator (QEMU)] Yocto Project Development Tasks Manual will gives you more understanding about running QEMU.

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:46:34Z

Alen Sunnny Mathew: /* Start the Build */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [https://docs.yoctoproject.org/ref-manual/index.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the [https://docs.yoctoproject.org/ Yocto Quickstart]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files Locating Yocto Project Source Files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy Network Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory], which is build in this case and is located in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory Source Directory ]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache Shared State Cache] mirrors and enabling [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence Hash Equivalence ]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory ]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Start the Build==

Continue with the following command to build an OS image for the target, which is core-image-sato in this example:

$ bitbake core-image-sato
For information on using the bitbake command, see the [https://docs.yoctoproject.org/overview-manual/concepts.html#bitbake BitBake] section in the Yocto Project Overview and Concepts Manual, or see The [https://docs.yoctoproject.org/bitbake/2.0/bitbake-user-manual/bitbake-user-manual-intro.html#the-bitbake-command BitBake Command] in the BitBake User Manual.

== Simulate Your Image Using QEMU==

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:40:12Z

Alen Sunnny Mathew: /* Build a baseline image */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [https://docs.yoctoproject.org/ref-manual/index.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the [https://docs.yoctoproject.org/ Yocto Quickstart]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files Locating Yocto Project Source Files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy Network Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory], which is build in this case and is located in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory Source Directory ]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache Shared State Cache] mirrors and enabling [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence Hash Equivalence ]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory ]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Start the Build==

Continue with the following command to build an OS image for the target, which is core-image-sato in this example:

$ bitbake core-image-sato
For information on using the bitbake command, see the [https://docs.yoctoproject.org/overview-manual/concepts.html#bitbake BitBake] section in the Yocto Project Overview and Concepts Manual, or see The [https://docs.yoctoproject.org/bitbake/2.0/bitbake-user-manual/bitbake-user-manual-intro.html#the-bitbake-command BitBake Command] in the BitBake User Manual.

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:36:19Z

Alen Sunnny Mathew: /* Use Git to Clone Poky */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [https://docs.yoctoproject.org/ref-manual/index.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the [https://docs.yoctoproject.org/ Yocto Quickstart]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files Locating Yocto Project Source Files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy Network Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory], which is build in this case and is located in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory Source Directory ]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache Shared State Cache] mirrors and enabling [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence Hash Equivalence ]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory ]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:35:33Z

Alen Sunnny Mathew: /* Building Your Image */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [https://docs.yoctoproject.org/ref-manual/index.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the [https://docs.yoctoproject.org/ Yocto Quickstart]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy Network Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory], which is build in this case and is located in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory Source Directory ]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache Shared State Cache] mirrors and enabling [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence Hash Equivalence ]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory ]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:33:30Z

Alen Sunnny Mathew: /* Building Your Image */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [https://docs.yoctoproject.org/ref-manual/index.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the [https://docs.yoctoproject.org/ Yocto Quickstart]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy Network Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory Build Directory], which is build in this case and is located in the [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory Source Directory ]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using Shared State Cache [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache]mirrors and enabling Hash Equivalence [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:30:43Z

Alen Sunnny Mathew: /* Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [https://docs.yoctoproject.org/ref-manual/index.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the [https://docs.yoctoproject.org/ Yocto Quickstart]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the oe-init-build-env [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory], which is build in this case and is located in the Source Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using Shared State Cache [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache]mirrors and enabling Hash Equivalence [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:28:50Z

Alen Sunnny Mathew: /* Overview */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [https://docs.yoctoproject.org/ref-manual/index.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the oe-init-build-env [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory], which is build in this case and is located in the Source Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using Shared State Cache [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache]mirrors and enabling Hash Equivalence [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:26:36Z

Alen Sunnny Mathew: /* Obtain the required Host packages for your host system to support Yocto */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto (Kirkstone 4.0.2) ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the oe-init-build-env [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory], which is build in this case and is located in the Source Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using Shared State Cache [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache]mirrors and enabling Hash Equivalence [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:25:25Z

Alen Sunnny Mathew: /* Building Your Image */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the oe-init-build-env [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory], which is build in this case and is located in the Source Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using Shared State Cache [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache]mirrors and enabling Hash Equivalence [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory]:

BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:24:10Z

Alen Sunnny Mathew: /* Building Your Image */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the oe-init-build-env [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory], which is build in this case and is located in the Source Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Source-Directory]. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using Shared State Cache [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache]mirrors and enabling Hash Equivalence [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory]:

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:23:01Z

Alen Sunnny Mathew: /* Building Your Image */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the oe-init-build-env [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory], which is build in this case and is located in the Source Directory. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.

Tip:
You can significantly speed up your build and guard against fetcher failures by using Shared State Cache [https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache]mirrors and enabling Hash Equivalence [https://docs.yoctoproject.org/overview-manual/concepts.html#hash-equivalence]. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

To use such mirrors, uncomment the below lines in your conf/local.conf file in the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory]:

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:18:25Z

Alen Sunnny Mathew: /* Building Your Image */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the oe-init-build-env [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory], which is build in this case and is located in the Source Directory. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

2.Examine Your Local Configuration File: When you set up the build environment, a local configuration file named local.conf becomes available in a conf subdirectory of the Build Directory. For this example, the defaults are set to build for a qemux86 target, which is suitable for emulation. The package manager used is set to the RPM package manager.
Tip:
You can significantly speed up your build and guard against fetcher failures by using Shared State Cache mirrors and enabling Hash Equivalence. This way, you can use pre-built artifacts rather than building them. This is relevant only when your network and the server that you use can download these artifacts faster than you would be able to build them.

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:16:32Z

Alen Sunnny Mathew: /* Building Your Image */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the oe-init-build-env [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

Among other things, the script creates the Build Directory [https://docs.yoctoproject.org/ref-manual/terms.html#term-Build-Directory], which is build in this case and is located in the Source Directory. After the script runs, your current working directory is set to the Build Directory. Later, when the build completes, the Build Directory contains all the files created during the build.

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:14:06Z

Alen Sunnny Mathew: /* Building Your Image */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the oe-init-build-env [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:12:39Z

Alen Sunnny Mathew: /* Building Your Image */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the oe-init-build-env [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.

You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.

The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org

For more information about OpenEmbedded see their website:
https://www.openembedded.org/

### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support

You can also run generated QEMU images with a command like 'runqemu qemux86-64'

Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:09:37Z

Alen Sunnny Mathew: /* Configure the build environment to build an emulator image */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Building Your Image ==

to build an image follow the instructions below.The build process creates an entire Linux distribution, including the toolchain, from source

Note:

If you are working behind a firewall and your build host is not set up for proxies, you may face problems with the build process when fetching source code (e.g. fetcher failures or Git failures).

If you do not know your proxy settings, consult your local network infrastructure resources and get that information. A good starting point could also be to check your web browser settings. Finally, you can find more information on the “Working Behind a Network Proxy” [https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy] page of the Yocto Project Wiki.

1.Initialize the Build Environment: From within the poky directory, run the oe-init-build-env [https://docs.yoctoproject.org/ref-manual/structure.html#oe-init-build-env] environment setup script to define Yocto Project’s build environment on your build host.

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:02:17Z

Alen Sunnny Mathew: /* Obtain the required packages for your host system to support Yocto */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required Host packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T11:01:09Z

Alen Sunnny Mathew: /* Use Git to Clone Poky */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files [https://docs.yoctoproject.org/dev-manual/start.html#locating-yocto-project-source-files] section in the Yocto Project Development Tasks Manual.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T10:59:44Z

Alen Sunnny Mathew: /* Use Git to Clone Poky */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

For more options and information about accessing Yocto Project related repositories, see the Locating Yocto Project Source Files section in the Yocto Project Development Tasks Manual.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T10:59:01Z

Alen Sunnny Mathew: /* Use Git to Clone Poky */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

For this example, check out the kirkstone branch based on the Kirkstone release:
$ git checkout -t origin/kirkstone -b my-kirkstone
Branch 'my-kirkstone' set up to track remote branch 'kirkstone' from 'origin'.
Switched to a new branch 'my-kirkston
The previous Git checkout command creates a local branch named my-kirkstone. The files available to you in that branch exactly match the repository’s files in the kirkstone release branch.

Note that you can regularly type the following command in the same directory to keep your local files in sync with the release branch:

$ git pull
Already up-to-date.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T10:50:23Z

Alen Sunnny Mathew: /* Use Git to Clone Poky */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

Go to Releases wiki page, and choose a release codename (such as kirkstone), corresponding to either the latest stable release or a Long Term Support release.

Then move to the poky directory and take a look at existing branches:

$ cd poky
$ git branch -a

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T10:42:06Z

Alen Sunnny Mathew: /* Use Git to Clone Poky */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.
$ git clone git://git.yoctoproject.org/poky

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T10:40:28Z

Alen Sunnny Mathew: /* Download and extract the Yocto 2.0 (Jethro) release */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Use Git to Clone Poky==

Once you complete the setup instructions for your machine, you need to get a copy of the Poky repository on your build host. Use the following commands to clone the Poky repository.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T10:06:22Z

Alen Sunnny Mathew: /* Overview */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0.2 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T09:59:08Z

Alen Sunnny Mathew: /* Obtain the required packages for your host system to support Yocto */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Note:
For host package requirements on all supported Linux distributions, see the Required Packages for the Build Host section in the Yocto Project Reference Manual.
Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T09:58:09Z

Alen Sunnny Mathew: /* Obtain the required packages for your host system to support Yocto */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Full details of system requirements and installation can be found in the Yocto Quickstart [https://docs.yoctoproject.org/]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T09:55:44Z

Alen Sunnny Mathew: /* Obtain the required packages for your host system to support Yocto */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Full details of system requirements and installation can be found in the Yocto Quickstart [http://www.yoctoproject.org/docs/2.0/yocto-project-qs/yocto-project-qs.html here]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T09:55:28Z

Alen Sunnny Mathew: /* Obtain the required packages for your host system to support Yocto */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Full details of system requirements and installation can be found in the Yocto Quickstart [http://www.yoctoproject.org/docs/2.0/yocto-project-qs/yocto-project-qs.html here]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T09:54:40Z

Alen Sunnny Mathew: /* Overview */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$

Full details of system requirements and installation can be found in the Yocto Quickstart [http://www.yoctoproject.org/docs/2.0/yocto-project-qs/yocto-project-qs.html here]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T09:53:30Z

Alen Sunnny Mathew: /* Overview */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

Compatible Linux Distribution:

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see
the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in
the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$

Full details of system requirements and installation can be found in the Yocto Quickstart [http://www.yoctoproject.org/docs/2.0/yocto-project-qs/yocto-project-qs.html here]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T09:51:29Z

Alen Sunnny Mathew: /* Overview */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

If your build host does not meet any of these three listed version requirements, you can take steps to prepare the system so that you can still use the Yocto Project. See the Required Git, tar, Python and gcc Versions section in the Yocto Project Reference Manual for information.

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$

Full details of system requirements and installation can be found in the Yocto Quickstart [http://www.yoctoproject.org/docs/2.0/yocto-project-qs/yocto-project-qs.html here]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T09:50:32Z

Alen Sunnny Mathew: /* Overview */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

Make sure your Build Host meets the following requirements:

* 50 Gbytes of free disk space
* Runs a supported Linux distribution (i.e. recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux distributions that support the Yocto Project, see the Supported Linux Distributions section in the Yocto Project Reference Manual. For detailed information on preparing your build host, see the Preparing the Build Host section in the Yocto Project Development Tasks Manual.
* Git 1.8.3.1 or greater
* tar 1.28 or greater
* Python 3.6.0 or greater.
* gcc 5.0 or greater.

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$

Full details of system requirements and installation can be found in the Yocto Quickstart [http://www.yoctoproject.org/docs/2.0/yocto-project-qs/yocto-project-qs.html here]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T09:48:21Z

Alen Sunnny Mathew: /* Obtain the required packages for your host system to support Yocto */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$

Full details of system requirements and installation can be found in the Yocto Quickstart [http://www.yoctoproject.org/docs/2.0/yocto-project-qs/yocto-project-qs.html here]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T09:47:22Z

Alen Sunnny Mathew: /* Obtain the required packages for your host system to support Yocto */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git
python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Full details of system requirements and installation can be found in the Yocto Quickstart [http://www.yoctoproject.org/docs/2.0/yocto-project-qs/yocto-project-qs.html here]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T09:46:55Z

Alen Sunnny Mathew: /* Obtain the required packages for your host system to support Yocto */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

You must install essential host packages on your build host. The following command installs the host packages based on an Ubuntu distribution:

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git

python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Full details of system requirements and installation can be found in the Yocto Quickstart [http://www.yoctoproject.org/docs/2.0/yocto-project-qs/yocto-project-qs.html here]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T09:45:24Z

Alen Sunnny Mathew: /* Obtain the required packages for your host system to support Yocto */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

First we will install the required host packages for Ubuntu as detailed in the quickstart, i.e.

$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

Full details of system requirements and installation can be found in the Yocto Quickstart [http://www.yoctoproject.org/docs/2.0/yocto-project-qs/yocto-project-qs.html here]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]

Building your own recipes from first principles

2022-07-20T09:41:37Z

Alen Sunnny Mathew: /* Overview */

== Overview ==

This walk-through has the aim of taking you from a clean system through to building and packaging an example project for inclusion in an image.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant, 
such as [[#Build an example project on the host for testing (optional)|building an example package on the host to test]] or [[#Build an example package based on a git repository commit|building an example package from a git commit]].

The following assumptions are made. You are:

* familiar with basic Linux admin tasks
* aware of the Yocto Project Reference Manual [http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html here].
* using [https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes Ubuntu 14.04 LTS] as your host build system
* working with Yocto 4.0 (Kirkstone) release

== Obtain the required packages for your host system to support Yocto ==

First we will install the required host packages for Ubuntu as detailed in the quickstart, i.e.

$ sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib build-essential chrpath socat libsdl1.2-dev xterm nano

Full details of system requirements and installation can be found in the Yocto Quickstart [http://www.yoctoproject.org/docs/2.0/yocto-project-qs/yocto-project-qs.html here]

Add some other packages which will be needed for the walk-through below.

$ sudo apt-get install autoconf libtool rpm

== Download and extract the Yocto 2.0 (Jethro) release ==

At the time of writing, the current release of Yocto (2.0) can be found [https://www.yoctoproject.org/downloads/core/jethro20 here]

$ cd ~
$ mkdir yocto
$ cd yocto
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.0/poky-jethro-14.0.0.tar.bz2
$ tar xjvf poky-jethro-14.0.0.tar.bz2

This will get you the Yocto 2.0 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

== Configure the build environment to build an emulator image ==

$ cd ~/yocto/poky-jethro-14.0.0
$ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using <code>oe-init-build-env</code>.

<code>oe-init-build-env</code> will create a default configuration file in <code>conf/local/conf</code> which will build an emulator image suitable for execution with <code>qemu</code>

== Build a baseline image ==

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

$ bitbake core-image-minimal

== Build an example project on the host for testing (optional) ==

The most straightforward way to cross-compile projects for different targets within Yocto is to make use of [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/index.html autotools]

Projects which support <code>autotools</code> provide a set of template files which are then used by the <code>autotools</code> to generate <code>Makefiles</code> and associated configuration files which are appropriate to build for the target environment.

A very basic example 'Hello World' style project called <code>bbexample</code> has been committed to GitHub [https://github.com/DynamicDevices/bbexample here]

If you take a look at the two source files [https://github.com/DynamicDevices/bbexample/blob/master/bbexample.c bbexample.c] and [https://github.com/DynamicDevices/bbexample/blob/master/bbexamplelib.c bbexamplelib.c] you can see the main entry point prints a Hello World message, then calls a function exported by the shared library which also prints a message.

Discussion of <code>autotools</code> template configuration is outside the scope of this guide, but the <code>bbexample</code> project is based on the 'Quick and Dirty' example which can be found [http://www.niksula.cs.hut.fi/~mkomu/docs/autohowto.html here]

The project itself builds an executable, <code>bbexample</code> which has a dependency on a shared library <code>libbbexample.so</code>.

To build the project on the host independently of Yocto first clone the example repository

$ mkdir ~/host
$ cd ~/host
$ git clone https://github.com/DynamicDevices/bbexample

Then run the autotools, configure the build, and make the project

$ cd bbexample
$ ./autogen.sh
$ ./configure
$ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There is a new executable <code>bbexample</code> which depends upon a shared library <code>.libs/libbbexample.so</code>

As this project has been built to run on the host you can

./bbexample

It will output

Hello Yocto World...
Hello World (from a shared library!)

So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

== Adding new recipes to the build system ==

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

=== Placing a recipe in an existing layer (example only) ===

For example you could

$ cd ~/yocto/poky-jethro-14.0.0/meta-yocto
$ mkdir recipes-example
$ mkdir bbexample
$ cd bbexample
$ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

$ cd ~/yocto/poky-jethro-14.0.0/build-qemux86
$ bitbake bbexample

=== Using a new layer for recipes ===

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the <code>yocto-layer</code> command and committed into a GitHub repository [https://github.com/DynamicDevices/meta-example here].

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in <code>conf/bblayers.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0
$ git clone https://github.com/DynamicDevices/meta-example
$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ nano conf/bblayers.conf

Your <code>bblayers.conf</code> should look similar to this

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
"
BBLAYERS_NON_REMOVABLE ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
"

Make the new layer visible to <code>bitbake</code> by adding a line to <code>BBLAYERS</code>

BBLAYERS ?= " \
/home/user/yocto/poky-jethro-14.0.0/meta \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto \
/home/user/yocto/poky-jethro-14.0.0/meta-yocto-bsp \
/home/user/yocto/poky-jethro-14.0.0/meta-example \
"

Now <code>bitbake</code> can see the recipes in the new layer.

You will also see when <code>bitbake</code> runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

Build Configuration:
BB_VERSION = "1.28.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.0"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "<unknown>:<unknown>"
meta-example = "master:5ee4f20b041bc886b1d2d913ac11814057317634"

== Build an example package based on a git repository commit ==

==== The bbexample recipe ====

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb].

The main points to note are that

* <code>SRC_URI</code> is set to point to the git repository
* <code>SRC_REV</code> corresponds to a particular commit into that repository (it is also possible to specify a branch)
* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,
* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
* The source directory for the recipe build, <code>${S}</code> is set to <code>"${WORKDIR}/git"</code> which is the default location to which the retrieved git repository will be checked out and unpacked.
* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.
* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To start the build

$ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample*

(Or instead of <code>bbexample</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the <code>bbexample</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a remote source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-rt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb].

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

We are using a release archived that was manually uploaded to GitHub. The archive corresponds to the bbexample source code tagged at v1.0. This is the preferred approach to using dynamically generated GitHub archives, as the checksums of these archives can change intermittently when they are regenerated. Manually uploaded archives will not change.

This tagged archive is what we set in the recipe <code>SRC_URI</code> to retrieve and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to the remote source archive

SRC_URI = "https://github.com/DynamicDevices/bbexample/releases/download/v1.0/bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

* There is no <code>SRC_REV</code> here but instead we have <code>SRC_URI</code> values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running <code>md5sum archivename</code> and <code>sha256sum archivename</code> but it is easier to set these incorrectly and let <code>bitbake</code> retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

SRC_URI[md5sum] = "e15723f0d5ac710bbe788cd3a797bc44"
SRC_URI[sha256sum] = "0b34eb133596348bb6f1a3ef5e05e4d5bf0c88062256affe768d8337d7cc8f83"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>S = "${WORKDIR}/bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the archive uploaded to GitHub.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any existing bbexample files

$ bitbake -f -c cleansstate bbexample

To start the build

$ bitbake bbexample-rt

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-rt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-rt*

(Or instead of <code>bbexample-rt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-rt-1.0-r0.0.i586.rpm

For the <code>bbexample-rt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-rt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Build an example package based on a local source archive ==

The recipe we are going to build is [https://github.com/DynamicDevices/meta-example/blob/master/recipes-example/bbexample/bbexample-lt_1.0.bb /home/user/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb].

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if <code>autotools</code> is not provided then the recipe may well have to be extended with a <code>do_install</code> function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive we uploaded to GitHub has been copied locally into the <code>meta-example</code> layer folder tree,

yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe <code>SRC_URI</code> to copy across, unpack, patch (if necessary) and build.

The main points to note are that

* <code>SRC_URI</code> is set to point to a local file archive

SRC_URI = "file://bbexample-${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

* We provide a search path to ensure <code>bitbake</code> can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed <code>bbexample-1.0.tar.gz</code>, and thus <code>bitbake</code> will find it to unpack.

~/yocto/poky-jethro-14.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

* There is no <code>SRC_REV</code> here or check-sum for the local archive.

* There is a <code>LICENSE</code> variable defined to indicate the type of license to the build environment (MIT) and another variable,

* <code>LIC_FILES_CHKSUM</code>, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

* The source directory for the recipe build, <code>${S}</code> is set to <code>"bbexample-${PV}"</code> which corresponds to the path to the source-code when extracted from the local archive.

# Make sure our source directory (for the build) matches the directory structure in the tarball
S = "${WORKDIR}/bbexample-${PV}"

* We inherit a class called <code>autotools</code> which provides the functionality to enable <code>bitbake</code> to automatically build projects with <code>autotools</code> support.

* There is a marked absence of a <code>do_install</code> function, which you may have seen elsewhere, as this is dealt with by the <code>autotools</code> support

To clean out any previous bbexample files

$ bitbake -f -c cleansstate bbexample bbexample-rt

To start the build

$ bitbake bbexample-lt

Bitbake will work through a number of tasks, including fetching the source archive from the local file-system, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes <code>bbexample</code>, <code>bbexample-rt</code> and <code>bbexample-lt</code> are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, <code>qemux86</code>, and are building RPM packages (the default), output packages will be in

~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-lt-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix <code>i586</code> will be replaced by a different machine-specific name. To find the location of the package you can use

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ find -name bbexample-lt*

(Or instead of <code>bbexample-lt</code> use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which <code>bitbake</code> builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see [http://www.embeddedlinux.org.cn/OEManual/recipes_packages.html here]

Having found the package you can check that the contents are as expected with

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/tmp/deploy/rpm
$ rpm -qlp i586/bbexample-lt-1.0-r0.0.i586.rpm

For the <code>bbexample-lt</code> recipe we expect to see the cross-compiled executable <code>bbexample</code> and the shared library it needs <code>libbbexample.so.1</code>

/usr
/usr/bin
/usr/bin/bbexample
/usr/lib
/usr/lib/libbbexample.so.1
/usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your <code>conf/local.conf</code>

IMAGE_INSTALL_append = " bbexample-lt"

Alternatively you could make the new packages available to existing target systems using the Yocto <code>package-management</code> tools such as <code>smart</code>.

If you wish to build and test your example on the emulator skip forward to [[#Testing the example on a target]]

== Testing the example on an emulated target ==

To to this we first want to add the appropriate recipe to an image and then run up that image in our emulator target. We could of course be targeting a real board, but with the wide variety of boards available this is outside the scope of this guide, and you should consult the documentation provided by your board vendor.

=== Adding a package to an image via local.conf ===

There are a couple of ways (at least!) to add a new package to an image. The simplest is to add the package to a variable within your <code>local.conf</code>

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86/
$ nano conf/local.conf

Add a line at the bottom. You may wish to use <code>bbexample-rt</code> or <code>bbexample-lt</code> instead of <code>bbexample</code>. There should be no different in the output files.

IMAGE_INSTALL_append = " bbexample"

Then bitbake an image of your choice. With this method any image you build will have the <code>bbexample</code> package added to it.

$ bitbake core-image-minimal

=== Adding a package to an image via a .bbappend ===

Alternatively you may wish to add specific packages to specific images, which is generally viewed as better practice.

We are using <code>core-image-minimal</code> for this guide. The recipe for this image can be found in

~/yocto/poky-jethro-14.0.0/meta/recipes-core/images/core-image-minimal.bb

We are also using our own new <code>meta-example</code> layer, so this seems an appropriate place to add a .bbappend to extend the original <code>core-image-minimal</code> recipe.

We need to recreate the path to the original recipe in our own layer, so

$ cd ~/yocto/poky-jethro-14.0.0/meta-example
$ mkdir -p recipes-core/images
$ cd recipes-core.images
$ nano core-image-minimal.bbappend

Add the following line to your empty new file

IMAGE_INSTALL += " bbexample"

(Ensure that you no longer have <code>bbexample</code> in your <code>conf/local.conf</code>. It won't break the build but you want to be sure your .bbappend is working correctly)

Now build the image

$ cd ~/yocto/poky-jethro-14.0.0/build_qemux86
$ bitbake core-image-minimal

=== Running the image in the emulator ===

To run up the image, simply use

$ runqemu qemux86

This will boot the emulator, load up the image, you'll see a kernel loading and with <code>core-image-minimal</code> a console prompt. If you were building, say <code>core-image-sato</code> instead you would see a basic user interface.

If you find that your keymap is incorrect you might wish to set this explicitly, for example

$ runqemu qemux86 qemuparams='-k en-gb'

or

$ runqemu qemux86 qemuparams='-k en-us'

Log into the emulator as 'root', no password and run the example

$ bbexample

You will see the Hello World output!

[[File:Bbexample.png|800px]]

== Recipe gotchas ==

=== Incorrect source folder ${S} ===

It is extremely important to make sure that the source folder, the <code>${S}</code> variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the <code>LICENSE</code> file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

$ bitbake -c devshell recipename

This will drop you into the configured location of <code>${S}</code>. If the sources defined in <code>SRC_URI</code> seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a <code>patches</code> folder, this is a good indication that the code has been unpacked into a different folder.

$ cd ..
$ ls

You often will see another folder in the directory level about <code>${S}</code> which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

=== Incorrect license checksum ===

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the <code>LICENSE</code> variable in the recipe reports the correct license type you can update the license checksum to that reported by <code>bitbake</code> by modifying <code>LIC_FILES_CHKSUM</code>

=== Incorrect source archive checksum ===

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads) or because the archive has actually been changed.

* You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

* Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with <code>md5sum</code> and <code>sha256sum</code>, or simply use what <code>bitbake</code> calculates and provides for you in the log.

SRC_URI[md5sum] = "???"
SRC_URI[sha256sum] = "???"

* Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the <code>downloads</code> folder for use by <code>bitbake</code>

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

=== Missing source archive ===

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with [[How_do_I#Q:How do I create my own source download mirror? | here]]

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-jethro-14.0.0/build_qemux86/downloads folder, creating an empty .done file with

$ touch archivename.done

It should then be picked up and used by <code>bitbake</code>

== Feedback ==

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon [mailto://ajlennon@dynamicdevices.co.uk here]