Building and running embedded Linux .NET applications from first principles

[Work in progress]

Overview

This walk-through has the aim of taking you from a clean system through to including Mono in a build image with the meta-mono layer, then building and packaging an example .NET project for inclusion in that image.

You may already have Yocto installed and just be looking to work with recipes for the first time, in which case you can jump forward to the section you find most relevant,
such as building an example package on the host to test or building an example package from a git commit.

The following assumptions are made. You are:

• familiar with basic Linux admin tasks
• aware of the Yocto Project Reference Manual here.
• using Ubuntu 12.04 LTS as your host build system
• working with Yocto 1.6 (daisy) release

Obtain the required packages for your host system to support Yocto

First we will install the required host packages for Ubuntu as detailed in the quickstart, i.e.

 $ sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib build-essential chrpath libsdl1.2-dev xterm nano

Full details of system requirements and installation can be found in the Yocto Quickstart here

Also install Mono on your host system as we'll use it to build and run some examples for test later

 $ sudo apt-get install mono-complete
 $ mono --version

With Ubuntu 12.04 LTS this will install Mono version 2.10 which is now quite old (December 2011).

If you wish to install a newer build of Mono to your host system you can follow the instructions here.

 $ sudo add-apt-repository ppa:directhex/monoxide
 $ sudo apt-get update
 $ sudo apt-get install mono-complete
 $ mono --version

This will install Mono 3.2.1 (August 2013)

If you wish to use the absolute latest Mono then there are instructions you can follow to build a release tarball here and from git here. Be aware this may not be straightforward and that there can be issues, such as with missing files, if you follow this process.

Download and extract the Yocto 1.6 release

At the time of writing, the current release of Yocto (1.6) can be found here

 $ cd ~
 $ mkdir yocto
 $ wget http://downloads.yoctoproject.org/releases/yocto/yocto-1.6/poky-daisy-11.0.0.tar.bz2
 $ tar xjvf poky-daisy-11.0.0.tar.bz2

This will get you the Yocto 1.6 base meta-data and the bitbake tool. You can also add in extra layers, usually of the form "meta-foo" to provide machine support and additional functionality.

Configure the build environment to build an emulator image

 $ cd ~/yocto/poky-daisy-11.0.0
 $ source oe-init-build-env build_qemux86

This will create a build tree in "build_qemux86" although you could use a different name if you so wish with no adverse effects.

It is entirely possible to have many build trees in parallel in different folders and to switch between them using oe-init-build-env.

oe-init-build-env will create a default configuration file in conf/local/conf which will build an emulator image suitable for execution with qemu

Build a baseline image

After configuring the environment you will be left in the build_qemux86 folder.

You should then build a baseline image, which will take some time (numbers of hours)

 $ bitbake core-image-minimal

Build an example project on the host for testing (optional)

Autotools

The most straightforward way to compile non-.NET projects for different targets within Yocto is to make use of autotools

Projects which support autotools provide a set of template files which are then used by the autotools to generate Makefiles and associated configuration files which are appropriate to build for the target environment.

Similarly it is possible to compile Mono/.NET projects using autotools

A very basic example 'Hello World' style project called mono-helloworld has been committed to GitHub here

If you take a look at the two source files helloworld.cs and helloworldform.cs you can see the first outputs a 'Hello World' message to the console, and the second creates a Windows Form titled 'Hello World'.

Discussion of autotools template configuration for Mono is outside the scope of this guide, but the mono-helloworld project is based on the mono-skel example which can be found in the Autotools section of the Mono Application Deployment guidelines here

The project itself builds two .NET executables, helloworldcode> and helloworldform respectively, the first of which is a console application and the second of which is a simple Windows Forms application.

To build the project on the host independently of Yocto first clone the example repository

 $ mkdir ~/host
 $ cd ~/host
 $ git clone https://github.com/DynamicDevices/mono-helloworld

Then run the autotools, configure the build, and make the project

 $ cd mono-helloworld
 $ ./autogen.sh
 $ ./configure
 $ make

Following a successful compilation you will have a number of new files in the root of the build folder.

There are two new .NET executables src/helloworld.exe and src/helloworldform.exe.

You can run the first with

 $ mono src/helloworld.exe

It will output

HelloWorld

You can run the second with

 $ mono src/helloworldform.exe

Depending on your host environment (e.g. using SSH) you may need to explicitly set the DISPLAY variable for this to work, with

 $ export DISPLAY=:0
 $ mono src/helloworldform.exe

This will give you a basic Windows Forms window title So you have now shown that you can successfully fetch configure and build the project on the host.

Next we will look at how Yocto automates the this process of fetching, configuring and building, then also installs and packages the output files.

Adding new recipes to the build system

There are different ways to add new recipes to Yocto.

One way is to simply create a new recipe_version.bb file in a recipe-foo/recipe folder within one of the existing layers used by Yocto.

Placing a recipe in an existing layer (example only)

For example you could

 $ cd ~/yocto/poky-daisy-11.0.0/meta-yocto
 $ mkdir recipes-example
 $ mkdir bbexample
 $ cd bbexample
 $ wget https://raw.githubusercontent.com/DynamicDevices/meta-example/master/recipes-example/bbexample/bbexample_1.0.bb

The recipe will then be picked up by bitbake and you can build the recipe with

 $ cd ~/yocto/poky-daisy-11.0.0/build-qemux86
 $ bitbake bbexample 

Using a new layer for recipes

A preferred method for adding recipes to the build environment, and the method you should use with this guide, is to place them within a new layer.

Layers isolate particular sets of build meta-data based on machine, functionality or similar, and help to keep the environment clean.

An example layer, meta-example, has been created using the yocto-layer command and committed into a GitHub repository here.

To use a new layer such as this you first clone the git repository and then add the layer to your bitbake configuration in conf/bblayers.conf

 $ cd ~/yocto/poky-daisy-11.0.0
 $ git clone https://github.com/DynamicDevices/meta-example
 $ cd ~/yocto/poky-daisy-11.0.0/build_qemux86
 $ nano conf/bblayers.conf

Your bblayers.conf should look similar to this

 # LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
 # changes incompatibly
 LCONF_VERSION = "6"
 
 BBPATH = "${TOPDIR}"
 BBFILES ?= ""
 
 BBLAYERS ?= " \
   /home/user/yocto/poky-daisy-11.0.0/meta \
   /home/user/yocto/poky-daisy-11.0.0/meta-yocto \
   /home/user/yocto/poky-daisy-11.0.0/meta-yocto-bsp \
   "
 BBLAYERS_NON_REMOVABLE ?= " \
   /home/user/yocto/poky-daisy-11.0.0/meta \
   /home/user/yocto/poky-daisy-11.0.0/meta-yocto \
   "

Make the new layer visible to bitbake by adding a line to BBLAYERS

 BBLAYERS ?= " \
   /home/user/yocto/poky-daisy-11.0.0/meta \
   /home/user/yocto/poky-daisy-11.0.0/meta-yocto \
   /home/user/yocto/poky-daisy-11.0.0/meta-yocto-bsp \
   /home/user/yocto/poky-daisy-11.0.0/meta-example \
   "

Now bitbake can see the recipes in the new layer.

You will also see when bitbake runs and shows the Build Configuration that the repository branch and hash of your layer is shown which is useful to know, particularly when comparing notes with others as to why a build fails, e.g.

 Build Configuration:
 BB_VERSION        = "1.22.0"
 BUILD_SYS         = "i686-linux"
 NATIVELSBSTRING   = "Ubuntu-12.04"
 TARGET_SYS        = "i586-poky-linux"
 MACHINE           = "qemux86"
 DISTRO            = "poky"
 DISTRO_VERSION    = "1.6"
 TUNE_FEATURES     = "m32 i586"
 TARGET_FPU        = ""
 meta
 meta-yocto
 meta-yocto-bsp    = "<unknown>:<unknown>"
 meta-example      = "master:da011ba5dda914dc8bc3768356e2a9fd726383bc"

Build an example package based on a git repository commit

The bbexample recipe

The recipe we are going to build is /home/user/yocto/poky-daisy-11.0.0/meta-example/recipes-example/bbexample/bbexample_1.0.bb.

The main points to note are that

• SRC_URI is set to point to the git repository
• SRC_REV corresponds to a particular commit into that repository (it is also possible to specify a branch)
• There is a LICENSE variable defined to indicate the type of license to the build environment (MIT) and another variable,
• LIC_FILES_CHKSUM, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.
• The source directory for the recipe build, ${S} is set to "${WORKDIR}/git" which is the default location to which the retrieved git repository will be checked out and unpacked.
• We inherit a class called autotools which provides the functionality to enable bitbake to automatically build projects with autotools support.
• There is a marked absence of a do_install function, which you may have seen elsewhere, as this is dealt with by the autotools support

To start the build

 $ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source from the git repository, unpacking, configuring, compiling, installing and packaging the output.

As we are building for the emulator, qemux86, and are building RPM packages (the default), output packages will be in

 ~/yocto/poky-daisy-11.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix i586 will be replaced by a different machine-specific name. To find the location of the package you can use

 $ cd ~/yocto/poky-daisy-11.0.0/build_qemux86/tmp/deploy/rpm
 $ find -name bbexample*

(Or instead of bbexample use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which bitbake builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see here

Having found the package you can check that the contents are as expected with

 $ cd ~/yocto/poky-daisy-11.0.0/build_qemux86/tmp/deploy/rpm
 $ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the bbexample recipe we expect to see the cross-compiled executable bbexample and the shared library it needs libbbexample.so.1

 /usr
 /usr/bin
 /usr/bin/bbexample
 /usr/lib
 /usr/lib/libbbexample.so.1
 /usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your conf/local.conf

 IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto package-management tools such as smart.

Build an example package based on a remote source archive

The recipe we are going to build is /home/user/yocto/poky-daisy-11.0.0/meta-example/recipes-example/bbexample/bbexample-rt_1.0.bb.

This is based on the previous recipe that builds from a git checkout, with the major difference being we are building from a remote tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if autotools is not provided then the recipe may well have to be extended with a do_install function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the git commit used above has been tagged 'bbexample-v1.0' in the GitHub repository, which causes GitHub to provide a release source archive.

This tagged archive is what we set in the recipe SRC_URI to retrieve and build.

The main points to note are that

• SRC_URI is set to point to the remote source archive

 SRC_URI = "https://github.com/DynamicDevices/bbexample/archive/bbexample-v${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-rt' I have hard-coded the names here.

• There is no SRC_REV here but instead we have SRC_URI values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running md5sum archivename and sha256sum archivename but it is easier to set these incorrectly and let bitbake retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

 SRC_URI[md5sum] = "a040115592f4200188a805368d9456ff"
 SRC_URI[sha256sum] = "589b22fc6c2c0b266223c5988f0e33f5c2f03a114da9145e722e3552a0edd308"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-daisy-11.0.0/build_qemux86/downloads) or because the archive has actually been changed.

• There is a LICENSE variable defined to indicate the type of license to the build environment (MIT) and another variable,
• LIC_FILES_CHKSUM, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

• The source directory for the recipe build, ${S} is set to "bbexample-bbexample-v${PV}" which corresponds to the path to the source-code when extracted from the archive generated by GitHub. This looks slightly strange but is due to the vagaries of the way the git repository tag was added, as this is embedded by GitHub within the archive.

 # Make sure our source directory (for the build) matches the directory structure in the tarball
 # A tagged tarball from github contains a folder which includes the github tag, so deal with it here
 S = "${WORKDIR}/bbexample-bbexample-v${PV}"

• We inherit a class called autotools which provides the functionality to enable bitbake to automatically build projects with autotools support.
• There is a marked absence of a do_install function, which you may have seen elsewhere, as this is dealt with by the autotools support

To start the build

 $ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes bbexample, bbexample-rt and bbexample-lt are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, qemux86, and are building RPM packages (the default), output packages will be in

 ~/yocto/poky-daisy-11.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix i586 will be replaced by a different machine-specific name. To find the location of the package you can use

 $ cd ~/yocto/poky-daisy-11.0.0/build_qemux86/tmp/deploy/rpm
 $ find -name bbexample*

(Or instead of bbexample use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which bitbake builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see here

Having found the package you can check that the contents are as expected with

 $ cd ~/yocto/poky-daisy-11.0.0/build_qemux86/tmp/deploy/rpm
 $ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the bbexample recipe we expect to see the cross-compiled executable bbexample and the shared library it needs libbbexample.so.1

 /usr
 /usr/bin
 /usr/bin/bbexample
 /usr/lib
 /usr/lib/libbbexample.so.1
 /usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your conf/local.conf

 IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto package-management tools such as smart.

Build an example package based on a local source archive

The recipe we are going to build is /home/user/yocto/poky-daisy-11.0.0/meta-example/recipes-example/bbexample/bbexample-lt_1.0.bb.

This is based on the previous recipe that builds from a remote source release, with the major difference being we are building from a local source tarball.

This tarball may, in theory, contain any sources we wish to build, although sometimes build failures may require patching of the sources, and if autotools is not provided then the recipe may well have to be extended with a do_install function to ensure appropriate files are installed, and the FILES_${PN} variable modified to ensure installed files are correctly packaged.

For this simple example the release source archive available from GitHub has been copied locally into the meta-example layer folder tree,

 yocto/poky-daisy-11.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0/bbexample-v1.0.tar.gz

This local file is what we set in the recipe SRC_URI to copy across, unpack, patch (if necessary) and build.

The main points to note are that

• SRC_URI is set to point to a local file archive

 SRC_URI = "file://bbexample-v${PV}.tar.gz"

Ideally we would use both of the variables ${PN} and ${PV} which would be replaced by the recipe name and recipe version, and could usually be expected to map to the naming convention employed for the source archive file. This makes the recipe more generic and easier to update to a new version of the source code by renaming the recipe .bb file. However as I am using a slightly different recipe name, 'bbexample-lt' I have hard-coded the names here.

• We provide a search path to ensure bitbake can find the archive

FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"

In this case the above will expand to the following folder, which is where we have placed bbexample-v1.0.tar.gz, and thus bitbake will find it to unpack.

 ~/yocto/poky-daisy-11.0.0/meta-example/recipes-example/bbexample/bbexample-lt-1.0

• There is no SRC_REV here but instead we have SRC_URI values for md5sum and an sha256sum check-sums

As you would expect, these correspond to the particular check-sums generated by those algorithms when run over the entire archive.

You can generate these by hand by retrieving the file yourself, running md5sum archivename and sha256sum archivename but it is easier to set these incorrectly and let bitbake retrieve the archive and error on incorrect checksums and which point it will tell you what the lines should be.

 SRC_URI[md5sum] = "a040115592f4200188a805368d9456ff"
 SRC_URI[sha256sum] = "589b22fc6c2c0b266223c5988f0e33f5c2f03a114da9145e722e3552a0edd308"

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-daisy-11.0.0/build_qemux86/downloads) or because the archive has actually been changed.

• There is a LICENSE variable defined to indicate the type of license to the build environment (MIT) and another variable,
• LIC_FILES_CHKSUM, points to a file within the source tree that will be retrieved, with a corresponding md5 check-sum to ensure that somebody has actually verified the source license file corresponds to that given to the build environment. Also that this file, and thus potentially the license, has not changed over time.

• The source directory for the recipe build, ${S} is set to "bbexample-bbexample-v${PV}" which corresponds to the path to the source-code when extracted from the local archive. This looks slightly strange but is due to the vagaries of the way the git repository tag was added, as this is embedded by GitHub within the archive. Alternatively we could have repacked the archive with a cleaner folder hierarchy.

 # Make sure our source directory (for the build) matches the directory structure in the tarball
 # A tagged tarball from github contains a folder which includes the github tag, so deal with it here
 S = "${WORKDIR}/bbexample-bbexample-v${PV}"

• We inherit a class called autotools which provides the functionality to enable bitbake to automatically build projects with autotools support.
• There is a marked absence of a do_install function, which you may have seen elsewhere, as this is dealt with by the autotools support

To start the build

 $ bitbake bbexample

Bitbake will work through a number of tasks, including fetching the source archive, unpacking, configuring, compiling, installing and packaging the output.

There may be some warnings shown as all three recipes bbexample, bbexample-rt and bbexample-lt are generating the same output and thus there is some collision of shared files. These warnings may be ignored.

As we are building for the emulator, qemux86, and are building RPM packages (the default), output packages will be in

 ~/yocto/poky-daisy-11.0.0/build_qemux86/tmp/deploy/rpm/i586/bbexample-1.0-r0.0.i586.rpm

For other machine targets the folder and suffix i586 will be replaced by a different machine-specific name. To find the location of the package you can use

 $ cd ~/yocto/poky-daisy-11.0.0/build_qemux86/tmp/deploy/rpm
 $ find -name bbexample*

(Or instead of bbexample use a prefix of the name of the recipe you are building)

This will show you both the main package and the subsidiary packages which bitbake builds for each recipe such as -dev, -debug, -staticdev and so forth. For more on this see here

Having found the package you can check that the contents are as expected with

 $ cd ~/yocto/poky-daisy-11.0.0/build_qemux86/tmp/deploy/rpm
 $ rpm -qlp i586/bbexample-1.0-r0.0.i586.rpm

For the bbexample recipe we expect to see the cross-compiled executable bbexample and the shared library it needs libbbexample.so.1

 /usr
 /usr/bin
 /usr/bin/bbexample
 /usr/lib
 /usr/lib/libbbexample.so.1
 /usr/lib/libbbexample.so.1.0.0

You could then add the output of this recipe to your output image by adding something like this to your conf/local.conf

 IMAGE_INSTALL_append = " bbexample"

Alternatively you could make the new packages available to existing target systems using the Yocto package-management tools such as smart.

Recipe gotchas

Incorrect source folder ${S}

It is extremely important to make sure that the source folder, the ${S} variable is set correctly.

When retrieving files from git repositories, or when archives are unpacked, it is entirely possible that the source folder default will not be correct for the actual unpacked location of the sources.

If this happens the build will fail, often with a message that the LICENSE file cannot be found, as this is checked early on in the unpack.

To check through this one option is to drop into a development shell with

 $ bitbake -c devshell recipename

This will drop you into the configured location of ${S}. If the sources defined in SRC_URI seemed to be retrieved correctly but you see nothing in your devshell, excepting perhaps a patches folder, this is a good indication that the code has been unpacked into a different folder.

 $ cd ..
 $ ls

You often will see another folder in the directory level about ${S} which contains the source code.

This being the case you need to exit your devshell and edit your recipe to modify the source directory to point to the correct location. e.g.

 ${S} = "${WORKDIR}/correctfoldername"

Dropping back into the devshell should then show the correct files, and you should be able to make progress with the build

Incorrect license checksum

This can occur, particularly when upgrading a recipe to use a new repository commit or source archive version, as the licensing file may have changed.

If this does occur it is important to check through the new licensing file to ensure that the build environment is still being given correct information on the type of license for the source code.

It may also be that a minor textual change has been made to the file (e.g. new copyright date) which doesn't affect the type of the license itself.

Having satisfied yourself that the LICENSE variable in the recipe reports the correct license type you can update the license checksum to that reported by bitbake by modifying LIC_FILES_CHKSUM

Incorrect source archive checksum

You should be aware that sometimes maintainers re-release archives under the same name but with updated contents. Should this happen the check-sum check will fail and you will have to look into whether this is because of a corrupted download (check the downloaded archive in ~/yocto/poky-daisy-11.0.0/build_qemux86/downloads) or because the archive has actually been changed.

• You can try removing the archive from the downloads folder, and the corresponding .done file. The archive will then be downloaded again which may work if there was a corrupt/interrupted download (although in my experience this occurrence is unlikely).

• Alternatively the archive may have changed and you may wish to use the new archive, in which case you will need to update the check-sums in the recipe to those you calculate yourself on the archive with md5sum and sha256sum, or simply use what bitbake calculates and provides for you in the log.

 SRC_URI[md5sum] = "???"
 SRC_URI[sha256sum] = "???"

• Lastly you may be able to source the correct archive file from a mirror or through Googling, in which case you can drop it into the downloads folder for use by bitbake

In the latter two cases you may wish to feed the issue back to the Yocto mailing list (or other relevant mailing list for the layer in which the recipe resides) as this type of thing means mirrored copies of primary sources become out of sync, and the layer/mirror maintainers may wish to address the issue.

Missing source archive

This is less of an issue than it has been in the past as primary sources are now mirrored in one or more locations, not least by the Yocto project itself.

You can also set up your own mirror sources, which is dealt with here

However for a one-off missing archive it is often quickest and easiest to Google to find it, then drop it into the ~/yocto/poky-daisy-11.0.0/build_qemux86/downloads folder, creating an empty .done file with

 $ touch archivename.done

It should then be picked up and used by bitbake

Feedback

This is a living document. Please feel free to send comments, questions, corrections to Alex Lennon here