Application Development with Extensible SDK

[WORK IN PROGRESS]. Back to the Cookbook home page.

Introduction

Welcome to the Yocto Project Test Kitchen! It can be a bit overwhelming to know where to start with the Yocto Project. You know you want to build an application that runs on embedded hardware, but where and how do you get started? A development environment can be an abstract, somewhat hard-to-imagine thing. This part of the wiki is intended to fill the knowledge gap between the Quick Start and the myriad of ways to build an image and run it on real (or emulated) hardware. So what better way to do that than with a "Test Kitchen" analogy?

Our Test Kitchen must be Linux-powered, of course. And using one of the supported distributions in our Linux-powered kitchen means that we'll already either have or have a way to easily get everything we need to follow the recipes in our Cookbook.

What devtool can do

One of the most powerful tools in our test kitchen is devtool. We can think of devtool as something that can be used to "mix" customization into our Yocto Project image. Some of the common ways to get mixing are:

To add new applications and libraries to a Yocto Project image,
To modify the source for an existing component,
To test changes on target hardware, and
To integrate that change into the rest of the OpenEmbedded (OE) build system.

In short, we can think of devtool as a single point of entry for creating, modifying and testing recipes. Read this section of the SDK manual to find out more on the types of projects devtool can support].

What devtool cannot do

Although devtool will make a good start at generating a recipe for your project, the reciope will probably need some manual tweaking before it does eaxctly what you want. Also devtool cannot publish your recipe to the layer index.

(From the Yocto Project documentation: Here's a presentation by Trevor Woerner that, while somewhat dated, provides detailed background information and a complete working Yocto Project Developer Workflow Tutorial.)

How to get devtool

Standard build environment

If you have set up the Yocto Project build environment using the Quick Start Guide, you already have access to devtool for qemux86 image.

Extensible SDK

Alternatively you can install an Extensible SDK (eSDK). These come in two flavors; "minimal" and "full". The "minimal" eSDK is the preferred option as it comes as a small 30Mb installer and downloads the components it needs depending on requirements of application project. The installer contains the URL of an sstate mirror so it does not no need to build all dependencies from source code. The "full" eSDK contains all the components already built so is much larger (similar in size to traditional SDK, i.e. at least 1GB). The Yocto SDK Manual contains more details on installing and using the eSDK.

Currently the Yocto Project autobuilder only builds "full" eSDKs. The qemux86 full eSDK can be found here. There is a feature enhancement request for the autobuilder to create minimal eSDKs.

In the meantime, an experimental minimal qemux86 eSDK can be found here.

To install the extensible SDK, make sure it is executable and then run

poky-glibc-x86_64-core-image-minimal-i586-toolchain-ext-2.2.sh -d /path/to/esdk

Note that installation will take 7-10 minutes.
The configure build environment, run

source /path/to/esdk/environment-setup-i586-poky-linux

Create a recipe from existing source

Generate recipe

Let's say you have a C application that you want to add to your OS image. Assume repo is https://github.com/whbruce/bbexample.git and you want to name recipe (and thus the executable it creates) bbexample. The best way of doing this is use devtool

devtool add bbexample https://github.com/whbruce/bbexample.git

You will see the following output

NOTE: Creating workspace layer in /path/to/build/workspace
NOTE: Enabling workspace layer in bblayers.conf
NOTE: Using default source tree path /path/to/build/workspace/sources/bbexample
NOTE: Recipe /path/to/build/workspace/recipes/bbexample/bbexample_git.bb has been automatically created; further editing may be required to make it fully functional

Key entries here are the locations of the source code in /path/to/build/workspace/sources/bbexample. Note that git will create a branch named devtool for your work. The recipe is in /path/to/build/workspace/recipes/bbexample/bbexample_git.bb
You can now look at the generated recipe with

devtool edit-recipe bbexample

We'll take a look at what's in the recipe. Comments and whitespace have been removed for easier reading

LICENSE = "Unknown"
LIC_FILES_CHKSUM = "file://LICENSE;md5=96af5705d6f64a88e035781ef00e98a8"
SRC_URI = "git://github.com/whbruce/bbexample.git;protocol=https"
# Modify these as desired
PV = "0.1+git${SRCPV}"
SRCREV = "${AUTOREV}"
S = "${WORKDIR}/git"
inherit autotools
EXTRA_OECONF = ""

Let's go through the bitbake commands and variables in the recipe and explain what they do:

LICENSE: The type of license. This is set to unknown because devtool cannot recognize the old MIT text.
LIC_FILES_CHKSUM: devtool looks in the root of the source folder for a file called LICENSE, and generates a checksum to ensure checkout has been successful.
SRC_URI: The git repo given in the 'devtool add' command. This is where the source will be pulled from.
PV: The package version. Usually taken from recipe filename (e.g. bbexample_0.1.bb would automatically set PV = "0.1")
SRCREV: the git revision to checkout. ${AUTOREV} means HEAD and must be used during app development. It should be set a specific commit before being added to a layer.
S: Where the source can be found after being fetched and unpacked.
inherit autotools: Use a bitbake class that can configure, build and install an autotools project
EXTRA_OECONF: Options to be passed to configure script. In this case it is empty, so is not necessary.

Build

$ devtool build bbexample
Parsing recipes: 100% |#########################################| ETA:  00:00:00
Parsing of 872 .bb files complete (0 cached, 872 parsed). 1302 targets, 47 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
NOTE: Preparing RunQueue
NOTE: Checking sstate mirror object availability (for 51 objects)
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: bbexample: compiling from external source tree /path/to/esdk/workspace/sources/bbexample
NOTE: Tasks Summary: Attempted 377 tasks of which 371 didn't need to be rerun and all succeeded.

This build command runs bitbake just as in a full Yocto Project OS image build. The key line in the output is Checking sstate mirror object availability (for 51 objects). As the extensible SDK installer gives you a minimal build environment it has to download all the dependencies (e.g. native build tools and necessary target libraries). Here 51 objects have been found and are downloaded in their pre-built state greatly speeding up the build process (i.e. we just run 377 tasks of which 371 didn't need to be rerun). This is the extensible part of the SDK in action.

Deploy

Before deploying, the target must be running, have a network connection, and have an ssh server running. In this case, we'll start the qemux86 target as follows.

$ runqemu qemux86

Then we'll deploy the package

$ devtool deploy-target -s bbexample root@192.168.7.2
Parsing recipes..done.
devtool_deploy.list                           100%  167     0.2KB/s   00:00    
devtool_deploy.sh                             100% 1002     1.0KB/s   00:00    
./
./usr/
./usr/bin/
./usr/bin/bbexample
./usr/include/
./usr/include/bbexample.h
./usr/lib/
./usr/lib/libbbexample.so.1
./usr/lib/libbbexample.so
./usr/lib/libbbexample.so.1.0.0
./usr/lib/libbbexample.la
NOTE: Successfully deployed /path/to/esdk/sdk/tmp/work/i586-poky-linux/bbexample/0.1+git999-r0/image

Run

root@qemux86:~# bbexample
Hello Yocto World...
Hello World (from a shared library!)

Update

Now the application is packaged, you'll probably want to make some changes. This examples assumes that you cannot push you changes upstream so have to apply them as a patch. Let's simply update the app to output

Hello Yocto World v2...

Make the change to bbexample.c and commit with message "Changed bbexample message" and check that bbexample operates as expected. Note that if you are using devtool from the eSDK it will create temporary folders oe-logs and oe-workdir so you should configure git to ignore them.

Publish

The devtool publish command first appeared in Yocto 2.2.

Publish with Yocto 2.2 and later

Now you'll want to add your new app to a layer. In this example, we'll create a new layer called meta-example in folder /path/to, but an existing layer can also be used. First create the layer and it add it BBLAYERS. We use the yocto-layer utility. When you run it, accept all default values. The bitbake-layers script will add the later to BBLAYERS.

$ layers/poky/scripts/yocto-layer create /path/to/meta-example
$ layers/poky/bitbake/bin/bitbake-layers add-layer /path/to/meta-example

Now we move over the recipe we created, along with the patch for the source code.

$ devtool finish bbexample meta-example

This will create a recipes folder called recipes-bbexample, with a recipe called bbexample_git.bb in it. You should change the recipe name to reflect the version (e.g. bbexample_0.1.bb)

Publish with Yocto 2.1 and ealier

Update Existing Recipe

Updating a recipe is similar to creating one; however, rather than using devtool add we use the devtool modify command.

devtool modify bbexample

The edit/build/test cycle is then the same as for adding a recipe but note that the recipe is not copied into devtool's workspace but remains in its layer.

Build Image

To test the updated OS image that now includes your application, you can build it as follows. If you are in a eSDK environment, devtool will automatically build the image used to create the SDK. If in the standard build environment you must specify an image.

devtool build-image [image-name]

Output will be as follows:

Parsing recipes..done.
NOTE: Building image core-image-minimal with the following additional packages: bbexample
Loading cache: 100% |###########################################| ETA:  00:00:00
Loaded 1302 entries from dependency cache.
Parsing recipes: 100% |#########################################| Time: 00:00:00
Parsing of 872 .bb files complete (870 cached, 2 parsed). 1302 targets, 49 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.30.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.1"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "krogoth:f7b994b75261550faa3ccf9005fc174950c7bee9"
workspace         = "<unknown>:<unknown>"

NOTE: Preparing RunQueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
bbexample-0.1+git999-r0 do_compile: NOTE: bbexample: compiling from external source tree /path/to/build/workspace/sources/bbexample
NOTE: Tasks Summary: Attempted 2062 tasks of which 2046 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/build/tmp/deploy/images/qemux86