TipsAndTricks/KernelDevelopmentWithEsdk

Overview

This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons

1. It contains the target toolchain which is not easily accessible in the bitbake environment
2. It contains devtool which gives you an easy way of getting the kernel source your target is using (if using linux-yocto or linux-intel)
3. devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process

1. Build or download an eSDK for your platform
2. Install and use that eSDK to build your kernel

Build Extensible SDK

Open a new terminal window, we'll refer to this terminal as your bitbake terminal. First we'll checkout poky and configure bitbake environment.

$ git clone -b pyro git://git.yoctoproject.org/poky 
$ source poky/oe-init-build-env

Now we need to do some configuration for the Minnowboard image.

• Set MACHINE (as default is qemux86)
• Enable kernel modules (disabled by default for minimal images)

Add the following to conf/local.conf

MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"

We also need to create a layer to take the kernel patches we'll be creating. You can accept the defaults when you run the "yocto-layer" command.

$ yocto-layer create my-kernel -o ../meta-my-kernel
$ bitbake-layers add-layer ../meta-my-kernel

Now build an extensible SDK installer for the Minnowboard in the bitbake terminal

$ bitbake core-image-minimal -c populate_sdk_ext 

The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/

./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that includes the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [here]. If you downloaded the installer, make it executable.

Install Extensible SDK

Run the installer as follows. If you do not want to use the default ~/poky_sdk directory for installation of the toolchain, use the -d option to specify a destination.

$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
 $ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux

Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

Setup your ESDK build environment (ESDK terminal)

YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL. Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.

$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this

WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."

You didn't open a new terminal!

Build initial image with ESDK

In the ESDK terminal, build the image

$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64

Now flash to image to USB stick, assuming USB stick is on /dev/sdd

$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync

Boot Minnowboard with this image and check all is OK.

Working with Yocto kernel

Checkout kernel source

First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.

$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto

Edit and build driver

Kernel source will be in the location shown at end of the devtool modify virtual/kernel output. Let's make a trivial change to the serial driver in drivers/tty/serial/8250/8250_core.c. After the line

pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %sabled\n",
         nr_uarts, share_irqs ? "en" : "dis");

Add the line

pr_info("Serial: 8250/16550 driver modified with eSDK");

To build the updated kernel source, in the SDK terminal run make in your kernel source folder. Adjust the -j option to match number of cores on your workstation.

$ make -C workspace/sources/linux-yocto -j4

Create image with new kernel

Normally you'd make a new image with devtool build-image but this can take a minute or so and messes with the kernel source folder. A faster option is to use kpartx to splice the new kernel into the image you have already built. First make a copy of your wic file in say /tmp

$ cp tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic /tmp

Then create loopback devices for each partition in wic file

$ sudo kpartx -v -a /tmp/core-image-minimal-genericx86-64.wic
add map loop3p1 (253:6): 0 47446 linear /dev/loop3 2048
add map loop3p2 (253:7): 0 119356 linear /dev/loop3 51200
add map loop3p3 (253:8): 0 90112 linear /dev/loop3 170556

Kernel is in the first device, so let's mount /dev/mapper/loop3p1 to take a look

$ sudo mkdir /mnt/wic-p1
$ sudo mount /dev/mapper/loop3p1 /mnt/wic-p1

Now copy over new kernel

$ sudo cp workspace/sources/linux-yocto/arch/x86/boot/bzImage /mnt/wic-p1

Then unmount and unkaprtx...

$ sudo umount mnt/wic-p1
$ sudo kaprtx -d /dev/loop3

Now flash image

$ dd if=/tmp/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync

Boot Minnowboard, log on and look in log for driver message

# root@genericx86-64:~# dmesg | grep Serial:
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
Serial: 8250/16550 driver modified with eSDK

It worked!

Export patches and create a bbappend file

To export your commits as patches and a bbappend file use the following command in the ESDK terminal. We'll target the "my-kernel" layer we created in the bitbake terminal

$ devtool finish linux-yocto /path/to/meta-my-kernel

The patches and the bbappend can be found in /path/to/meta-my-kernel/recipes-kernel/linux.

Working with upstream kernel

Slightly different workflow where we use a kernel from kernel.org.

Content TBD

Build image with your modified kernel

You can now build an image which will include your kernel patches. Execute the following command from your build directory in the bitbake terminal

bitbake core-image-minimal