BKM: starting a new BSP

This is a practical 'quick start' on creating a new BSP. For detailed background and instructions on how to to formally do this, please see the BSP Developer's Guide and the Kernel Manual. This document is essentially a pragmatic distillation of those documents but oriented toward the mechanics of quickly getting the initial BSP infrastructure in place for a minimally functional new BSP.

Mostly it's a straightforward exercise:

• copy a meta-* subdirectory representing the metadata of the BSP
• add it as a layer and configure the build system to use it
• add or tweak the machine configuration and recipes in the layer to get things working
• create a specialized branch for the BSP in the kernel and add kernel metadata to customize it for the BSP

The final step above (dealing with the kernel) is actually the most time-consuming and least straightforward aspect of not only creating the initial BSP, but of maintaining it going forward, so much of this document will be dedicated to addressing that aspect of BSP creation/maintenance.

Starting Out

As mentioned in the BSP Developer's Guide, the easiest way to start a new BSP is to base it on a similar already-existing working BSP. For this guide, we'll essentially go through the steps used to create the BSP for the Fish River Island, which exists in its current form as meta-fishriver in the meta-intel repository.

For this particular BSP, a bit of background on the base hardware and its relationship with some similar Intel hardware, the Crown Bay platform, will be useful when describing how the fishriver BSP was created. For the purposes of this guide, the relevant components of the Crown Bay platform that we care about are the 'EG20T platform controller hub' (Topcliff) which provides 'platform' components such as USB hardware and a gigabit ethernet adapter, etc, and the processor, which in the case of the Crown Bay platform, is the E660 Atom 'Tunnel Creek' processor with on-board graphics. The Fish River Island I is also based on the Topcliff, but instead of a Tunnel Creek processor, it uses a Z530 Atom 'eMenlow' processor. The Fish River Island II will make use of the Tunnel Creek processor instead of the eMenlow; the relevant aspect of this last fact relative to BSP development relates to the on-board graphics of these chips, in that both the Crown Bay and Fish River Island II BSPs can use the same configuration of the Intel EMGD graphics driver support to drive the video subsystems, whereas the Fish River Island I uses a simpler graphics setup based on VESA graphics.

So, from the perspective of the BSP developer, the important facts are:

• both the Crown Bay and Fish River Island I/II use the EG20T and can share support for this
• both the Crown Bay and Fish River Island II use the same graphics configuration and can share support for EMGD
• the Fish River Island I uses different graphics from the Crown Bay, so can't share anything with respect to graphics

Since this guide only really covers the Fish River Island I, we won't go into any more detail on anything related to the Fish River Island II - in addition to the above similarities, it adds a set of additional components to justify its existence as a separate BSP - adding that is left as an exercise for the reader ;-)

Before we start, we need to do a basic checkout of the Yocto sources. For this example, we'll assume you've done so by following the instructions in the following tutorial, Transcript: from git checkout to meta-intel BSP, but only up to and including the step starting with 'Now check out the meta-intel repo into the top-level Yocto repo dir'.

So the first step for the Fish River BSP would be to take a look at what's already available in meta-intel, and use one of the existing BSPs as a starting point.

Here's a list of the current set of BSPs available in the meta-intel repository:

trz@copacabana:~/yocto/dev/meta-intel$ ls -al
total 52
drwxr-xr-x 11 trz trz 4096 2011-06-24 17:39 .
drwxr-xr-x 13 trz trz 4096 2011-06-24 17:38 ..
drwxr-xr-x  5 trz trz 4096 2011-06-24 17:39 common
drwxr-xr-x  8 trz trz 4096 2011-06-24 17:39 .git
-rw-r--r--  1 trz trz 1854 2011-06-24 17:39 MAINTAINERS
drwxr-xr-x  8 trz trz 4096 2011-06-24 17:39 meta-crownbay
drwxr-xr-x 10 trz trz 4096 2011-06-24 17:39 meta-emenlow
drwxr-xr-x  8 trz trz 4096 2011-06-24 17:39 meta-jasperforest
drwxr-xr-x  8 trz trz 4096 2011-06-24 17:39 meta-n450
drwxr-xr-x  8 trz trz 4096 2011-06-24 17:39 meta-sugarbay
-rw-r--r--  1 trz trz   71 2011-06-24 17:39 README

For us, since we already know the answer, the obvious starting point would be the meta-crownbay BSP. Here's a listing of the files contained within the meta-crownbay 'layer':

trz@copacabana:~/yocto/dev/meta-intel/meta-crownbay$ find .
.
./conf
./conf/machine
./conf/machine/crownbay.conf
./conf/machine/crownbay-noemgd.conf
./conf/layer.conf
./recipes-kernel
./recipes-kernel/linux
./recipes-kernel/linux/linux-yocto_2.6.37.bbappend
./recipes-kernel/linux/linux-yocto_2.6.34.bbappend
./recipes-core
./recipes-core/tasks
./recipes-core/tasks/task-core-tools.bbappend
./README
./recipes-bsp
./recipes-bsp/formfactor
./recipes-bsp/formfactor/formfactor
./recipes-bsp/formfactor/formfactor/crownbay
./recipes-bsp/formfactor/formfactor/crownbay/machconfig
./recipes-bsp/formfactor/formfactor/crownbay-noemgd
./recipes-bsp/formfactor/formfactor/crownbay-noemgd/machconfig
./recipes-bsp/formfactor/formfactor_0.0.bbappend
./recipes-graphics
./recipes-graphics/xorg-xserver
./recipes-graphics/xorg-xserver/xserver-xf86-config
./recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay
./recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay/xorg.conf
./recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay-noemgd
./recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay-noemgd/xorg.conf
./recipes-graphics/xorg-xserver/emgd-driver-bin-1.6
./recipes-graphics/xorg-xserver/emgd-driver-bin-1.6/.gitignore
./recipes-graphics/xorg-xserver/xserver-xf86-emgd-bin
./recipes-graphics/xorg-xserver/xserver-xf86-emgd-bin/.gitignore
./recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend
./recipes-graphics/xorg-xserver/emgd-driver-bin_1.6.bb
./README.sources
./binary
./binary/.gitignore
./COPYING.MIT

Since meta-crownbay is the most similar BSP to the new BSP we want to create, we'll simply copy the contents of meta-crownbay into a new directory for our BSP, meta-fishriver, and rework the contents into something useful for our new BSP.

In particular, for the Fish River Island I, we don't need anything to do with EMGD, so we remove all the emgd-related recipes and other files. We do, however, keep the xorg.conf contained in the crownbay-noemgd subdir (but rename the subdir for our new bsp) and we also keep the crownbay-noemgd machconfig, again with an appropriate rename. The initial set of files for the fish river island BSP then becomes:

trz@copacabana:~/yocto/dev/meta-intel$ cp -a meta-crownbay meta-fishriver
delete what we don't want, rename what we do, etc
trz@copacabana:~/yocto/dev/meta-intel/meta-fishriver$ find .
.
./conf
./conf/machine
./conf/machine/fishriver.conf
./conf/layer.conf
./recipes-kernel
./recipes-kernel/linux
./recipes-kernel/linux/linux-yocto_2.6.37.bbappend
./recipes-kernel/linux/linux-yocto_2.6.34.bbappend
./recipes-core
./recipes-core/tasks
./recipes-core/tasks/task-core-tools.bbappend
./README
./recipes-bsp
./recipes-bsp/formfactor
./recipes-bsp/formfactor/formfactor
./recipes-bsp/formfactor/formfactor/fishriver
./recipes-bsp/formfactor/formfactor/fishriver/machconfig
./recipes-bsp/formfactor/formfactor_0.0.bbappend
./recipes-graphics
./recipes-graphics/xorg-xserver
./recipes-graphics/xorg-xserver/xserver-xf86-config
./recipes-graphics/xorg-xserver/xserver-xf86-config/fishriver
./recipes-graphics/xorg-xserver/xserver-xf86-config/fishriver/xorg.conf
./recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend
./README.sources
./binary
./binary/.gitignore
./COPYING.MIT

One additional task needed before we attempt building the new BSP is to slightly change the kernel configuration by modifying the contents of recipes-kernel/linux/linux-yocto_2.6.37.bbappend. For this BSP, we'll remove the lines having to do with crownbay, but keep the ones having to do with crownbay-noemgd (replacing 'crownbay-noemgd' with 'fishriver'):

COMPATIBLE_MACHINE_fishriver = "fishriver"
KMACHINE_fishriver  = "yocto/standard/crownbay"
KERNEL_FEATURES_append_fishriver += " cfg/smp.scc"

Note that the above changes leave the KMACHINE branch as "yocto/standard/crownbay". For the initial build and boot of the BSP, we'll simply use the crownbay machine branch in the kernel, which is sufficient for initial work (later we'll create a dedicated machine branch for fishriver).

Finally, you need to make a couple changes to actually use the new layer in a build. First, you need to add the new layer to build/conf/bblayers.conf:

BBLAYERS = " \
 /usr/local/src/yocto/crownbay-emgd-1.6/meta \
 /usr/local/src/yocto/crownbay-emgd-1.6/meta-yocto \
 /usr/local/src/yocto/crownbay-emgd-1.6/meta-intel/meta-fishriver \
 "

Then, you need to specify the new machine in build/conf/local.conf:

MACHINE ??= "fishriver"

At this point, you should be able to build and boot a fully-functional graphical fishriver BSP image. If not, you'll need to go through the standard cycle of recipe debugging to get things to work. Similarly, if you need to add or modify capabilities to the image, it would be via the standard recipe development/debug cycle, which this document doesn't cover, there being copious amounts of documentation and wisdom in that area already existent elsewhere.

What this guide will discuss more is the kernel and modifications to the kernel recipe and kernel-related components of the BSP.

Creating a new kernel branch for the BSP

Up to this point, all we've done is create the build system infrastructure for the new fishriver BSP. It still uses the same kernel as the BSP it was based on. If we want to customize the kernel differently in any way from the BSP it was based on, we need to create a new machine branch for it, and add new BSP-specific information in the 'meta' branch to go along with it.

To do this, let's first create a 'bare' clone of the current linux-yocto kernel repository (the Yocto kernel tools require using a bare clone):

$ git clone --bare git://git.yoctoproject.org/linux-yocto-2.6.37 linux-yocto-2.6.37.git

Note the branches 'yocto/standard/crownbay' and 'yocto/emgd' in the linux-yocto-2.6.37 repo:

$ git branch -a
* master
  meta
  yocto/base
  yocto/eg20t
  yocto/emgd
  yocto/gma500
  yocto/standard/arm-versatile-926ejs
  yocto/standard/base
  yocto/standard/beagleboard
  yocto/standard/common-pc-64/base
  yocto/standard/common-pc-64/jasperforest
  yocto/standard/common-pc-64/romley
  yocto/standard/common-pc-64/sugarbay
  yocto/standard/common-pc/atom-pc
  yocto/standard/common-pc/base
  yocto/standard/crownbay
  yocto/standard/emenlow
  yocto/standard/fsl-mpc8315e-rdb
  yocto/standard/mti-malta32-be
  yocto/standard/mti-malta32-le
  yocto/standard/preempt-rt/base
  yocto/standard/preempt-rt/common-pc
  yocto/standard/preempt-rt/common-pc-64
  yocto/standard/qemu-ppc32
  yocto/standard/routerstationpro

What we want is a new branch, 'yocto/standard/fishriver', which we'll base on the current 'yocto/standard/crownbay'. To do this, we first create a local clone of the bare clone:

$ git clone linux-yocto-2.6.37.git linux-yocto-2.6.37

and in the new clone simply check out 'yocto/standard/crownbay' as 'yocto/standard/fishriver':

$ git checkout -b yocto/standard/fishriver remotes/origin/yocto/standard/crownbay

You should now be able to see the new branch:

$ git branch -a
  master
* yocto/standard/fishriver
  remotes/origin/HEAD -> origin/master

Now that you have a new fishriver branch, you need to push it to the bare clone (which we'll set the kernel tools up to use in a later step):

$ git push origin yocto/standard/fishriver2:yocto/standard/fishriver2
  Total 0 (delta 0), reused 0 (delta 0)
  To /home/trz/work/linux-yocto-2.6.37.git
  * [new branch]      yocto/standard/fishriver -> yocto/standard/fishriver

You should now be able to see the new branch in the bare clone:

$ git branch -a
* master
  meta
  yocto/base
  yocto/eg20t
  yocto/emgd
  yocto/gma500
  yocto/standard/arm-versatile-926ejs
  yocto/standard/base
  yocto/standard/beagleboard
  yocto/standard/common-pc-64/base
  yocto/standard/common-pc-64/jasperforest
  yocto/standard/common-pc-64/romley
  yocto/standard/common-pc-64/sugarbay
  yocto/standard/common-pc/atom-pc
  yocto/standard/common-pc/base
  yocto/standard/crownbay
  yocto/standard/emenlow
  yocto/standard/fishriver
  yocto/standard/fsl-mpc8315e-rdb
  yocto/standard/mti-malta32-be
  yocto/standard/mti-malta32-le
  yocto/standard/preempt-rt/base
  yocto/standard/preempt-rt/common-pc
  yocto/standard/preempt-rt/common-pc-64
  yocto/standard/qemu-ppc32
  yocto/standard/routerstationpro

You now have a 'machine' branch dedicated to your BSP. The next step is to add BSP-specific data to the 'meta' branch of the kernel repository. First, check out the meta branch:

$ git checkout -b meta-fishriver remotes/origin/meta

That will give you a new 'meta' directory in the kernel filesystem. The subdirectory within that that we're interested in is cfg/kernel-cache:

meta/cfg/kernel-cache$ ls
 00-README  arch  bsp  cfg  features  ktypes  kver  patches  scripts  small

Within that, for the time being, we're interested in the 'bsp' directory, which is where we'll add our initial kernel infrastructure for our new BSP:

cfg/kernel-cache/bsp$ ls
 arm-versatile-926ejs  beagleboard  common-pc-64  emenlow    fsl-mpc8315e-rdb  mti-malta32  romley            sugarbay
 atom-pc               common-pc    crownbay      fishriver  jasperforest      qemu-ppc32   routerstationpro

Again, since we're basing the new BSP directly on crownbay, we'll simply re-use the crownbay infrastructure:

meta/cfg/kernel-cache/bsp$ cp -a crownbay fishriver

Here are the contents of the new fishriver directory before any changes:

meta/cfg/kernel-cache/bsp/fishriver$ ls -al
total 24
drwxr-xr-x  2 trz trz 4096 2011-06-27 18:49 .
drwxr-xr-x 18 trz trz 4096 2011-06-28 07:24 ..
-rw-r--r--  1 trz trz 1400 2011-06-27 18:49 crownbay.cfg
-rw-r--r--  1 trz trz  362 2011-06-27 18:49 crownbay.scc
-rw-r--r--  1 trz trz  122 2011-06-27 18:49 crownbay-standard.scc
-rw-r--r--  1 trz trz  656 2011-06-27 18:49 eg20t.cfg

We'll rename these as needed, but will focus on the changes needed to the file contents to make them work for fishriver.

Starting at the 'top', we'll make the changes we need first to crownbay-standard.scc after renaming it to fishriver-standard.scc. Here are the contents without any changes:

define KMACHINE crownbay
define KTYPE standard
define KARCH i386

scc_leaf ktypes/standard crownbay

include crownbay.scc

Here we can see several fields that need to be changed, most importantly KMACHINE and scc_leaf. Scc_leaf is the most important – the definition provided here allows the kernel tools to find the ‘top-level’ of the kernel configuration for the machine, which allows it to start the process of creating a final .config for the kernel from the complete set of kernel config fragments up the inheritance chain for the BSP, given the machine name specified here along with the kernel type. So making these changes, what we end up with is this:

define KMACHINE fishriver
define KTYPE standard
define KARCH i386

scc_leaf ktypes/standard fishriver

include fishriver.scc

Notice that we've also changed the last line to include fishriver.scc, which is just crownbay.scc renamed to fishriver.scc:

cfg/kernel-cache/bsp/fishriver$ mv crownbay.scc fishriver.scc
cfg/kernel-cache/bsp/fishriver$ cat fishriver.scc

kconf hardware crownbay.cfg
kconf hardware eg20t.cfg

git merge yocto/emgd

include features/intel-e1xxxx/intel-e1xxxx.scc
include features/drm-emgd/drm-emgd.scc
include features/dmaengine/dmaengine.scc
include features/serial/8250.scc

include features/logbuf/size-normal.scc

include features/latencytop/latencytop.scc
include features/profiling/profiling.scc

fishriver.scc is actually where the fishriver-specific configuration gets pulled together. The first two lines include straight (though extensive) 'config fragments' directly - these are just copied directly into the final .config. For example, crownbay.cfg contains things like:

CONFIG_X86_32=y
CONFIG_MATOM=y
CONFIG_PRINTK=y
.
.
.

And eg20t.cfg contains egt20t-specific config fragments:

# Basic hardware support for the box - network, USB, PCI, sound
CONFIG_NETDEVICES=y
CONFIG_ATA=y

# Hardware support for the Platform Controller Hub EG20T

CONFIG_PCH_DMA=y
CONFIG_PCH_UART_DMA=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_PCH=y
CONFIG_SERIAL_8250_PCH_DMA=y
.
.
.

NOTE: Currently the eg20t.cfg is copied a couple places in the tree, which is of course suboptimal; the intent is to add a 'feature' for this which along with the (currently unused) eg20t branch would encapsulate the eg20t patches and configuration for easy re-use.

The other items in this file do make use of encapsulated features that result in other config fragments corresponding to those features being added to the final .config.

The final important piece of the configuration for this BSP is the line:

git merge yocto/emgd

which, combined with the 'drm-emgd' feature:

include features/drm-emgd/drm-emgd.scc

adds emgd support to the BSP.

NOTE: the fishriver BSP actually removes this line, but the fishriver2 would add it back in; in any case, it illustrates an important concept so we'll cover it here anyway.

If you look at the branch listing of the kernel repo, you'll see a 'feature branch' named 'yocto/emgd':

$ git branch -a
  master
* meta0
  yocto/standard/fishriver2
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
  remotes/origin/meta
  remotes/origin/remotes/origin/yocto/standard/common-pc-64/base
  remotes/origin/yocto/base
  remotes/origin/yocto/eg20t
  remotes/origin/yocto/emgd
  remotes/origin/yocto/gma500
  remotes/origin/yocto/standard/arm-versatile-926ejs

If you get a log listing, you'll see a series of patches at the top which have been added to yocto/base and which implement the kernel support for EMGD:

$ git log remotes/origin/yocto/emgd

commit 2264cdc31a6e222fb65caad042505934e156b7ad
Author: Tom Zanussi <tom.zanussi@intel.com>
Date:   Wed May 11 09:05:59 2011 -0500

   emgd: backfix for 2.6.37
   
   Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>

commit fc264f1dfb3b17f0239e08ad7c3c0f98b09fd5c2
Author: Tom Zanussi <tom.zanussi@intel.com>
Date:   Fri May 6 22:50:20 2011 -0500

   yocto/emgd: 2.6.39 fixes
   
   Fixes required for migration to 2.6.39.
   
   Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>

commit 7946104c07ac52a7e5816d2156f9b6f8ed09dbfa
Author: Tom Zanussi <tom.zanussi@intel.com>
Date:   Tue May 3 16:56:56 2011 -0500

   yocto/emgd: build fixups
   
   Add emgd config option (DRM_EGD) and modify Makefiles for in-tree
   builds.
   
   Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>

commit eab1e2634aece1f89cdc7b7b95364c54a532a2f3
Author: Tom Zanussi <tom.zanussi@intel.com>
Date:   Tue May 3 16:44:37 2011 -0500

   yocto/emgd: emgd 1.61 driver

When the kernel tools actually process the .scc files to produce the final build kernel, the command 'git merge yocto/emgd' causes the code (patch commits) in the yocto/emgd feature branch to be merged with the yocto/standard/fishriver branch to produce the branch that actually gets built.

In order to enable the code from the merged emgd branch, we need a few matching config options. These are defined by a couple files in the features/drm-emgd directory, a one-line .scc file and .cfg file containing the EMGD kernel config fragment:

$ cat meta/cfg/kernel-cache/features/drm-emgd/drm-emgd.cfg

CONFIG_DRM=y
CONFIG_DRM_EGD=m
CONFIG_DRM_KMS_HELPER=m

Once we fix up all these files by replacing 'crownbay' with 'fishriver' everywhere, we're ready to check it all in and push it to our bare clone's meta branch:

$ git add meta/cfg/kernel-cache/bsp/fishriver
$ git commit -a -s
$ git push origin meta-fishriver:meta

At this point, the kernel should have everything needed in order to build using the new fishriver branch. We do of course need to remember to actually change our kernel .bbappend to point to the new branch, so go back to recipes-kernel/linux/linux-yocto_2.6.37.bbappend and make sure it looks like this:

COMPATIBLE_MACHINE_fishriver = "fishriver"
KMACHINE_fishriver  = "yocto/standard/fishriver"
KERNEL_FEATURES_append_fishriver += " cfg/smp.scc"

There is one bit of admistrativia we need to configure first, though, in order to have the build actually use our local changes.