Combo-layer

Introduction

combo-layer is a command-line utility for creating and maintaining git repositories that are a combination of several other git repositories. It was created to help stitch together multiple layers on top of OE-Core, in particular to manage the Poky repository, although there is nothing specific in the code about OE layers.

It works by extracting the changes from each component repository as patches, modifying them in a customisable way, and then applying them to the combo repository. It can handle multiple branches.

Note: because of the patch-shipping method combo-layer uses, it does not work well with merge commits. Typically with OE-Core and most other layers we avoid merges (i.e. branches are rebased before applying) so this is not a significant limitation for us.

Why not use other similar tools (repo, submodules, subtree)?

combo-layer performs a similar function to other available tools; however each of these has their own limitations and none of them allows modification of changes as they pass through (either manually or automatically). Additionally combo-layer allows you to take just part of a repository, i.e. one or more subdirectories or files.

Setup

Note: the following assumes OE-Core/Poky's scripts subdirectory is in your PATH (as it would be after sourcing oe-init-build-env); however this is not mandatory - you can just run it directly e.g. ../../oe-core/scripts/combo-layer ....

1. Create an empty directory to hold your combo repository and cd into it
2. Create a configuration file that tells combo-layer about the component repositories. You can copy the example config file from the scripts subdirectory and modify it. By default, combo-layer expects to find this config file at conf/combo-layer.conf from the current directory, however it can be anywhere if you use the -c option. If the configuration file is within the combo repository and not set to be ignored, it will be committed if changed.
3. Run combo-layer init - this will fetch the component repos and create a combo repo based on the current revision on the specified branches for each component.
4. You can now make any desired changes to the initial state of the combo repository, and when you're ready use git commit to commit it.

Updating

Updating is as easy as combo-layer update - this will pull each component repository, prepare patches for the changes since the last update, and then apply them to the combo repository. If desired, you can specify -i to use interactive mode which allows you to edit the list of patches and the patches themselves before they are applied.

Advanced configuration

Hook script

A script can be specified for each component to be called automatically for each patch before it is applied using the hook option in the configuration file. The script is called with three arguments: the patch file, the revision from the component repo, and the component name in that order. There is an example hook script in scripts/combo-layer-hook-default.sh which will add the component name and upstream revision to the commit message and prefix the first line (shortlog) with the component name.

Split local configuration

It is possible to split out the local parts of the configuration (e.g. local_repo_dir and last_revision) so that the generic parts within the config file can be committed to the combo repository and the local parts (and last_revision which changes often) remain local. To set this up:

1. Create a file with the same name and location as the main config file except with '.conf' replaced with '-local.conf'. Thus, if you're using the default path/name of conf/combo-layer.conf, the local counterpart would be conf/combo-layer-local.conf.
2. Move the local configuration options from the main config file to the local one. The last_revision option should be within a section named using the component name and combo layer branch name separated by a | (pipe) character. Other options need not use this style.
3. If the configuration files are within the combo repository, add the local conf file to .gitignore and commit it.

An example local configuration file (values may be artificial):

[bitbake]
local_repo_dir = ../repos/bitbake
   
[oe-core]
local_repo_dir = ../repos/oe-core
   
[bitbake|master]
last_revision = db689a99beffea1a285cdfc74a58fe73f1666987
   
[oe-core|master]
last_revision = 121a1499a81706366acc0081272a6bff634d4d62
   
[bitbake|denzil]
last_revision = 24b631acdaa143a4de39c6e1328849660c66f219
   
[oe-core|denzil]
last_revision = 741146fa90f28f7ce8d82ee7f7e254872d519724

Python code in configuration

If a value in the configuration file starts with @ it will be assumed to be a python statement to be evaluated. For example, you could use it to read the value of an environment variable:

[bitbake]
...
local_repo_dir = @os.getenv("LOCAL_REPO_DIR") + "/bitbake"
...