Building a swupd-based OS

This guide will help introduce the Software Updater (swupd) and how meta-swupd can be used to develop a custom Linux-based OS with swupd updates using the Yocto Project tools.

What is swupd?

The SoftWare UPDater — swupd — from Clear Linux provides a new way of adding functionality to and updating a Linux-based OS.

swupd uses binary-delta technology to efficiently update only the files that have changed between OS updates. This means that updates are small, resulting in fast downloads, and fast to apply.

Functionality is added to the OS via bundles, rather than individual packages, which compose a set of projects into a logical unit which can be added or removed from the OS wholesale.

swupd concepts

swupd encourages a certain way of constructing a Linux-based OS, via the following concepts:

Bundles

Bundles in swupd are used to define logical units of functionality. All swupd-based operating systems will define at least os-core bundle, whilst most will also define several other bundles containing complete sets of functionality composed of one or more packages generated by the build system.

Immutable os-core

Building on the bundle concept, swupd requires that the base operating system is immutable, enabling software deployed on the system to rely on the base OS not changing unexpectedly.

Therefore files which are included in the os-core bundle must be the final version of a file which supports all bundles that may be installed on top of it. Adding a bundle to the OS must not modify a file provided by os-core. Most notably this means that the passwd and groups should contain all potential account information, regardless of whether or not these users and groups are utilised by the base OS.

Single, atomic, OS-wide version number

swupd based operating systems atomically upgrade the entire OS from one version to the next, rather than upgrading at a more granular, package-based, level. This means that the entire contents of the OS can be determined from a single version number.

Stateless OS

Clear Linux, the Linux-based OS which hosts swupd development, is a stateless OS — that is it comes with an empty /etc by default and all default OS configuration is enabled until overridden by per-system configuration of files in /etc.

Due to this swupd by default will not update files /etc under the assumption that these are local configuration and the default configuration is provided by read-only files in other parts of the directory hierarchy in os-core.

The swupd way

The concepts described above lead to a swupd using OS being constructed a certain way which provides several advantages to the OS developer, not least of all the reduces test matrix through a much smaller set of combinations of software that might be installed on the OS.

References

For more details see the Clear Linux Project's documentation: * Bundles overview. * About software update * Stateless feature description

OpenEmbedded/Yocto Project considerations

Several of the concepts in the swupd way contradict the default configuration of operating systems built with the Yocto Project tools and OpenEmbedded.

When building a distribution with swupd the OS developer should be aware of the following requirements:

Version number

As the OS version number is atomic and identifies an entire OS build we generate a new set of swupd update artefacts only when the OS_VERSION variable is changed (by default refusing to write over an existing set of generated artefacts with the same version number).

swupd-client checks the VERSION_ID in the the os-release file when checking for updates and we need to be certain that this is changed when building an OS update for deployment, to that end the swupd-image class automatically writes the OS_VERSION to the VERSION_ID field in the os-release file on each image creation.

update-alternatives

As the os-core bundle must provide an unchanging (at least within the same OS version) which other bundles can't modify we must be wary of recipes which use update-alternatives.

Due to the way meta-swupd constructs the swupd-based OS and the resulting images and update stream artefacts we can find ourselves in a situation where the symlink for an alternative is installed pointing to a file which doesn't exist in the os-core (because it's provided by a bundle).

It is recommended that the OS developer carefully review all uses of update-alternatives in their metadata and ensure that:

the ALTERNATIVE_PRIORITY is set appropriately so that the desired provider of an ALTERNATIVE_LINK_NAME is correctly chosen and consistent across bundles.
the highest ALTERNATIVE_PRIORITY provider for an ALTERNATIVE_LINK_NAME is included in the os-core bundle.

To help detect the dangling symlink scenario the swupd-image class includes a mechanism to check for such dangling symlinks in a constructed image, enable it by adding the swupd_check_dangling_symlinks sanity check to SWUPD_IMAGE_SANITY_CHECKS i.e.:

SWUPD_IMAGE_SANITY_CHECKS += " \
     swupd_check_dangling_symlinks \
"

useradd

Another aspect of the immutable os-core bundle is that the account information in the passwd and group files installed in the os-core bundle must include user and group information required for all bundles provided by the OS.

We would also recommend the use of useradd-staticids to ensure that the uid and gid are consistent across rebuilds so that os-core has better reproducibility.

See the Yocto Project Reference Manual section on useradd*.bbclass for more details.

NOTE: due to the stateless nature of swupd group and passwd will not be updated, for more information see the stateful section below.

Stateful

Linux-based OS built with OpenEmbedded and the Yocto Project Tools are stateful OS, that is the default configuration is provided by files in /etc and without extra work on top of the default packages the OS won't boot without a populated /etc. This contradicts the swupd way and means that currently any files a package installs into /etc won't be updated between OS version.

Unfortunately this means the solution in meta-swupd does not support adding users or groups in future updates. It's possible to solve this on a per-OS basis by implementing custom swupd helper scripts (as described below), we are investigating methods to solve this within meta-swupd by patching swupd-client (see Yocto #9199) as well as investigating providing easier support for developing stateless OS with OpenEmbedded/Yocto Project tools.

swupd-client checks the VERSION_ID in the the os-release file when checking for updates. As this file is usually installed in /etc we include a bbappend which instead creates this file in /usr/lib/ (a location swupd-client prefers) and symlink it to /etc.

NOTE: once a version of swupd-client which supports updating configuration files in /etc has been developed it will be possible to deploy support for additional users and groups to an existing swupd-based OS in a 2-step fashion: first deploy an update which includes the new version of swupd-client, after which any future updates will be able to include modified configuration files.

swupd helper scripts

swupd-client tries to call out to certain helper scripts during an update: * before the update clr_pre_update.sh is called * after the update systemdboot_updater.sh and kernel_updater.sh are called * the systemd update-triggers.target is restarted once the update is complete

The oe-swupd-helpers recipe provides skeletal implementations of the scripts (clr_pre_update.sh, systemdboot_updater.sh and kernel_updater.sh) which simply print out that they have been called, these are clearly insufficient for a production environment and should be replaced with OS-specific implementations.

The systemd units in oe-swupd-helpers are a little more useful, being based on the units used by the Clear Linux project, but likely need tweaking and adding to in order to better accommodate the OS being deployed.

Quick steps

add the meta-swupd layer to bblayers.conf
designate an image recipe as the base OS image (os-core, in swupd parlance) and inherit swupd-image in that recipe
ensure the OS_VERSION variable is assigned an integer value and that this number is increased before each build which should generate swupd update artefacts
Vet the os-core bundle to address the considerations documented above:

review any included recipes that inherit update-alternatives and ensure that the desired provider of a binary is correctly chosen and consistent across bundles with the highest priority provider being included in the os-core bundle. (we recommend the use of the swupd_check_dangling_symlinks to help catch this issue at image construction time)
ensure all users and groups required by all bundles in the OS are defined in the os-core bundle (we recommend the use of useradd-staticids).

Implement appropriate versions of the scripts and units in oe-swupd-helpers
(optional) Define additional bundles for any features you want to add assign a list of bundle names to SWUPD_BUNDLES and for each named bundle, assign a list of packages for which their content should be included in the bundle to a varflag of BUNDLE_CONTENTS which matches the bundle name i.e:
```
SWUPD_BUNDLES = "feature_one feature_two"
BUNDLE_CONTENTS[feature_one] = "package_one package_three package_six"
```
NOTE: beware of reserved bundle names, the names full and mega are reserved and cannot be used for bundle names.
TODO mention optional images generated from os-core + additional bundles via SWUPD_IMAGES

Control variables

Several variables can be set to tune the way swupd-image works:

SWUPD_GENERATE — if set to 0 i.e. SWUPD_GENERATE = "0" swupd update artefact processing will be skipped but all tasks of the swupd-image class will be executed. This is useful both for debugging the swupd-image class and in a scenario where it might be desirable to generate the chroot-like bundle directories without performing an processing with swupd.
SWUPD_DELTAPACKS — if set to 0 i.e. SWUPD_DELTAPACKS="0" swupd delta-packs will not be generated.
SWUPD_N_DELTAPACKS — the number of previous releases against which to generate delta-packs, defaults to 2.
SWUPD_VERSION_STEP — Amount the OS_VERSION should be increased by for each release. Used by the delta pack looping to generate delta packs going back up to SWUPD_N_DELTAPACK releases.
SWUPD_IMAGES TODO

Using swupd client

TODO

Meta-swupd

Contents