Meta-swupd
This page provides preliminary documentation for using the meta-swupd layer.
Building a swupd-based OS
This guide will help introduce the Software Updater (swupd) and how meta-swupd can be used to develop a custom Linux-based OS with swupd updates using the Yocto Project tools.
What is swupd?
The SoftWare UPDater — swupd — from Clear Linux provides a new way of adding functionality to and updating a Linux-based OS.
swupd uses binary-delta technology to efficiently update only the files that have changed between OS updates. This means that updates are small, resulting in fast downloads, and fast to apply.
Functionality is added to the OS via bundles, rather than individual packages, which compose a set of projects into a logical unit which can be added or removed from the OS wholesale.
swupd concepts
swupd encourages a certain way of constructing a Linux-based OS, via the following concepts:
Bundles
Bundles in swupd are used to define logical units of functionality. All swupd-based operating systems will define at least os-core bundle, whilst most will also define several other bundles containing complete sets of functionality composed of one or more packages generated by the build system.
Immutable os-core
Building on the bundle concept, swupd requires that the base operating system is immutable, enabling software deployed on the system to rely on the base OS not changing unexpectedly.
Therefore files which are included in the os-core bundle must be the final version of a file which supports all bundles that may be installed on top of it. Adding a bundle to the OS must not modify a file provided by os-core. Most notably this means that the passwd and groups should contain all potential account information, regardless of whether or not these users and groups are utilised by the base OS.
Single, atomic, OS-wide version number
swupd based operating systems atomically upgrade the entire OS from one version to the next, rather than upgrading at a more granular, package-based, level. This means that the entire contents of the OS can be determined from a single version number.
Stateless OS
Clear Linux, the Linux-based OS which hosts swupd development, is a stateless OS — that is it comes with an empty /etc
by default and all default OS configuration is enabled until overridden by per-system configuration of files in /etc
.
Due to this swupd by default will not update files /etc
under the assumption that these are local configuration and the default configuration is provided by read-only files in other parts of the directory hierarchy in os-core.
The swupd way
The concepts described above lead to a swupd using OS being constructed a certain way which provides several advantages to the OS developer, not least of all the reduces test matrix through a much smaller set of combinations of software that might be installed on the OS.
References
For more details see the Clear Linux Project's documentation: * Bundles overview. * About software update * Stateless feature description
OpenEmbedded/Yocto Project considerations
Several of the concepts in the swupd way contradict the default configuration of operating systems built with the Yocto Project tools and OpenEmbedded.
When building a distribution with swupd the OS developer should be aware of the following requirements:
systemd
swupd assumes the use of systemd both by using systemd units to trigger update checks and by explicitly calling the systemd binary to restart the update-triggers.target
after an OS update has completed (see the swupd helper scripts section for more information about this target).
Therefore at this time one must use systemd as init manager in order to make use of swupd.
Version number
As the OS version number is atomic and identifies an entire OS build we generate a new set of swupd update artefacts only when the OS_VERSION
variable is changed (by default refusing to write over an existing set of generated artefacts with the same version number).
swupd-client checks the VERSION_ID
in the the os-release
file when checking for updates and we need to be certain that this is changed when building an OS update for deployment, to that end the swupd-image class automatically writes the OS_VERSION
to the VERSION_ID
field in the os-release
file on each image creation.
update-alternatives
As the os-core
bundle must provide an unchanging (at least within the same OS version) which other bundles can't modify we must be wary of recipes which use update-alternatives
.
Due to the way meta-swupd constructs the swupd-based OS and the resulting images and update stream artefacts we can find ourselves in a situation where the symlink for an alternative is installed pointing to a file which doesn't exist in the os-core (because it's provided by a bundle).
It is recommended that the OS developer carefully review all uses of update-alternatives
in their metadata and ensure that:
- the
ALTERNATIVE_PRIORITY
is set appropriately so that the desired provider of anALTERNATIVE_LINK_NAME
is correctly chosen and consistent across bundles. - the highest
ALTERNATIVE_PRIORITY
provider for anALTERNATIVE_LINK_NAME
is included in the os-core bundle.
To help detect the dangling symlink scenario the swupd-image
class includes a mechanism to check for such dangling symlinks in a constructed image, enable it by adding the swupd_check_dangling_symlinks
sanity check to SWUPD_IMAGE_SANITY_CHECKS
i.e.:
SWUPD_IMAGE_SANITY_CHECKS += " \ swupd_check_dangling_symlinks \ "
Users and groups
Another aspect of the immutable os-core bundle is that the account information in the passwd and group files installed in the os-core bundle must include user and group information required for all bundles provided by the OS.
We would also recommend the use of useradd-staticids
to ensure that the uid and gid are consistent across rebuilds so that os-core has better reproducibility.
See the Yocto Project Reference Manual section on useradd*.bbclass for more details.
NOTE: due to the stateless nature of swupd group and passwd will not be updated, for more information see the stateful section below.
Stateful
Linux-based OS built with OpenEmbedded and the Yocto Project Tools are stateful OS, that is the default configuration is provided by files in /etc
and without extra work on top of the default packages the OS won't boot without a populated /etc
. This contradicts the swupd way and means that currently any files a package installs into /etc
won't be updated between OS version.
swupd-client checks the VERSION_ID
in the the os-release
file when checking for updates. As this file is usually installed in /etc
we include a bbappend
which instead creates this file in /usr/lib/
(a location swupd-client prefers) and symlink it to /etc
.
In order to support deploying updates to files in /etc
via swupd we have developed patches for both swupd-client and swupd-server to include files in /etc
in updates. These changes are enabled in swupd-server (resulting in swupd manifests which list files in /etc
) but disabled by default in swupd-client.
This is because enabling updating of files in /etc
requires some forethought by the OS developer to ensure that files in /etc
which shouldn't be updated are not included in the bundle directories. For example if you would like users of your OS to be able to add users and groups without OS updates overwriting those users and groups you would need to ensusre that the passwd and group files are not included in the bundle directories during the do_swupd_update
task.
You might achieve this by having a prefunc
for the do_swupd_update
task which removes said files from the os-core bundle directory.
To enable stateful OS support in swupd-client disable the stateless
PACKAGE_CONFIG
option for the swupd-client recipe, i.e. with a swupd-client_%.bbappend
file:
PACKAGE_CONFIG_remove = "stateless"
swupd helper scripts
swupd-client tries to call out to certain helper scripts during an update: * before the update clr_pre_update.sh
is called * after the update systemdboot_updater.sh
and kernel_updater.sh
are called * the systemd update-triggers.target
is restarted once the update is complete
The oe-swupd-helpers
recipe provides skeletal implementations of the scripts (clr_pre_update.sh
, systemdboot_updater.sh
and kernel_updater.sh
) which simply print out that they have been called, these are clearly insufficient for a production environment and should be replaced with OS-specific implementations.
The systemd units in oe-swupd-helpers
are a little more useful, being based on the units used by the Clear Linux project, but likely need tweaking and adding to in order to better accommodate the OS being deployed.
Quick steps
- add the meta-swupd layer to bblayers.conf
- designate an image recipe as the base OS image (os-core, in swupd parlance) and
inherit swupd-image
in that recipe - ensure the
OS_VERSION
variable is assigned an integer value and that this number is increased before each build which should generate swupd update artefacts - Vet the os-core bundle to address the considerations documented above:
- review any included recipes that inherit update-alternatives and ensure that the desired provider of a binary is correctly chosen and consistent across bundles with the highest priority provider being included in the os-core bundle. (we recommend the use of the
swupd_check_dangling_symlinks
to help catch this issue at image construction time) - ensure all users and groups required by all bundles in the OS are defined in the os-core bundle (we recommend the use of
useradd-staticids
).
- Implement appropriate versions of the scripts and units in oe-swupd-helpers
(optional) Define additional bundles for any features you want to add assign a list of bundle names to
SWUPD_BUNDLES
and for each named bundle, assign a list of packages for which their content should be included in the bundle to a varflag ofBUNDLE_CONTENTS
which matches the bundle name i.e:SWUPD_BUNDLES = "feature_one feature_two" BUNDLE_CONTENTS[feature_one] = "package_one package_three package_six"
NOTE: beware of reserved bundle names, both full and mega have special meaning and cannot be used for bundle names.(optional) Define extra images, consisting of the os-core with any number of additional bundles installed, which can be built. Do this by setting the
SWUPD_IMAGES
variable to a list of additional image name suffixes and assigning the names of bundles to inclde to a varflag matching the defined name. For example:SWUPD_IMAGES = "product1" SWUPD_IMAGES[product1] = "product1"
defined in an image named myco-image-core would enable bitbake to be invoked with the target myco-image-core-product1. The myco-image-core-product1 image would consist of the base OS (os-core bundle) and the contents of the product1 bundle and might, for example, be used to provide an image which can be directly flashed to the product hardware.
Control variables
Several variables can be set to tune the way swupd-image works:
SWUPD_GENERATE
— if set to 0 i.e.SWUPD_GENERATE = "0"
swupd update artefact processing will be skipped but all tasks of theswupd-image
class will be executed. This is useful both for debugging theswupd-image
class and in a scenario where it might be desirable to generate the chroot-like bundle directories without performing an processing with swupd.SWUPD_DELTAPACKS
— if set to 0 i.e.SWUPD_DELTAPACKS="0"
swupd delta-packs will not be generated.SWUPD_N_DELTAPACKS
— the number of previous releases against which to generate delta-packs, defaults to 2.SWUPD_VERSION_STEP
— Amount the OS_VERSION should be increased by for each release. Used by the delta pack looping to generate delta packs going back up to SWUPD_N_DELTAPACK releases.SWUPD_IMAGES
— TODO
Using swupd client
The swupd-client package installs a binary named swupd which can be used to perform updates, etc on the target.
Take extra care with swupd-client versions prior to v3.4.0 as they hard-code the update URLs to clearlinux.org. When steps haven't been taken to change the default update URLs be sure to operate swupd-client with the -u switch and pass a URL to your update server.
Examples of swupd-client use follow:
swupd check-update -u example.com/updates swupd update -u example.com/updates swupd verify -u example.com/updates