Stateless

From Yocto Project
Jump to navigationJump to search

Definition

A "stateless" distro is one where /etc and /var are entirely empty when booting. According to this definition, populating /etc to factory defaults during the boot process is allowed. However, also allowing persistent modifications of the copied files then leads to problems when updating the defaults. See http://0pointer.net/blog/projects/stateless.html for more information.

So a stricter definition of "stateless" as used by the Clear Linux project (called "fully stateless" in this page) is that the system must work also when /etc remains empty at runtime, while (optionally) allowing an admin to configure the system by creating files there. See https://clearlinux.org/features/stateless

Purpose

In a stateless image, factory reset can be achieved by wiping out the content of /etc, without exceptions.

A system update can be done without touching local configuration files.

Limitations

Ideally, no configuration files are needed by the OS in /etc at all, while using them when they exist. Only the local admin or configuration tools create files there. In such a setup, configuration files in /etc may become incompatible with an updated OS. That is a problem that needs to be solved by the person or tool which created these incompatible local configuration files. It can be mitigated a bit by keeping the configuration API of OS components stable across updates.

In practice, a lot of files are still expected in /etc. They can be placed there from factory defaults via the systemd tmpfiles.d service. After a system update, either a factory reset must be done or these copied files must be overwritten with more recent ones from the updated factory defaults.

Status and goals for "stateless" in Yocto

Stateless support does not exit in current Yocto. Work is under way to improve that. Related Bugzilla entries are https://bugzilla.yoctoproject.org/show_bug.cgi?id=9527, https://bugzilla.yoctoproject.org/show_bug.cgi?id=1593

It is not realistic to make all components in Yocto stateless. This will only work for some selected components and thus only those images using those components. The initial goal is to make the "refkit-image-common" from the IoT OS Reference Kit stateless. In that image, at least the following local configuration changes are meant to work:

  • adding users and groups with user/groupadd
  • replacing the default motd

The current prototype is hosted in https://github.com/pohly/intel-iot-refkit/tree/stateless. Previous work in Ostro that led to a considerably reduced /etc is in https://github.com/pohly/ostro-os/tree/stateless-rebased.

The generic mechanism is https://github.com/pohly/intel-iot-refkit/blob/stateless/meta-refkit/classes/stateless.bbclass. It operates entirely during rootfs construction. STATELESS_MV_ROOTFS is for cases where configuration files were put into /etc in do_install that can also be elsewhere. It also supports moving files into the factory defaults. STATELESS_RM_ROOTFS is for items that can be removed.

The actual configuration used for Refkit is https://github.com/pohly/intel-iot-refkit/blob/stateless/meta-refkit/conf/distro/include/stateless.inc. refkit-image-common is built with an empty /etc, and accidentally putting something into /etc triggers a build error.

Some BKMs for gradually moving content out of /etc:

  • precede each entry with a detailed technical explanation of why the change is working
  • sort entries in the file as follows:
    • header
    • STATELESS_RM_ROOTFS (no particular order, add at the bottom)
    • STATELESS_MV_ROOTFS (no particular order, add at the bottom)

Alternatives

There are other solutions with similar goals. However, stateless has some properties that, when it is applicable, make stateless a better solution.

overlayfs

The OS configuration files are in /etc. At runtime, an overlay is activated. Editing items in /etc then stores a new copy of the modified files in the overlay. A system update is done when the overlay is not active.

Drawback: files in the overlay continue to shadow an updated system configuration, i.e. new settings from the updated OS configuration files are ignored. Addressing this would require merging potentially complex files during an update.

config generator

Instead of allowing local modifications in the normal /etc configuration files, one or more additional config files are used. A tool takes those files and the default system configuration to produce the actual content of /etc, either by writing to it directly or into an overlay.

Drawback: custom solution, the normal tools like "useradd" do not work; no such generator exists (?) and thus would have to be written from scratch Advantage: full control over the format of allowed configuration changes; the generator can be updated together with the OS so that the output always matches what the OS components expect

Such a config generator can be combined with a stateless distro.