Stateless

Definition

A "stateless" distro is one where /etc and /var are entirely empty when booting. According to this definition, populating /etc to factory defaults during the boot process is allowed. However, also allowing persistent modifications of the copied files then leads to problems when updating the defaults. See http://0pointer.net/blog/projects/stateless.html for more information.

So a stricter definition of "stateless" as used by the Clear Linux project (called "fully stateless" in this page) is that the system must work also when /etc remains empty at runtime, while (optionally) allowing an admin to configure the system by creating files there. See https://clearlinux.org/features/stateless

Purpose

In a stateless image, factory reset can be achieved by wiping out the content of /etc, without exceptions.

A fully stateless image, updates can be done without touching local configuration files.

Limitations

Ideally, no configuration files are needed by the OS in /etc at all, while using them when they exist. Only the local admin or configuration tools create files there. In such a setup, configuration files in /etc may become incompatible with an updated OS. That is a problem that needs to be solved by the person or tool which created these incompatible local configuration files. It can be mitigated a bit by keeping the configuration API of OS components stable across updates.

In practice, a lot of files are still expected in /etc. They can be placed there from factory defaults via the systemd tmpfiles.d service. After a system update, either a factory reset must be done or these copied files must be overwritten with more recent ones from the updated factory defaults.

Status and goals for "stateless" in Yocto

Stateless support does not exit in current Yocto. Work is under way to improve that. Related Bugzilla entries are https://bugzilla.yoctoproject.org/show_bug.cgi?id=9527, https://bugzilla.yoctoproject.org/show_bug.cgi?id=1593

It is not realistic to make all components in Yocto stateless. This will only work for some selected components and thus only those images using those components. The initial goal is to make the "refkit-image-common" from the IoT OS Reference Kit stateless, with user handling fully stateless. In that image, at least the following local configuration changes are meant to work in combination with system update:

• adding users and groups with user/groupadd
• replacing the default motd

The current prototype is hosted in https://github.com/pohly/intel-iot-refkit/tree/stateless-pr. It does not yet achieve all of the goals, but the general approach is ready for review in https://github.com/intel/intel-iot-refkit/pull/233. Previous work in Ostro that led to a considerably reduced /etc is in https://github.com/pohly/ostro-os/tree/stateless-rebased.

The generic mechanism is https://github.com/pohly/intel-iot-refkit/blob/stateless-pr/meta-refkit-core/classes/stateless.bbclass. It operates almost entirely during rootfs construction, with some patches integrated into various recipes as an interim solution until it gets decided whether those patches can be upstreamed and/or included in OE-core.

The actual configuration is in various stateless*.inc files in https://github.com/pohly/intel-iot-refkit/blob/stateless-pr/meta-refkit-core/conf/distro/include.

Alternatives

There are other solutions with similar goals. However, stateless has some properties that, when it is applicable, make stateless a better solution.

overlayfs

The OS configuration files are in /etc. At runtime, an overlay is activated. Editing items in /etc then stores a new copy of the modified files in the overlay. A system update is done when the overlay is not active.

Drawback: files in the overlay continue to shadow an updated system configuration, i.e. new settings from the updated OS configuration files are ignored. Addressing this would require merging potentially complex files during an update.

config generator

Instead of allowing local modifications in the normal /etc configuration files, one or more additional config files are used. A tool takes those files and the default system configuration to produce the actual content of /etc, either by writing to it directly or into an overlay.

Drawback: custom solution, the normal tools like "useradd" do not work; no such generator exists (?) and thus would have to be written from scratch Advantage: full control over the format of allowed configuration changes; the generator can be updated together with the OS so that the output always matches what the OS components expect

Such a config generator can be combined with a stateless distro.