Stateless: Difference between revisions

From Yocto Project
Jump to navigationJump to search
 
(11 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Definition ==
== Definition ==


A "stateless" distro is one where /etc is reserved entirely for local, machine specific customizations. Default OS configuration must be stored elsewhere or be compiled into the binaries. In such a distro, the OS must be able to boot with an entirely empty /etc.
A "stateless" distro is one where /etc and /var are entirely empty
when booting. According to this definition, populating /etc to factory
defaults during the boot process is allowed. However, also allowing
persistent modifications of the copied files then leads to problems
when updating the defaults. See
http://0pointer.net/blog/projects/stateless.html for more information.


See http://0pointer.net/blog/projects/stateless.html for more information.
So a stricter definition of "stateless" as used by the Clear Linux
project (called "fully stateless" in this page) is that the system must work also when /etc remains empty at
runtime, while (optionally) allowing an admin to configure the system
by creating files there. See https://clearlinux.org/features/stateless


== Purpose ==
== Purpose ==


In a stateless image, factory reset can be achieved by wiping out the content of /etc.
In a stateless image, factory reset can be achieved by wiping out the content of /etc, without exceptions.


A system update can be done without touching local configuration files.
A fully stateless image, updates can be done without touching local configuration files.


== Limitations ==
== Limitations ==


Configuration files in /etc may become incompatible with an updated OS. That is a problem that needs to be solved by the person or tool which created these incompatible local configuration files. It can be mitigated a bit by keeping the configuration API of OS components stable across updates.
Ideally, no configuration files are needed by the OS in /etc at all, while using them when they exist. Only the local admin or configuration tools create files there. In such a setup, configuration files in /etc may become incompatible with an updated OS. That is a problem that needs to be solved by the person or tool which created these incompatible local configuration files. It can be mitigated a bit by keeping the configuration API of OS components stable across updates.


 
In practice, a lot of files are still expected in /etc. They can be placed there from factory defaults via the systemd tmpfiles.d service. After a system update, either a factory reset must be done or these copied files must be overwritten with more recent ones from the updated factory defaults.
== Implementation ==
 
Ideally, an OS component itself knows how to update the default configuration with small, independent configuration fragments in /etc, i.e. the actual configuration is constructed dynamically at runtime. systemd works like that.
 
Less suitable, but still "stateless" is an OS component which reads a configuration file from /etc if it exists and falls back to some read-only default when it doesn't. This is less suitable when a real human creates the configuration in /etc because that person then has to ensure that updated system defaults get copied into the config file in /etc, but it may work when the file is created using some tool that can be re-run after an update.
 
Finally, an OS component can be configured to ignore /etc entirely and just read its configuration files from a read-only location. It can then still be configured when building the OS, but not locally. Depending on the use-cases, such local modifications might not be needed.
 
Some legacy components use /etc as location for files created at runtime. This is undesirable because it makes it less obvious for an admin which files in /etc may be modified, but if the files get recreated without data loss after a factory reset or system update, then using /etc is acceptable.


== Status and goals for "stateless" in Yocto ==
== Status and goals for "stateless" in Yocto ==
Line 30: Line 29:
Stateless support does not exit in current Yocto. Work is under way to improve that. Related Bugzilla entries are https://bugzilla.yoctoproject.org/show_bug.cgi?id=9527, https://bugzilla.yoctoproject.org/show_bug.cgi?id=1593
Stateless support does not exit in current Yocto. Work is under way to improve that. Related Bugzilla entries are https://bugzilla.yoctoproject.org/show_bug.cgi?id=9527, https://bugzilla.yoctoproject.org/show_bug.cgi?id=1593


It is not realistic to make all components in Yocto stateless. This will only work for some selected components and thus only those images using those components. The initial goal is to make the "refkit-image-common" from the IoT OS Reference Kit stateless. In that image, at least the following local configuration changes are meant to work:
It is not realistic to make all components in Yocto stateless. This will only work for some selected components and thus only those images using those components. The initial goal is to make the "refkit-image-common" from the IoT OS Reference Kit stateless, with user handling fully stateless. In that image, at least the following local configuration changes are meant to work in combination with system update:
* adding users and groups with user/groupadd
* adding users and groups with user/groupadd
* replacing the default motd
* replacing the default motd


The current prototype is hosted in https://github.com/pohly/intel-iot-refkit/tree/stateless
The current prototype is hosted in https://github.com/pohly/intel-iot-refkit/tree/stateless-pr. It does not yet achieve all of the goals, but the general approach is ready for review in https://github.com/intel/intel-iot-refkit/pull/233. Previous work in Ostro that led to a considerably reduced /etc is in https://github.com/pohly/ostro-os/tree/stateless-rebased.
 
The generic mechanism is https://github.com/pohly/intel-iot-refkit/blob/stateless/meta-refkit/classes/stateless.bbclass. It supports STATELESS_RELOCATE_pn-<recipe> = "True" for compiling a recipe such that it uses a read-only sysconfig directory. STATELESS_MV is for cases where a recipe puts files into /etc in do_install that can also be elsewhere. STATELESS_RM is for items that can be removed. The same exists as STATELESS_RM/MV_ROOTFS for items created as part of package installation or rootfs functions.
 
The actual configuration used for Refkit is https://github.com/pohly/intel-iot-refkit/blob/stateless/meta-refkit/conf/distro/include/stateless.inc. It does not yet enforce an empty /etc. Instead one has to run "bitbake refkit-image-common:do_rootfs" and check the content of /etc manually.
 
Some BKMs for gradually moving content out of /etc:
 
* use STATELESS_MV_pn-<recipe> to avoid repackaging other recipes
* precede each entry with a detailed technical explanation of why the change is working
* sort entries in the file as follows:
** header
** STATELESS_ETC_WHITELIST
** STATELESS_RM/MV sorted by pn-<reciped> (alphabetically)
** STATELESS_RM_ROOTFS (no particular order, add at the bottom)
** STATELESS_MV_ROOTFS (no particular order, add at the bottom)
** special python functions


refkit-image-common is stateless once /etc is empty after do_rootfs and the resulting image works without problems. Right now, it still has the following content:
The generic mechanism is https://github.com/pohly/intel-iot-refkit/blob/stateless-pr/meta-refkit-core/classes/stateless.bbclass. It operates almost entirely during rootfs construction, with some patches integrated into various recipes as an interim solution until it gets decided whether those patches can be upstreamed and/or included in OE-core.


$ tree tmp-glibc/work/intel_corei7_64-refkit-linux/refkit-image-common/1.0-r0/rootfs/etc/
The actual configuration is in various stateless*.inc files in https://github.com/pohly/intel-iot-refkit/blob/stateless-pr/meta-refkit-core/conf/distro/include.
tmp-glibc/work/intel_corei7_64-refkit-linux/refkit-image-common/1.0-r0/rootfs/etc/
├── asound.conf
├── audisp
│   ├── audispd.conf
│   └── plugins.d
│      ├── af_unix.conf
│      └── syslog.conf
├── audit
│   ├── auditd.conf
│   ├── audit.rules
│   └── rules.d
│      └── audit.rules
├── bluetooth
│   ├── input.conf
│   └── network.conf
├── build
├── busybox.links.nosuid
├── busybox.links.suid
├── ca-certificates
│   └── update.d
├── ca-certificates.conf
├── dbus-1
│   ├── session.conf
│   ├── system.conf
│   └── system.d
│      ├── bluetooth.conf
│      ├── connman.conf
│      ├── dbus-wpa_supplicant.conf
│      ├── org.freedesktop.hostname1.conf
│      ├── org.freedesktop.locale1.conf
│      ├── org.freedesktop.login1.conf
│      ├── org.freedesktop.machine1.conf
│      ├── org.freedesktop.systemd1.conf
│      └── org.freedesktop.timedate1.conf
├── default
│   ├── auditd
│   ├── mountall
│   ├── usbd
│   ├── useradd
│   └── volatiles
│      ├── 99_dbus
│      ├── 99_pam
│      ├── 99_sshd
│      └── 99_wpa_supplicant
├── depmod.d
├── environment
├── filesystems
├── fstab
├── group
├── gshadow
├── host.conf
├── hostname
├── hosts
├── init.d
│   └── smack
├── inputrc
├── issue
├── issue.net
├── ld.so.conf
├── libaudit.conf
├── libnl
│   ├── classid
│   └── pktloc
├── login.defs
├── machine-id
├── mke2fs.conf
├── modprobe.d
├── modules-load.d
│   ├── iwlwifi.conf
│   └── uio.conf
├── motd
├── mtab -> /proc/mounts
├── network
│   ├── if-down.d
│   ├── if-post-down.d
│   │   └── wpa-supplicant -> ../if-pre-up.d/wpa-supplicant
│   └── if-pre-up.d
│      └── wpa-supplicant
├── nsswitch.conf
├── os-release
├── pam.d
│   ├── chage
│   ├── chfn
│   ├── chgpasswd
│   ├── chpasswd
│   ├── chsh
│   ├── common-account
│   ├── common-auth
│   ├── common-password
│   ├── common-session
│   ├── common-session-noninteractive
│   ├── groupadd
│   ├── groupdel
│   ├── groupmems
│   ├── groupmod
│   ├── login
│   ├── newusers
│   ├── other
│   ├── passwd
│   ├── runuser
│   ├── runuser-l
│   ├── sshd
│   ├── su
│   ├── systemd-user
│   ├── useradd
│   ├── userdel
│   └── usermod
├── passwd
├── profile
├── protocols
├── request-key.conf
├── request-key.d
├── rpc
├── securetty
├── security
│   ├── access.conf
│   ├── group.conf
│   ├── limits.conf
│   ├── limits.d
│   ├── namespace.conf
│   ├── namespace.d
│   ├── namespace.init
│   ├── pam_env.conf
│   └── time.conf
├── services
├── shadow
├── shells
├── skel
├── smack
│   ├── accesses.d
│   └── cipso.d
├── ssh
│   ├── moduli
│   ├── ssh_config
│   ├── sshd_config
│   └── sshd_config_readonly
├── ssl
│   ├── certs
│   │   ├── 02265526.0 -> /usr/share/ca-certificates/mozilla/Entrust_Root_Certification_Authority_-_G2.crt
│   │   ├── 024dc131.0 -> /usr/share/ca-certificates/mozilla/Microsec_e-Szigno_Root_CA.crt
│   │   ├── 03179a64.0 -> /usr/share/ca-certificates/mozilla/Staat_der_Nederlanden_EV_Root_CA.crt
...
│   │   └── XRamp_Global_CA_Root.pem -> /usr/share/ca-certificates/mozilla/XRamp_Global_CA_Root.crt
│   ├── openssl.cnf
│   └── private
├── sysctl.d
├── systemd
│   ├── journald.conf
│   ├── logind.conf
│   ├── network
│   ├── system
│   │   ├── bluetooth.target.wants
│   │   │   └── bluetooth.service -> /lib/systemd/system/bluetooth.service
│   │   ├── ctrl-alt-del.target -> ../../../lib/systemd/system/reboot.target
│   │   ├── dbus-org.bluez.service -> /lib/systemd/system/bluetooth.service
│   │   ├── default.target -> /lib/systemd/system/multi-user.target
│   │   ├── getty.target.wants
│   │   │   └── getty@tty1.service -> ../../../../lib/systemd/system/getty@.service
│   │   ├── local-fs.target.wants
│   │   │   └── var-volatile-lib.service -> /lib/systemd/system/var-volatile-lib.service
│   │   ├── multi-user.target.wants
│   │   │   ├── auditd.service -> /lib/systemd/system/auditd.service
│   │   │   ├── connman.service -> /lib/systemd/system/connman.service
│   │   │   ├── machines.target -> ../../../../lib/systemd/system/machines.target
│   │   │   └── remote-fs.target -> ../../../../lib/systemd/system/remote-fs.target
│   │   ├── network.target.wants
│   │   │   ├── ip6tables.service -> /lib/systemd/system/ip6tables.service
│   │   │   └── iptables.service -> /lib/systemd/system/iptables.service
│   │   ├── sockets.target.wants
│   │   │   └── sshd.socket -> /lib/systemd/system/sshd.socket
│   │   ├── sysinit.target.wants
│   │   │   ├── run-postinsts.service -> /lib/systemd/system/run-postinsts.service
│   │   │   └── systemd-timesyncd.service -> ../../../../lib/systemd/system/systemd-timesyncd.service
│   │   └── systemd-random-seed.service.wants
│   │      └── var-volatile-lib.service -> /lib/systemd/system/var-volatile-lib.service
│   ├── system.conf
│   ├── timesyncd.conf
│   ├── user
│   └── user.conf
├── terminfo
│   ├── a
│   │   └── ansi
│   ├── d
│   │   └── dumb
│   ├── l
│   │   └── linux
│   ├── r
│   │   └── rxvt
│   ├── s
│   │   ├── screen
│   │   ├── screen-256color
│   │   └── sun
│   ├── v
│   │   ├── vt100
│   │   ├── vt102
│   │   ├── vt200
│   │   ├── vt220
│   │   └── vt52
│   └── x
│      ├── xterm -> xterm-color
│      ├── xterm-256color
│      ├── xterm-color
│      └── xterm-xfree86
├── timestamp
├── tmpfiles.d
│   ├── 00-create-volatile.conf
│   ├── audit-volatile.conf
│   └── connman_resolvconf.conf
├── udev
│   ├── hwdb.d
│   ├── rules.d
│   │   ├── touchscreen.rules
│   │   └── udev-smack-default.rules
│   └── udev.conf
├── udhcpc.d
│   └── 50default
├── version
├── wpa_supplicant.conf
└── xdg
    └── systemd
        └── user -> ../../systemd/user
62 directories, 500 files


== Alternatives ==
== Alternatives ==

Latest revision as of 13:31, 7 July 2017

Definition

A "stateless" distro is one where /etc and /var are entirely empty when booting. According to this definition, populating /etc to factory defaults during the boot process is allowed. However, also allowing persistent modifications of the copied files then leads to problems when updating the defaults. See http://0pointer.net/blog/projects/stateless.html for more information.

So a stricter definition of "stateless" as used by the Clear Linux project (called "fully stateless" in this page) is that the system must work also when /etc remains empty at runtime, while (optionally) allowing an admin to configure the system by creating files there. See https://clearlinux.org/features/stateless

Purpose

In a stateless image, factory reset can be achieved by wiping out the content of /etc, without exceptions.

A fully stateless image, updates can be done without touching local configuration files.

Limitations

Ideally, no configuration files are needed by the OS in /etc at all, while using them when they exist. Only the local admin or configuration tools create files there. In such a setup, configuration files in /etc may become incompatible with an updated OS. That is a problem that needs to be solved by the person or tool which created these incompatible local configuration files. It can be mitigated a bit by keeping the configuration API of OS components stable across updates.

In practice, a lot of files are still expected in /etc. They can be placed there from factory defaults via the systemd tmpfiles.d service. After a system update, either a factory reset must be done or these copied files must be overwritten with more recent ones from the updated factory defaults.

Status and goals for "stateless" in Yocto

Stateless support does not exit in current Yocto. Work is under way to improve that. Related Bugzilla entries are https://bugzilla.yoctoproject.org/show_bug.cgi?id=9527, https://bugzilla.yoctoproject.org/show_bug.cgi?id=1593

It is not realistic to make all components in Yocto stateless. This will only work for some selected components and thus only those images using those components. The initial goal is to make the "refkit-image-common" from the IoT OS Reference Kit stateless, with user handling fully stateless. In that image, at least the following local configuration changes are meant to work in combination with system update:

  • adding users and groups with user/groupadd
  • replacing the default motd

The current prototype is hosted in https://github.com/pohly/intel-iot-refkit/tree/stateless-pr. It does not yet achieve all of the goals, but the general approach is ready for review in https://github.com/intel/intel-iot-refkit/pull/233. Previous work in Ostro that led to a considerably reduced /etc is in https://github.com/pohly/ostro-os/tree/stateless-rebased.

The generic mechanism is https://github.com/pohly/intel-iot-refkit/blob/stateless-pr/meta-refkit-core/classes/stateless.bbclass. It operates almost entirely during rootfs construction, with some patches integrated into various recipes as an interim solution until it gets decided whether those patches can be upstreamed and/or included in OE-core.

The actual configuration is in various stateless*.inc files in https://github.com/pohly/intel-iot-refkit/blob/stateless-pr/meta-refkit-core/conf/distro/include.

Alternatives

There are other solutions with similar goals. However, stateless has some properties that, when it is applicable, make stateless a better solution.

overlayfs

The OS configuration files are in /etc. At runtime, an overlay is activated. Editing items in /etc then stores a new copy of the modified files in the overlay. A system update is done when the overlay is not active.

Drawback: files in the overlay continue to shadow an updated system configuration, i.e. new settings from the updated OS configuration files are ignored. Addressing this would require merging potentially complex files during an update.

config generator

Instead of allowing local modifications in the normal /etc configuration files, one or more additional config files are used. A tool takes those files and the default system configuration to produce the actual content of /etc, either by writing to it directly or into an overlay.

Drawback: custom solution, the normal tools like "useradd" do not work; no such generator exists (?) and thus would have to be written from scratch Advantage: full control over the format of allowed configuration changes; the generator can be updated together with the OS so that the output always matches what the OS components expect

Such a config generator can be combined with a stateless distro.