Stateless - Revision history

Patrick Ohly: /* Status and goals for "stateless" in Yocto */

2017-07-07T13:31:28Z

Status and goals for "stateless" in Yocto

← Older revision		Revision as of 13:31, 7 July 2017
Line 35:		Line 35:
	The current prototype is hosted in https://github.com/pohly/intel-iot-refkit/tree/stateless-pr. It does not yet achieve all of the goals, but the general approach is ready for review in https://github.com/intel/intel-iot-refkit/pull/233. Previous work in Ostro that led to a considerably reduced /etc is in https://github.com/pohly/ostro-os/tree/stateless-rebased.		The current prototype is hosted in https://github.com/pohly/intel-iot-refkit/tree/stateless-pr. It does not yet achieve all of the goals, but the general approach is ready for review in https://github.com/intel/intel-iot-refkit/pull/233. Previous work in Ostro that led to a considerably reduced /etc is in https://github.com/pohly/ostro-os/tree/stateless-rebased.

	The generic mechanism is https://github.com/pohly/intel-iot-refkit/blob/stateless-pr/meta-refkit/classes/stateless.bbclass. It operates almost entirely during rootfs construction, with some patches integrated into various recipes as an interim solution until it gets decided whether those patches can be upstreamed and/or included in OE-core.		The generic mechanism is https://github.com/pohly/intel-iot-refkit/blob/stateless-pr/meta-refkit-core/classes/stateless.bbclass. It operates almost entirely during rootfs construction, with some patches integrated into various recipes as an interim solution until it gets decided whether those patches can be upstreamed and/or included in OE-core.

	The actual configuration is in various stateless*.inc files in https://github.com/pohly/intel-iot-refkit/blob/stateless-pr/meta-refkit/conf/distro/include.		The actual configuration is in various stateless*.inc files in https://github.com/pohly/intel-iot-refkit/blob/stateless-pr/meta-refkit-core/conf/distro/include.

	== Alternatives ==		== Alternatives ==

Patrick Ohly: /* Status and goals for "stateless" in Yocto */

2017-07-07T13:30:49Z

Status and goals for "stateless" in Yocto

← Older revision		Revision as of 13:30, 7 July 2017
Line 29:		Line 29:
	Stateless support does not exit in current Yocto. Work is under way to improve that. Related Bugzilla entries are https://bugzilla.yoctoproject.org/show_bug.cgi?id=9527, https://bugzilla.yoctoproject.org/show_bug.cgi?id=1593		Stateless support does not exit in current Yocto. Work is under way to improve that. Related Bugzilla entries are https://bugzilla.yoctoproject.org/show_bug.cgi?id=9527, https://bugzilla.yoctoproject.org/show_bug.cgi?id=1593

	It is not realistic to make all components in Yocto stateless. This will only work for some selected components and thus only those images using those components. The initial goal is to make the "refkit-image-common" from the IoT OS Reference Kit stateless. In that image, at least the following local configuration changes are meant to work:		It is not realistic to make all components in Yocto stateless. This will only work for some selected components and thus only those images using those components. The initial goal is to make the "refkit-image-common" from the IoT OS Reference Kit stateless, with user handling fully stateless. In that image, at least the following local configuration changes are meant to work in combination with system update:
	* adding users and groups with user/groupadd		* adding users and groups with user/groupadd
	* replacing the default motd		* replacing the default motd

	The current prototype is hosted in https://github.com/pohly/intel-iot-refkit/tree/stateless. Previous work in Ostro that led to a considerably reduced /etc is in https://github.com/pohly/ostro-os/tree/stateless-rebased.		The current prototype is hosted in https://github.com/pohly/intel-iot-refkit/tree/stateless-pr. It does not yet achieve all of the goals, but the general approach is ready for review in https://github.com/intel/intel-iot-refkit/pull/233. Previous work in Ostro that led to a considerably reduced /etc is in https://github.com/pohly/ostro-os/tree/stateless-rebased.

	The generic mechanism is https://github.com/pohly/intel-iot-refkit/blob/stateless/meta-refkit/classes/stateless.bbclass. It operates entirely during rootfs construction~~. STATELESS_MV_ROOTFS is for cases where configuration files were put~~ into /~~etc~~ in ~~do_install that can also be elsewhere. It also supports moving files into the factory defaults. STATELESS_RM_ROOTFS is for items that can be removed~~.		The generic mechanism is https://github.com/pohly/intel-iot-refkit/blob/stateless-pr/meta-refkit/classes/stateless.bbclass. It operates almost entirely during rootfs construction, with some patches integrated into various recipes as an interim solution until it gets decided whether those patches can be upstreamed and/or included in OE-core.

	The actual configuration ~~used for Refkit~~ is https://github.com/pohly/intel-iot-refkit/blob/stateless/meta-refkit/conf/distro/include~~/stateless.inc~~. ~~refkit-image-common is built with an empty /etc, and accidentally putting something into /etc triggers a build error.~~		The actual configuration is in various stateless*.inc files in https://github.com/pohly/intel-iot-refkit/blob/stateless-pr/meta-refkit/conf/distro/include.

	~~Some BKMs for gradually moving content out of /etc:~~

	* precede each entry with a detailed technical explanation of why the change is working
	* sort entries in the file as follows:
	** header
	** STATELESS_RM_ROOTFS (no particular order, add at the bottom)
	** STATELESS_MV_ROOTFS (no particular order, add at the bottom)

	== Alternatives ==		== Alternatives ==

Patrick Ohly: /* Purpose */

2017-07-07T13:24:02Z

Purpose

← Older revision		Revision as of 13:24, 7 July 2017
Line 17:		Line 17:
	In a stateless image, factory reset can be achieved by wiping out the content of /etc, without exceptions.		In a stateless image, factory reset can be achieved by wiping out the content of /etc, without exceptions.

	A ~~system update~~ can be done without touching local configuration files.		A fully stateless image, updates can be done without touching local configuration files.

	== Limitations ==		== Limitations ==

Patrick Ohly: /* Definition */

2017-07-07T13:23:32Z

Definition

← Older revision		Revision as of 13:23, 7 July 2017
Line 9:		Line 9:

	So a stricter definition of "stateless" as used by the Clear Linux		So a stricter definition of "stateless" as used by the Clear Linux
	project is that the system must work also when /etc remains empty at		project (called "fully stateless" in this page) is that the system must work also when /etc remains empty at
	runtime, while (optionally) allowing an admin to configure the system		runtime, while (optionally) allowing an admin to configure the system
	by creating files there. See https://clearlinux.org/features/stateless		by creating files there. See https://clearlinux.org/features/stateless

Patrick Ohly: /* Definition */

2017-07-07T13:22:37Z

Definition

← Older revision		Revision as of 13:22, 7 July 2017
Line 1:		Line 1:
	== Definition ==		== Definition ==

	A "stateless" distro is one where /etc and /var are entirely empty when booting. ~~No state persists across reboots~~, ~~hence~~ the ~~name~~. ~~A more relaxed version~~ of ~~this preserves config~~ files in /~~etc, but still allows them to be removed at any time to do a factory reset~~.		A "stateless" distro is one where /etc and /var are entirely empty
			when booting. According to this definition, populating /etc to factory
			defaults during the boot process is allowed. However, also allowing
			persistent modifications of the copied files then leads to problems
			when updating the defaults. See
			http://0pointer.net/blog/projects/stateless.html for more information.

	See ~~http~~://~~0pointer~~.~~net/blog~~/~~projects~~/stateless~~.html for more information.~~		So a stricter definition of "stateless" as used by the Clear Linux
			project is that the system must work also when /etc remains empty at
			runtime, while (optionally) allowing an admin to configure the system
			by creating files there. See https://clearlinux.org/features/stateless

	== Purpose ==		== Purpose ==

Patrick Ohly: /* Status and goals for "stateless" in Yocto */

2017-03-10T12:32:31Z

Status and goals for "stateless" in Yocto

Show changes

Patrick Ohly: /* Implementation */

2017-03-10T12:28:09Z

Implementation

← Older revision		Revision as of 12:28, 10 March 2017
Line 16:		Line 16:

	In practice, a lot of files are still expected in /etc. They can be placed there from factory defaults via the systemd tmpfiles.d service. After a system update, either a factory reset must be done or these copied files must be overwritten with more recent ones from the updated factory defaults.		In practice, a lot of files are still expected in /etc. They can be placed there from factory defaults via the systemd tmpfiles.d service. After a system update, either a factory reset must be done or these copied files must be overwritten with more recent ones from the updated factory defaults.

	~~== Implementation ==~~

	Ideally, an OS component itself knows how to update the default configuration with small, independent configuration fragments in /etc, i.e. the actual configuration is constructed dynamically at runtime. systemd works like that.

	Less suitable, but still "stateless" is an OS component which reads a configuration file from /etc if it exists and falls back to some read-only default when it doesn't. This is less suitable when a real human creates the configuration in /etc because that person then has to ensure that updated system defaults get copied into the config file in /etc, but it may work when the file is created using some tool that can be re-run after an update.

	Finally, an OS component can be configured to ignore /etc entirely and just read its configuration files from a read-only location. It can then still be configured when building the OS, but not locally. Depending on the use-cases, such local modifications might not be needed.

	Some legacy components use /etc as location for files created at runtime. This is undesirable because it makes it less obvious for an admin which files in /etc may be modified, but if the files get recreated without data loss after a factory reset or system update, then using /etc is acceptable.

	== Status and goals for "stateless" in Yocto ==		== Status and goals for "stateless" in Yocto ==

Patrick Ohly: /* Limitations */

2017-03-10T12:27:57Z

Limitations

← Older revision		Revision as of 12:27, 10 March 2017
Line 13:		Line 13:
	== Limitations ==		== Limitations ==

	Ideally, no configuration files are needed by the OS in /etc at all, while using them when they exist. Only the local admin or configuration tools create files there. In such a setup, configuration files in /etc may become incompatible with an updated OS. That is a problem that needs to be solved by the person or tool which created these incompatible local configuration files. It can be mitigated a bit by keeping the configuration API of OS components stable across updates.		Ideally, no configuration files are needed by the OS in /etc at all, while using them when they exist. Only the local admin or configuration tools create files there. In such a setup, configuration files in /etc may become incompatible with an updated OS. That is a problem that needs to be solved by the person or tool which created these incompatible local configuration files. It can be mitigated a bit by keeping the configuration API of OS components stable across updates.

	In practice, a lot of files are still expected in /etc. They can be placed there from factory defaults via the systemd tmpfiles.d service. After a system update, either a factory reset must be done or these copied files must be overwritten with more recent ones from the updated factory defaults.		In practice, a lot of files are still expected in /etc. They can be placed there from factory defaults via the systemd tmpfiles.d service. After a system update, either a factory reset must be done or these copied files must be overwritten with more recent ones from the updated factory defaults.

Patrick Ohly: /* Limitations */

2017-03-10T11:27:18Z

Limitations

← Older revision		Revision as of 11:27, 10 March 2017
Line 13:		Line 13:
	== Limitations ==		== Limitations ==

	~~Configuration~~ files in /etc may become incompatible with an updated OS. That is a problem that needs to be solved by the person or tool which created these incompatible local configuration files. It can be mitigated a bit by keeping the configuration API of OS components stable across updates.		Ideally, no configuration files are needed by the OS in /etc at all, while using them when they exist. Only the local admin or configuration tools create files there. In such a setup, configuration files in /etc may become incompatible with an updated OS. That is a problem that needs to be solved by the person or tool which created these incompatible local configuration files. It can be mitigated a bit by keeping the configuration API of OS components stable across updates.

			In practice, a lot of files are still expected in /etc. They can be placed there from factory defaults via the systemd tmpfiles.d service. After a system update, either a factory reset must be done or these copied files must be overwritten with more recent ones from the updated factory defaults.

	== Implementation ==		== Implementation ==

Patrick Ohly: /* Purpose */

2017-03-10T11:21:01Z

Purpose

← Older revision		Revision as of 11:21, 10 March 2017
Line 7:		Line 7:
	== Purpose ==		== Purpose ==

	In a stateless image, factory reset can be achieved by wiping out the content of /etc.		In a stateless image, factory reset can be achieved by wiping out the content of /etc, without exceptions.

	A system update can be done without touching local configuration files.		A system update can be done without touching local configuration files.