Enable sstate cache - Revision history

PaulEggleton: Fix for changes in 1.3+

2013-11-08T11:23:26Z

Fix for changes in 1.3+

← Older revision		Revision as of 11:23, 8 November 2013
Line 1:		Line 1:
	== sstate background ==		== sstate background ==
	To speed up the build process, sstate provides a cache mechanism, where sstate files from server can be reused to avoid build from scratch if the producer and consumer of the sstate ~~has~~ same environment. With perfect case, we can achieve more than 80% time decreasing.		To speed up the build process, sstate provides a cache mechanism, where sstate files from server can be reused to avoid build from scratch if the producer and consumer of the sstate have the same environment. With perfect case, we can achieve more than 80% time decreasing.

	== use sstate cache server ==		== use sstate cache server ==
	Another build machine is needed to produce the sstate file periodically. The default sstate files directory is '''sstate-cache''' under build dir, which is needed to be exported via NFS, http or ftp server.		Another build machine is needed to produce the sstate file periodically. The default sstate files directory is '''sstate-cache''' under build dir, which is needed to be exported via NFS, http or ftp server.

	On your local machine, change the conf/local.conf as ~~following~~:		On your local machine, change the conf/local.conf as follows:

	SSTATE_MIRRORS ?= "\		<pre><nowiki>
	file://.* http://someserver.tld/share/sstate/"		SSTATE_MIRRORS ?= "\
			file://.* http://someserver.tld/share/sstate/PATH"
			</nowiki></pre>

	for NFS server		for an NFS server:
	~~SSTATE_MIRRORS ?= "\~~
	~~file~~:~~//.* file:///local/mounted/dir/sstate/"~~

			<pre><nowiki>
			SSTATE_MIRRORS ?= "\
			file://.* file:///local/mounted/dir/sstate/PATH"
			</nowiki></pre>

			The special "PATH" suffix is important as it is automatically expanded to the correct value for native packages (which are split by the host distribution being used).


Line 34:		Line 40:
	If there is no setscene tasks running, sstate cache can't be used because sstate function got broken or some environment value changes invalidate sstate cache. You can use following tools to check any changes between sstate cache and new produced one.		If there is no setscene tasks running, sstate cache can't be used because sstate function got broken or some environment value changes invalidate sstate cache. You can use following tools to check any changes between sstate cache and new produced one.

	#bitbake-diffsigs /~~mnt/sstate/sstate~~-~~packagename-checksum1-tgz.siginfo builddir~~/~~sstate~~-~~cache/sstate-packagename~~-~~checksum2~~-~~tgz.~~siginfo		If you wish to compare signatures to see any changes that caused a rebuild since the last built version, you can use <code>bitbake-diffsigs</code> with the <code>-t</code> option:

			$ bitbake-diffsigs -t quilt-native do_configure

			Alternatively you can compare siginfo files directly:

			$ bitbake-diffsigs /mnt/sstate/sstate-packagename-checksum1-tgz.siginfo builddir/sstate-cache/sstate-packagename-checksum2-tgz.siginfo

	== ~~know~~ issues ==		== Known issues ==
	* Various build distributions may lead to checksum mismatch		* New changeset may invalidate part or all of the sstate cache occasionally - this is by design
	* New changeset may invalidate ~~whole~~ sstate cache occasionally
	* Native package’s sstate cache can’t be reused among 32bit and 64bit host		* Native package’s sstate cache can’t be reused among 32bit and 64bit host
	* Missing sstate file on http/ftp server cause wget hang for a long time due to the retries and timeout		* Missing sstate file on http/ftp server cause wget hang for a long time due to the retries and timeout

Eflanagan: /* Use local sstate cache */ Removing warning about race condition as it is no longer accurate.

2012-05-14T18:07:44Z

Use local sstate cache: Removing warning about race condition as it is no longer accurate.

← Older revision		Revision as of 18:07, 14 May 2012
Line 23:		Line 23:

	So that each build would consume the sstate file if match, and produce new sstate file if not match.		So that each build would consume the sstate file if match, and produce new sstate file if not match.

	~~'''Pls. do not start multiple builds at one time, because of race condition issue'''~~

	== Verify the sstate is working ==		== Verify the sstate is working ==

Yzhao: /* Use local sstate cache */

2011-08-29T08:54:52Z

Use local sstate cache

← Older revision		Revision as of 08:54, 29 August 2011
Line 20:		Line 20:
	You can make all the builds pointing to the same SSTATE_DIR to share sstate files between them, like this:		You can make all the builds pointing to the same SSTATE_DIR to share sstate files between them, like this:

	SSTATE_DIR ?= "/share/sstate-cache		SSTATE_DIR ?= "/share/sstate-cache"

	So that each build would consume the sstate file if match, and produce new sstate file if not match.		So that each build would consume the sstate file if match, and produce new sstate file if not match.

Gzhai: /* know issues */

2011-02-17T14:53:53Z

know issues

← Older revision		Revision as of 14:53, 17 February 2011
Line 42:		Line 42:
	* Various build distributions may lead to checksum mismatch		* Various build distributions may lead to checksum mismatch
	* New changeset may invalidate whole sstate cache occasionally		* New changeset may invalidate whole sstate cache occasionally
	* ~~Bbclass~~		* Native package’s sstate cache can’t be reused among 32bit and 64bit host
	* ~~Base packages (libc, …)~~		* Missing sstate file on http/ftp server cause wget hang for a long time due to the retries and timeout

Gzhai: /* know issues */

2011-02-17T14:51:55Z

know issues

← Older revision		Revision as of 14:51, 17 February 2011
Line 42:		Line 42:
	* Various build distributions may lead to checksum mismatch		* Various build distributions may lead to checksum mismatch
	* New changeset may invalidate whole sstate cache occasionally		* New changeset may invalidate whole sstate cache occasionally
	* Bbclass		* Bbclass
	* Base packages (libc, …)		* Base packages (libc, …)

Gzhai: /* know issues */

2011-02-17T14:51:18Z

know issues

← Older revision		Revision as of 14:51, 17 February 2011
Line 42:		Line 42:
	* Various build distributions may lead to checksum mismatch		* Various build distributions may lead to checksum mismatch
	* New changeset may invalidate whole sstate cache occasionally		* New changeset may invalidate whole sstate cache occasionally
	Bbclass		* Bbclass
	Base packages (libc, …)		* Base packages (libc, …)

Gzhai: /* know issues */

2011-02-17T14:50:59Z

know issues

← Older revision		Revision as of 14:50, 17 February 2011
Line 40:		Line 40:

	== know issues ==		== know issues ==
			* Various build distributions may lead to checksum mismatch
			* New changeset may invalidate whole sstate cache occasionally
			Bbclass
			Base packages (libc, …)

Gzhai: /* Verify the sstate is working */

2011-02-17T14:48:47Z

Verify the sstate is working

← Older revision		Revision as of 14:48, 17 February 2011
Line 37:		Line 37:

	#bitbake-diffsigs /mnt/sstate/sstate-packagename-checksum1-tgz.siginfo builddir/sstate-cache/sstate-packagename-checksum2-tgz.siginfo		#bitbake-diffsigs /mnt/sstate/sstate-packagename-checksum1-tgz.siginfo builddir/sstate-cache/sstate-packagename-checksum2-tgz.siginfo


			== know issues ==

Gzhai: /* Verify the sstate is working */

2011-02-17T14:47:37Z

Verify the sstate is working

← Older revision		Revision as of 14:47, 17 February 2011
Line 34:		Line 34:
	NOTE: Running setscene task 119 of 155 (/home/lulianhao/poky-build/edwin/poky/meta/recipes-devtools/quilt/quilt-native_0.48.bb:do_populate_sysroot_setscene)		NOTE: Running setscene task 119 of 155 (/home/lulianhao/poky-build/edwin/poky/meta/recipes-devtools/quilt/quilt-native_0.48.bb:do_populate_sysroot_setscene)

	If there is no setscene tasks running, sstate cache can't be used because sstate function got broken or some environment value changes invalidate sstate cache. You can use following tools to check ~~what~~ changes ~~happened~~.		If there is no setscene tasks running, sstate cache can't be used because sstate function got broken or some environment value changes invalidate sstate cache. You can use following tools to check any changes between sstate cache and new produced one.

			#bitbake-diffsigs /mnt/sstate/sstate-packagename-checksum1-tgz.siginfo builddir/sstate-cache/sstate-packagename-checksum2-tgz.siginfo

Gzhai: /* Verify the sstate is working */

2011-02-17T14:37:32Z

Verify the sstate is working

← Older revision		Revision as of 14:37, 17 February 2011
Line 33:		Line 33:
	NOTE: Running setscene task 118 of 155 (virtual:native:/home/lulianhao/poky-build/edwin/poky/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot_setscene)		NOTE: Running setscene task 118 of 155 (virtual:native:/home/lulianhao/poky-build/edwin/poky/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot_setscene)
	NOTE: Running setscene task 119 of 155 (/home/lulianhao/poky-build/edwin/poky/meta/recipes-devtools/quilt/quilt-native_0.48.bb:do_populate_sysroot_setscene)		NOTE: Running setscene task 119 of 155 (/home/lulianhao/poky-build/edwin/poky/meta/recipes-devtools/quilt/quilt-native_0.48.bb:do_populate_sysroot_setscene)

			If there is no setscene tasks running, sstate cache can't be used because sstate function got broken or some environment value changes invalidate sstate cache. You can use following tools to check what changes happened.