TipsAndTricks/LockSharedState

Generating Hashes

From the bitbake help docs:

             -S SIGNATURE_HANDLER, --dump-signatures=SIGNATURE_HANDLER
                       Dump out the signature construction information, with
                       no task execution. The SIGNATURE_HANDLER parameter is
                       passed to the handler. Two common values are none and
                       printdiff but the handler may define more/less. none
                       means only dump the signature, printdiff means compare
                       the dumped signature with the cached one.

In simpler terms, this function can generate the list of signatures that bitbake uses to determine if a task needs to be rerun. If we set the "handler" to none, it will conveniently produce these hashes in a file called "locked-sigs.inc".

   $bitbake -S none core-image-minimal core-image-sato

In this example we generate the hashes for all tasks required to build core-image-minimal and core-image-sato, however it can be any recipe or image.

Copy Required SSTATE Artifacts (optional)

You may want to distribute the sstate, for example if you are generating a BSP and wish to supply pre-built artifacts. The gen-lockedsig-cache can be used to accomplish this:

 $ gen-lockedsig-cache
 syntax: gen-lockedsig-cache <locked-sigs.inc> <input-cachedir> <output-cachedir> <nativelsbstring> [filterfile]
 
 e.g.
 gen-lockedsig-cache ./locked-sigs.inc /current/sstate-cache/ ~/new/sstate-cache universal

Your nativelsbstring variable can be found by running bitbake -e:

 $ bitbake -e | grep NATIVELSBSTRING=
 NATIVELSBSTRING="universal"

Locking the Shared State Cache

As the locked-sigs.inc file implies, you will need to include this file in your configuration if you wish to lock the SSTATE. You can do so by using the include keyword in any recipe, or simply in the local.conf file (e.g. include locked-sigs.inc). Once you have done this, you will want to set the SIGGEN_LOCKEDSIGS_TASKSIG_CHECK variable to warn in order to ignore any errors. Settings include "error" and "warn", where error will stop the build and warn will not.

Rather than using the entire file, you can pick and choose which tasks you wish to lock. For example, you may only need to copy a few tasks into the local.conf file:

SIGGEN_LOCKEDSIGS = "\
gcc-cross:do_populate_sysroot:a8d91b35b98e1494957a2ddaf4598956 \
eglibc:do_populate_sysroot:13e8c68553dc61f9d67564f13b9b2d67 \
eglibc:do_packagedata:bfca0db1782c719d373f8636282596ee \
gcc-cross:do_packagedata:4b601ff4f67601395ee49c46701122f6 \
"

Now even if a task hash has changed for some reason, bitbake will use the shared state cache of the hash you have provided. This allows you to force tasks to be cached, regardless of what has changed in your configuration.

TODO: note why this is dangerous and update the manual with SIGGEN_LOCKEDSIGS_TASKSIG_CHECK definition.