TipsAndTricks/PrelinkSomePointersAndWorkarounds - Revision history

Bavery: /* The Ugly */

2016-09-20T20:55:15Z

The Ugly

← Older revision		Revision as of 20:55, 20 September 2016
Line 9:		Line 9:
	=== The Ugly ===		=== The Ugly ===
	* We run prelink with the -R flag which chooses a random base for libraries. This means that even if you run prelink two times on the same binary or library, they will be different due to the random library placement. This is done as a security enhancement. This prelink call is located in meta/classes/image-prelink.bbclass :		* We run prelink with the -R flag which chooses a random base for libraries. This means that even if you run prelink two times on the same binary or library, they will be different due to the random library placement. This is done as a security enhancement. This prelink call is located in meta/classes/image-prelink.bbclass :
	<~~code~~> ${STAGING_SBINDIR_NATIVE}/prelink --root ${IMAGE_ROOTFS} -amR -N ~~</code>~~		<pre>
	~~<code>~~ -c ${sysconfdir}/prelink.conf --dynamic-linker $dynamic_loader </~~code~~>		${STAGING_SBINDIR_NATIVE}/prelink --root ${IMAGE_ROOTFS} -amR -N
			-c ${sysconfdir}/prelink.conf --dynamic-linker $dynamic_loader
			</pre>

	== So, How DO I Compare my Prelinked Binaries? ==		== So, How DO I Compare my Prelinked Binaries? ==

Bavery: /* So, How DO I Compare my Prelinked Binaries? */

2016-09-20T20:46:48Z

So, How DO I Compare my Prelinked Binaries?

← Older revision		Revision as of 20:46, 20 September 2016
Line 16:		Line 16:
	In theory this can be done using		In theory this can be done using
	<pre>		<pre>
	$>prelink -y --verify ~~which "Verify file consistency by undoing and~~		$>prelink -y --verify binary
	~~redoing~~ ~~prelink and printing original to standard output"~~
	</pre>		</pre>

	from the host environment. Getting prelink to run successfully from the host side is complicated by the fact that it needs a number of arguments to be happy. My example will be for qemux86 and core-image-minimal. Paths will need to be adjusted for other combinations. Note that the dynamic linker and the target are relative to the -root arg.<br>		from the host environment. This will "Verify file consistency by undoing and
			redoing prelink and printing original to standard output". Getting prelink to run successfully from the host side is complicated by the fact that it needs a number of arguments to be happy. My example will be for qemux86 and core-image-minimal. Paths will need to be adjusted for other combinations. Note that the dynamic linker and the target are relative to the -root arg.<br>
	This will print out the md5 sum of the binary:		This will print out the md5 sum of the binary:
	<pre>		<pre>

Bavery: /* So, How DO I Compare my Prelinked Binaries? */

2016-09-20T20:45:32Z

So, How DO I Compare my Prelinked Binaries?

← Older revision		Revision as of 20:45, 20 September 2016
Line 15:		Line 15:
	Once they are prelinked, you need to use the prelink tool itself to do the comparison.		Once they are prelinked, you need to use the prelink tool itself to do the comparison.
	In theory this can be done using		In theory this can be done using
	<~~code~~> prelink -y --verify which "Verify file consistency by undoing and		<pre>
	redoing prelink and printing original to standard output" <~~code~~>		$>prelink -y --verify which "Verify file consistency by undoing and
			redoing prelink and printing original to standard output"
			</pre>

	from the host environment. Getting prelink to run successfully from the host side is complicated by the fact that it needs a number of arguments to be happy. My example will be for qemux86 and core-image-minimal. Paths will need to be adjusted for other combinations. Note that the dynamic linker and the target are relative to the -root arg.<br>		from the host environment. Getting prelink to run successfully from the host side is complicated by the fact that it needs a number of arguments to be happy. My example will be for qemux86 and core-image-minimal. Paths will need to be adjusted for other combinations. Note that the dynamic linker and the target are relative to the -root arg.<br>
	This will print out the md5 sum of the binary:		This will print out the md5 sum of the binary:
	<~~code~~> ./tmp/sysroots/x86_64-linux/usr/sbin/prelink --root ./tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs --dynamic-linker /lib/ld-linux.so.2 --verify --md5 /bin/busybox.nosuid </~~code~~>		<pre>
			$>./tmp/sysroots/x86_64-linux/usr/sbin/prelink
			--root ./tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs
			--dynamic-linker /lib/ld-linux.so.2 --verify --md5
			/bin/busybox.nosuid
			</pre>
	This will let you dump the original unprelinked binary for comparison:		This will let you dump the original unprelinked binary for comparison:
	<~~code~~> ./tmp/sysroots/x86_64-linux/usr/sbin/prelink --root ./tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs --dynamic-linker /lib/ld-linux.so.2 -y --verify /bin/busybox.nosuid > busybox.nosuid.unmodded </~~code~~>		<pre>
			$>./tmp/sysroots/x86_64-linux/usr/sbin/prelink
			--root ./tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs
			--dynamic-linker /lib/ld-linux.so.2 -y --verify
			/bin/busybox.nosuid > busybox.nosuid.unmodded
			</pre>

	== How do I turn off Prelink if I find this too Cumbersome?==		== How do I turn off Prelink if I find this too Cumbersome?==

Bavery: /* So, How DO I Compare my Prelinked Binaries? */

2016-09-20T20:42:35Z

So, How DO I Compare my Prelinked Binaries?

← Older revision		Revision as of 20:42, 20 September 2016
Line 15:		Line 15:
	Once they are prelinked, you need to use the prelink tool itself to do the comparison.		Once they are prelinked, you need to use the prelink tool itself to do the comparison.
	In theory this can be done using		In theory this can be done using
	<code> prelink -y --verify which "Verify file consistency by undoing and redoing prelink and printing original to standard output" <code>		<code> prelink -y --verify which "Verify file consistency by undoing and
			redoing prelink and printing original to standard output" <code>
	from the host environment. Getting prelink to run successfully from the host side is complicated by the fact that it needs a number of arguments to be happy. My example will be for qemux86 and core-image-minimal. Paths will need to be adjusted for other combinations. Note that the dynamic linker and the target are relative to the -root arg.<br>		from the host environment. Getting prelink to run successfully from the host side is complicated by the fact that it needs a number of arguments to be happy. My example will be for qemux86 and core-image-minimal. Paths will need to be adjusted for other combinations. Note that the dynamic linker and the target are relative to the -root arg.<br>
	This will print out the md5 sum of the binary:		This will print out the md5 sum of the binary:

Bavery: /* The Ugly */

2016-09-20T20:42:03Z

The Ugly

← Older revision		Revision as of 20:42, 20 September 2016
Line 9:		Line 9:
	=== The Ugly ===		=== The Ugly ===
	* We run prelink with the -R flag which chooses a random base for libraries. This means that even if you run prelink two times on the same binary or library, they will be different due to the random library placement. This is done as a security enhancement. This prelink call is located in meta/classes/image-prelink.bbclass :		* We run prelink with the -R flag which chooses a random base for libraries. This means that even if you run prelink two times on the same binary or library, they will be different due to the random library placement. This is done as a security enhancement. This prelink call is located in meta/classes/image-prelink.bbclass :
	<code> ${STAGING_SBINDIR_NATIVE}/prelink --root ${IMAGE_ROOTFS} -amR -N -c ${sysconfdir}/prelink.conf --dynamic-linker $dynamic_loader </code>		<code> ${STAGING_SBINDIR_NATIVE}/prelink --root ${IMAGE_ROOTFS} -amR -N </code>
			<code> -c ${sysconfdir}/prelink.conf --dynamic-linker $dynamic_loader </code>

	== So, How DO I Compare my Prelinked Binaries? ==		== So, How DO I Compare my Prelinked Binaries? ==
	Once they are prelinked, you need to use the prelink tool itself to do the comparison.		Once they are prelinked, you need to use the prelink tool itself to do the comparison.

Bavery: /* The Bad */

2016-09-20T20:41:17Z

The Bad

← Older revision		Revision as of 20:41, 20 September 2016
Line 5:		Line 5:

	=== The Bad ===		=== The Bad ===
	* Prelink adds a timestamp in the files it modifies. ~~This is used (~~I believe but have not verified) that these timestamps are used to speed up subsequent runs of prelinking by allowing prelink to skip libraries that have not changed (based on time) since it's last run ~~(which~~ isn't really bad, I guess, though I'd like to be able to ~~turn~~ this feature ~~off)~~. Unfortunately, since timestamps tend to be monotonically increasing, this also prevents the use of cmp to compare 2 binaries that are effectively the same.		* Prelink adds a timestamp in the files it modifies. I believe but have not verified that these timestamps are used to speed up subsequent runs of prelinking by allowing prelink to skip libraries that have not changed (based on time) since it's last run. This isn't really bad, I guess, though I'd like to be able to disable this feature in some cases. Unfortunately, since timestamps tend to be monotonically increasing, this also prevents the use of cmp to compare 2 binaries that are effectively the same.

	=== The Ugly ===		=== The Ugly ===

Bavery: /* The Bad */

2016-09-20T20:40:06Z

The Bad

← Older revision		Revision as of 20:40, 20 September 2016
Line 5:		Line 5:

	=== The Bad ===		=== The Bad ===
	* ~~prelink~~ adds a timestamp in the files it modifies. This is used (I believe but have not verified) that these timestamps are used to speed up subsequent runs of prelinking by allowing prelink to skip libraries that have not changed (based on time) since it's last run (which isn't really bad, I guess, though I'd like to be able to turn this feature off). Unfortunately, since timestamps tend to be monotonically increasing, this also prevents the use of cmp to compare 2 binaries that are effectively the same.		* Prelink adds a timestamp in the files it modifies. This is used (I believe but have not verified) that these timestamps are used to speed up subsequent runs of prelinking by allowing prelink to skip libraries that have not changed (based on time) since it's last run (which isn't really bad, I guess, though I'd like to be able to turn this feature off). Unfortunately, since timestamps tend to be monotonically increasing, this also prevents the use of cmp to compare 2 binaries that are effectively the same.

	=== The Ugly ===		=== The Ugly ===

Bavery: /* The Good */

2016-09-20T20:39:56Z

The Good

← Older revision		Revision as of 20:39, 20 September 2016
Line 2:		Line 2:
	Prelink is run across all the binaries and libraries as part of creating an image, by default. Because we prelink the libraries into the binaries, the binaries can no longer be binarily compared using standard tools like cmp.		Prelink is run across all the binaries and libraries as part of creating an image, by default. Because we prelink the libraries into the binaries, the binaries can no longer be binarily compared using standard tools like cmp.
	=== The Good ===		=== The Good ===
	This reduces startup time by "prelinking" the shared libs into the executable by relocating [ ~~Wikpedia page on prelink~~ https://en.wikipedia.org/wiki/Prelink]. In my brief experiments, I found a ~15% speedup in boot time alone for instance.		This reduces startup time by "prelinking" the shared libs into the executable by relocating [https://en.wikipedia.org/wiki/Prelink Wikpedia page on prelink]. In my brief experiments, I found a ~15% speedup in boot time alone for instance.

	=== The Bad ===		=== The Bad ===

Bavery: Created page with "=Prelink, What it does and how you can deal with it= Prelink is run across all the binaries and libraries as part of creating an image, by default. Because we prelink the lib..."

2016-09-20T20:36:50Z

Created page with "=Prelink, What it does and how you can deal with it= Prelink is run across all the binaries and libraries as part of creating an image, by default. Because we prelink the lib..."

New page

=Prelink, What it does and how you can deal with it=
Prelink is run across all the binaries and libraries as part of creating an image, by default. Because we prelink the libraries into the binaries, the binaries can no longer be binarily compared using standard tools like cmp.
=== The Good ===
This reduces startup time by "prelinking" the shared libs into the executable by relocating [ Wikpedia page on prelink https://en.wikipedia.org/wiki/Prelink]. In my brief experiments, I found a ~15% speedup in boot time alone for instance.

=== The Bad ===
* prelink adds a timestamp in the files it modifies. This is used (I believe but have not verified) that these timestamps are used to speed up subsequent runs of prelinking by allowing prelink to skip libraries that have not changed (based on time) since it's last run (which isn't really bad, I guess, though I'd like to be able to turn this feature off). Unfortunately, since timestamps tend to be monotonically increasing, this also prevents the use of cmp to compare 2 binaries that are effectively the same.

=== The Ugly ===
* We run prelink with the -R flag which chooses a random base for libraries. This means that even if you run prelink two times on the same binary or library, they will be different due to the random library placement. This is done as a security enhancement. This prelink call is located in meta/classes/image-prelink.bbclass :
<code> ${STAGING_SBINDIR_NATIVE}/prelink --root ${IMAGE_ROOTFS} -amR -N -c ${sysconfdir}/prelink.conf --dynamic-linker $dynamic_loader </code>
== So, How DO I Compare my Prelinked Binaries? ==
Once they are prelinked, you need to use the prelink tool itself to do the comparison.
In theory this can be done using
<code> prelink -y --verify which "Verify file consistency by undoing and redoing prelink and printing original to standard output" <code>
from the host environment. Getting prelink to run successfully from the host side is complicated by the fact that it needs a number of arguments to be happy. My example will be for qemux86 and core-image-minimal. Paths will need to be adjusted for other combinations. Note that the dynamic linker and the target are relative to the -root arg.<br>
This will print out the md5 sum of the binary:
<code> ./tmp/sysroots/x86_64-linux/usr/sbin/prelink --root ./tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs --dynamic-linker /lib/ld-linux.so.2 --verify --md5 /bin/busybox.nosuid </code>
This will let you dump the original unprelinked binary for comparison:
<code> ./tmp/sysroots/x86_64-linux/usr/sbin/prelink --root ./tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs --dynamic-linker /lib/ld-linux.so.2 -y --verify /bin/busybox.nosuid > busybox.nosuid.unmodded </code>

== How do I turn off Prelink if I find this too Cumbersome?==
in <code> conf/local.conf</code> override the default value of <code> USER_CLASSES </code>. This defaults to:
<code> USER_CLASSES ?= "buildstats image-mklibs image-prelink" </code>
and change it to
<code> USER_CLASSES = "buildstats image-mklibs " </code>

This will result in your binaries <strong>not</strong> being prelinked, which means they will be slower to start up but easier if you need binary comparisons.

== What If I Want to be Able to Run Prelink on my Target ==
You can add <code> IMAGE_INSTALL_append = " prelink" </code> (note the space) to you conf/local.conf file. For qemux86 this worked fine for me. For x86-64 there was an issue and the workaround is documented in this bug: [https://bugzilla.yoctoproject.org/show_bug.cgi?id=10282].