Performance Test

Poky/Bitbake Performance:
Performance is an essential point for measuring the quality of a build system.
We are continously working on improving poky/bitbake performance, including major parts of build time, disk footprint, and file parsing speed.
This page provides some performance optimization results. Part of the optimization patches have been accepted in master tree, while some others are pending for review.

The following optimization patches are mainly provided by:
Richard Purdie <richard.purdie@linuxfoundation.org>
Qing He <qing.he@intel.com>
Dongxiao Xu <dongxiao.xu@intel.com>

Hardware and software configuration:
CPU: 4-core * 2-threads Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
Memory: 4GB
Harddisk: 1TB

OS: Ubuntu 10.04 x86_64
Kernel: 2.6.32-21

local.conf:
CONF_VERSION = "1"
DL_DIR ?= "/sda1/sources/downloads" # Using a local download dir to avoid fetch.
BB_NUMBER_THREADS = "9"
PARALLEL_MAKE = "-j 9"
MACHINE ?= "qemux86"
DISTRO ?= "poky"
BBMASK = ""
EXTRA_IMAGE_FEATURES = "tools-debug tools-profile tools-testapps debug-tweaks"
EXTRA_IMAGE_FEATURES_c7x0 = "tools-testapps debug-tweaks"
EXTRA_IMAGE_FEATURES_mx31phy = "debug-tweaks"
EXTRA_IMAGE_FEATURES_mx31ads = "tools-testapps debug-tweaks"
PACKAGE_CLASSES ?= "package_rpm package_ipk"
USER_CLASSES ?= "image-prelink"
BBINCLUDELOGS = "yes"
ENABLE_BINARY_LOCALE_GENERATION = "1"
NO32LIBS = "1"

Source code and commits:
Green-3.3.1: http://autobuilder.yoctoproject.org/downloads/poky/poky-green-3.3.1.tar.bz2
Yocto-0.9: http://autobuilder.yoctoproject.org/downloads/poky/poky-laverne-4.0.tar.bz2
01/07/2011 build: commit b22e345e05efcc3f66278af8f09fb083afe32b68
02/14/2011 build: commit 6cb8fd6def4912e4aa76330649ba42a9ed2694fd

Other detail commit number for optimization will be annotated in the table note. See following.

Performance results:

Table 1: File Parsing Time Optmization Results

File parsing speed	BB files	Parsing time (s)	Standardized BB files	Standardized time (s)	Performance compare
Green-3.3.1	925	16.5	770	13.7	100%
Yocto-0.9	844	44.8	770	40.9	298%
Optimization A (rm distro variables)	757	29.6	770	30.1	220%
01/07/2011 build	770	16.6	770	16.6	121%
Optimization B ("??=" re-implementation)	770	15.6	770	15.6	114%

Note:
Optimization A: commit: 53aff7d6775eb1c2c8f419f325b91c062d85eed5
01/07/2011 build: Latest master with parallel parsing mechanism
Optimization B: "??=" re-implementation. (Pending for review)

Table 2: Build Time Optmization Results

Build time (poky-image-minimal)	Recipe number	Build time (s)	Standardized recipe number	Standardized time (s)	Performance compare
Green-3.3.1	88	32m57s	147	53m54s	100%
Yocto-0.9	147	88m50s	147	88m50s	165%
Exec (before pseudo wrapper)	147	114m6s	147	114m6s	212%
Fork (after pseudo wrapper)	147	99m52s	147	99m52s	185%
01/07/2011 build	147	134m39s	147	134m39s	250%
pipe buffer 1024 (before pipe buffer fix)	170	101m8s	147	87m27s	162%
pipe buffer 102400 (after pipe buffer fix)	170	97m32s	147	84m20s	156%
02/14/2011 build	227	75m10s	147	48m40s	90%

Note:
Exec (before pseudo wrapper): commit 84263dbf43eba5cd99ce59062cef807a966e7312
Fork (after pseudo wrapper): commit 995d4679d4bb6a28dd6fbdb1becc4231369311b7
pipe buffer 1024 (before pipe buffer fix): commit 754047b6ec01df5f3159cce93b17b8493d0af5e1
pipe buffer 102400 (after pipe buffer fix): commit 06c6db7929c75f576a395fb442abe447b833fc3b

Table 3: Disk Footprint Optmization Results (poky-image-minimal)

Disk space (poky-image-minimal)	Recipe number	Disk footprint (GB)	Standardized recipe number	Standardized disk footprint (GB)	Performance compare
Green-3.3.1	88	7.6	147	12.7	100%
Yocto-0.9	147	29	147	29	228%
Optimization A (rm sstate-build-*)	147	24	147	24	189%
01/07/2011 build	147	24	147	24	189%
Optimization B (hard link files)	147	19	147	19	150%

Note:
Optimization A: commit: 7e7bb24b8512dd1452a11d8aeafc8dedcbc4d623).
Optimization B: Using hardlink to replace copy between "package-->packages-split", "image-->sysroot-destdir", and "sysroot-destdir-->/tmp/sysroots" (Pending for review).

There are some other data we may add later.
1) Using socket data accessing approach to avoid data re-parsing when fork/exec new bitbake processes.