Performance Test

Poky/Bitbake Performance:
Performance is an essential point for measuring the quality of a build system.
This page is to track performance trend of poky build. Detailed development work is on https://wiki.yoctoproject.org/wiki/Performance.

Hardware and software configuration:
CPU: 4-core * 2-threads Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
Memory: 4GB
Harddisk: 1TB

OS: Ubuntu 11.10 x86_64
Kernel: 2.6.35-22

local.conf:
CONF_VERSION = "1"
DL_DIR ?= "/sda1/sources/downloads" # Using a local download dir to avoid fetch.
BB_NUMBER_THREADS = "9"
PARALLEL_MAKE = "-j 9"
MACHINE ?= "qemux86"
DISTRO ?= "poky"
BBMASK = ""
EXTRA_IMAGE_FEATURES = "tools-debug tools-profile tools-testapps debug-tweaks"
EXTRA_IMAGE_FEATURES_c7x0 = "tools-testapps debug-tweaks"
EXTRA_IMAGE_FEATURES_mx31phy = "debug-tweaks"
EXTRA_IMAGE_FEATURES_mx31ads = "tools-testapps debug-tweaks"
PACKAGE_CLASSES ?= "package_rpm package_ipk"
USER_CLASSES ?= "image-prelink"
BBINCLUDELOGS = "yes"
ENABLE_BINARY_LOCALE_GENERATION = "1"
NO32LIBS = "1"

Source code and commits:
Green-3.3.1: http://autobuilder.yoctoproject.org/downloads/poky/poky-green-3.3.1.tar.bz2
Yocto-0.9: http://autobuilder.yoctoproject.org/downloads/poky/poky-laverne-4.0.tar.bz2
01/07/2011 build: commit b22e345e05efcc3f66278af8f09fb083afe32b68
02/14/2011 build: commit 6cb8fd6def4912e4aa76330649ba42a9ed2694fd

Other detail commit number for optimization will be annotated in the table note. See following.

Performance results:

Table 1: File Parsing Time Optmization Results

File parsing speed	BB files	Parsing time (s)	Standardized BB files	Standardized time (s)	Performance compare
Green-3.3.1	925	16.5	770	13.7	100%
Yocto-0.9	844	44.8	770	40.9	298%
Optimization A (rm distro variables)	757	29.6	770	30.1	220%
01/07/2011 build	770	16.6	770	16.6	121%
Optimization B ("??=" re-implementation)	770	15.6	770	15.6	114%

Note:
Optimization A: commit: 53aff7d6775eb1c2c8f419f325b91c062d85eed5
01/07/2011 build: Latest master with parallel parsing mechanism
Optimization B: "??=" re-implementation. (Pending for review)

Table 2: Build Time Optmization Results

Build time (poky-image-minimal)	Recipe number	Build time (s)	Standardized recipe number	Standardized time (s)	Performance compare
Green-3.3.1	88	32m57s	147	53m54s	100%
Yocto-0.9	147	88m50s	147	88m50s	165%
Exec (before pseudo wrapper)	147	114m6s	147	114m6s	212%
Fork (after pseudo wrapper)	147	99m52s	147	99m52s	185%
01/07/2011 build	147	134m39s	147	134m39s	250%
pipe buffer 1024 (before pipe buffer fix)	170	101m8s	147	87m27s	162%
pipe buffer 102400 (after pipe buffer fix)	170	97m32s	147	84m20s	156%
02/14/2011 build	227	75m10s	147	48m40s	90%

Note:
Exec (before pseudo wrapper): commit 84263dbf43eba5cd99ce59062cef807a966e7312
Fork (after pseudo wrapper): commit 995d4679d4bb6a28dd6fbdb1becc4231369311b7
pipe buffer 1024 (before pipe buffer fix): commit 754047b6ec01df5f3159cce93b17b8493d0af5e1
pipe buffer 102400 (after pipe buffer fix): commit 06c6db7929c75f576a395fb442abe447b833fc3b

Table 3: Disk Footprint Optmization Results (poky-image-minimal)

Disk space (poky-image-minimal)	Recipe number	Disk footprint (GB)	Standardized recipe number	Standardized disk footprint (GB)	Performance compare
Green-3.3.1	88	7.6	147	12.7	100%
Yocto-0.9	147	29	147	29	228%
Optimization A (rm sstate-build-*)	147	24	147	24	189%
01/07/2011 build	147	24	147	24	189%
Optimization B (hard link files)	147	19	147	19	150%

Note:
Optimization A: commit: 7e7bb24b8512dd1452a11d8aeafc8dedcbc4d623).
Optimization B: Using hardlink to replace copy between "package-->packages-split", "image-->sysroot-destdir", and "sysroot-destdir-->/tmp/sysroots" (Pending for review).

There are some other data we may add later.
1) Using socket data accessing approach to avoid data re-parsing when fork/exec new bitbake processes.