Performance

Poky/Bitbake Performance:
Performance is an essential point for measuring the quality of a build system.
We are continously working on improving poky/bitbake performance, including major parts of build time, disk footprint, and file parsing speed.
This page provides some performance optimization results. Part of the optimization patches have been accepted in master tree, while some others are pending for review.

The following optimization patches are mainly provided by:
Richard Purdie <richard.purdie@linuxfoundation.org>
Qing He <qing.he@intel.com>
Dongxiao Xu <dongxiao.xu@intel.com>

Hardware and software configuration:
CPU: 4-core * 2-threads Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
Memory: 4GB
Harddisk: 1TB

OS: Ubuntu 10.04 x86_64
Kernel: 2.6.32-21

local.conf:
CONF_VERSION = "1"
DL_DIR ?= "/sda1/sources/downloads" # Using a local download dir to avoid fetch.
BB_NUMBER_THREADS = "9"
PARALLEL_MAKE = "-j 9"
MACHINE ?= "qemux86"
DISTRO ?= "poky"
BBMASK = ""
EXTRA_IMAGE_FEATURES = "tools-debug tools-profile tools-testapps debug-tweaks"
EXTRA_IMAGE_FEATURES_c7x0 = "tools-testapps debug-tweaks"
EXTRA_IMAGE_FEATURES_mx31phy = "debug-tweaks"
EXTRA_IMAGE_FEATURES_mx31ads = "tools-testapps debug-tweaks"
PACKAGE_CLASSES ?= "package_rpm package_ipk"
USER_CLASSES ?= "image-prelink"
BBINCLUDELOGS = "yes"
ENABLE_BINARY_LOCALE_GENERATION = "1"
NO32LIBS = "1"

Source code and commits:
Green-3.3.1: http://autobuilder.yoctoproject.org/downloads/poky/poky-green-3.3.1.tar.bz2
Yocto-0.9: http://autobuilder.yoctoproject.org/downloads/poky/poky-laverne-4.0.tar.bz2
Current: commit b22e345e05efcc3f66278af8f09fb083afe32b68

Other detail commit number for optimization will be annotated in the table note. See following.

Performance results:

Table 1: File Parsing Time Optmization Results

File parsing speed	BB files	Parsing time (s)	Standardized BB files	Standardized time (s)	Performance compare
Green-3.3.1	925	16.5	770	13.7	100%
Yocto-0.9	844	44.8	770	40.9	298%
Optimization A (rm distro variables)	757	29.6	770	30.1	220%
Current	770	16.6	770	16.6	121%
Optimization B ("??=" re-implementation)	770	15.6	770	15.6	114%

Note:
Optimization A: commit: 53aff7d6775eb1c2c8f419f325b91c062d85eed5
Current: Latest master with parallel parsing mechanism
Optimization B: "??=" re-implementation. (Pending for review)

Table 2: Build Time Optmization Results

Build time (poky-image-minimal)	Recipe number	Build time (s)	Standardized recipe number	Standardized time (s)	Performance compare
Green-3.3.1	88	32m57s	147	53m54s	100%
Yocto-0.9	147	88m50s	147	88m50s	165%
Exec (before pseudo wrapper)	147	114m6s	147	114m6s	212%
Fork (after pseudo wrapper)	147	99m52s	147	99m52s	185%
Current	147	134m39s	147	134m39s	250%

Note:
Exec (before pseudo wrapper): commit 84263dbf43eba5cd99ce59062cef807a966e7312
Fork (after pseudo wrapper): commit 995d4679d4bb6a28dd6fbdb1becc4231369311b7
The current build time is somewhat long, which we will look at.

Table 3: Disk Footprint Optmization Results (poky-image-minimal)

Disk space (poky-image-minimal)	Recipe number	Disk footprint (GB)	Standardized recipe number	Standardized disk footprint (GB)	Performance compare
Green-3.3.1	88	7.6	147	12.7	100%
Yocto-0.9	147	29	147	29	228%
Optimization A (rm sstate-build-*)	147	24	147	24	189%
Current	147	24	147	24	189%
Optimization B (hard link files)	147	19	147	19	150%

Note:
Optimization A: commit: 7e7bb24b8512dd1452a11d8aeafc8dedcbc4d623).
Optimization B: Using hardlink to replace copy between "package-->packages-split", "image-->sysroot-destdir", and "sysroot-destdir-->/tmp/sysroots" (Pending for review).

There are some other data we may add later.
1) Using socket data accessing approach to avoid data re-parsing when fork/exec new bitbake processes.