Performance: Difference between revisions

From Yocto Project
Jump to navigationJump to search
No edit summary
(Replaced content with "See https://wiki.yoctoproject.org/wiki/Profiling")
 
(One intermediate revision by one other user not shown)
Line 1: Line 1:
'''Poky/Bitbake Performance:'''<br>
See https://wiki.yoctoproject.org/wiki/Profiling
Performance is an essential point for measuring the quality of a build system.<br>
We are continously working on improving poky/bitbake performance, including major parts of build time, disk footprint, and file parsing speed.<br>
This page provides some performance optimization results. Part of the optimization patches have been accepted in master tree, while some others are pending for review.<br>
<br>
The following optimization patches are mainly provided by:<br>
Richard Purdie <richard.purdie@linuxfoundation.org><br>
Qing He <qing.he@intel.com><br>
Dongxiao Xu <dongxiao.xu@intel.com><br>
<br>
'''Hardware and software configuration:'''<br>
CPU: 4-core * 2-threads Intel(R) Core(TM) i7 CPU 860  @ 2.80GHz<br>
Memory: 4GB<br>
Harddisk: 1TB<br>
<br>
OS: Ubuntu 10.04 x86_64<br>
Kernel: 2.6.32-21<br>
<br>
local.conf:<br>
CONF_VERSION = "1"<br>
DL_DIR ?= "/sda1/sources/downloads" # Using a local download dir to avoid fetch.<br>
BB_NUMBER_THREADS = "9"<br>
PARALLEL_MAKE = "-j 9"<br>
MACHINE ?= "qemux86"<br>
DISTRO ?= "poky"<br>
BBMASK = ""<br>
EXTRA_IMAGE_FEATURES = "tools-debug tools-profile tools-testapps debug-tweaks"<br>
EXTRA_IMAGE_FEATURES_c7x0 = "tools-testapps debug-tweaks"<br>
EXTRA_IMAGE_FEATURES_mx31phy = "debug-tweaks"<br>
EXTRA_IMAGE_FEATURES_mx31ads = "tools-testapps debug-tweaks"<br>
PACKAGE_CLASSES ?= "package_rpm package_ipk"<br>
USER_CLASSES ?= "image-prelink"<br>
BBINCLUDELOGS = "yes"<br>
ENABLE_BINARY_LOCALE_GENERATION = "1"<br>
NO32LIBS = "1"<br>
<br>
'''Source code and commits:'''<br>
Green-3.3.1: http://autobuilder.yoctoproject.org/downloads/poky/poky-green-3.3.1.tar.bz2<br>
Yocto-0.9: http://autobuilder.yoctoproject.org/downloads/poky/poky-laverne-4.0.tar.bz2<br>
01/07/2011 build: commit b22e345e05efcc3f66278af8f09fb083afe32b68<br>
02/14/2011 build: commit 6cb8fd6def4912e4aa76330649ba42a9ed2694fd<br>
<br>
Other detail commit number for optimization will be annotated in the table note. See following.<br>
<br>
 
'''Performance results:'''<br>
{| cellpadding="10" cellspacing="0" border="1"
|+Table 1: File Parsing Time Optmization Results
|align="center"|File parsing speed
|align="center"|BB files
|align="center"|Parsing time (s)
|align="center"|Standardized BB files
|align="center"|Standardized time (s)
|align="center"|Performance compare
|-
|align="center"|Green-3.3.1
|align="center"|925
|align="center"|16.5
|align="center"|770
|align="center"|13.7
|align="center"|100%
|-
|align="center"|Yocto-0.9
|align="center"|844
|align="center"|44.8
|align="center"|770
|align="center"|40.9
|align="center"|298%
|-
|align="center"|Optimization A (rm distro variables)
|align="center"|757
|align="center"|29.6
|align="center"|770
|align="center"|30.1
|align="center"|220%
|-
|align="center"|01/07/2011 build
|align="center"|770
|align="center"|16.6
|align="center"|770
|align="center"|16.6
|align="center"|121%
|-
|align="center"|Optimization B ("??=" re-implementation)
|align="center"|770
|align="center"|15.6
|align="center"|770
|align="center"|15.6
|align="center"|114%
|}
Note:<br>
Optimization A: commit: 53aff7d6775eb1c2c8f419f325b91c062d85eed5<br>
Current: Latest master with parallel parsing mechanism<br>
Optimization B: "??=" re-implementation. (Pending for review)<br>
<br>
 
<br>
{| cellpadding="10" cellspacing="0" border="1"
|+Table 2: Build Time Optmization Results
|align="center"|Build time (poky-image-minimal)
|align="center"|Recipe number
|align="center"|Build time (s)
|align="center"|Standardized recipe number
|align="center"|Standardized time (s)
|align="center"|Performance compare
|-
|align="center"|Green-3.3.1
|align="center"|88
|align="center"|32m57s
|align="center"|147
|align="center"|53m54s
|align="center"|100%
|-
|align="center"|Yocto-0.9
|align="center"|147
|align="center"|88m50s
|align="center"|147
|align="center"|88m50s
|align="center"|165%
|-
|align="center"|Exec (before pseudo wrapper)
|align="center"|147
|align="center"|114m6s
|align="center"|147
|align="center"|114m6s
|align="center"|212%
|-
|align="center"|Fork (after pseudo wrapper)
|align="center"|147
|align="center"|99m52s
|align="center"|147
|align="center"|99m52s
|align="center"|185%
|-
|align="center"|01/07/2011 build
|align="center"|147
|align="center"|134m39s
|align="center"|147
|align="center"|134m39s
|align="center"|250%
|-
|align="center"|pipe buffer 1024 (before pipe buffer fix)
|align="center"|170
|align="center"|101m8s
|align="center"|147
|align="center"|87m27s
|align="center"|162%
|-
|align="center"|pipe buffer 102400 (after pipe buffer fix)
|align="center"|170
|align="center"|97m32s
|align="center"|147
|align="center"|84m20s
|align="center"|156%
|-
|align="center"|02/14/2011 build
|align="center"|227
|align="center"|75m10s
|align="center"|147
|align="center"|48m40s
|align="center"|90%
|}
 
Note:<br>
Exec (before pseudo wrapper): commit 84263dbf43eba5cd99ce59062cef807a966e7312<br>
Fork (after pseudo wrapper):  commit 995d4679d4bb6a28dd6fbdb1becc4231369311b7<br>
pipe buffer 1024 (before pipe buffer fix): commit 754047b6ec01df5f3159cce93b17b8493d0af5e1<br>
pipe buffer 102400 (after pipe buffer fix): commit 06c6db7929c75f576a395fb442abe447b833fc3b<br>
<br>
{| cellpadding="10" cellspacing="0" border="1"
|+Table 3: Disk Footprint Optmization Results (poky-image-minimal)
|align="center"|Disk space (poky-image-minimal)
|align="center"|Recipe number
|align="center"|Disk footprint (GB)
|align="center"|Standardized recipe number
|align="center"|Standardized disk footprint (GB)
|align="center"|Performance compare
|-
|align="center"|Green-3.3.1
|align="center"|88
|align="center"|7.6
|align="center"|147
|align="center"|12.7
|align="center"|100%
|-
|align="center"|Yocto-0.9
|align="center"|147
|align="center"|29
|align="center"|147
|align="center"|29
|align="center"|228%
|-
|align="center"|Optimization A (rm sstate-build-*)
|align="center"|147
|align="center"|24
|align="center"|147
|align="center"|24
|align="center"|189%
|-
|align="center"|01/07/2011 build
|align="center"|147
|align="center"|24
|align="center"|147
|align="center"|24
|align="center"|189%
|-
|align="center"|Optimization B (hard link files)
|align="center"|147
|align="center"|19
|align="center"|147
|align="center"|19
|align="center"|150%
|}
Note:<br>
Optimization A: commit: 7e7bb24b8512dd1452a11d8aeafc8dedcbc4d623).<br>
Optimization B: Using hardlink to replace copy between "package-->packages-split", "image-->sysroot-destdir", and "sysroot-destdir-->/tmp/sysroots" (Pending for review).<br>
<br>
 
There are some other data we may add later.<br>
1) Using socket data accessing approach to avoid data re-parsing when fork/exec new bitbake processes.

Latest revision as of 14:28, 18 June 2015

See https://wiki.yoctoproject.org/wiki/Profiling

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Performance&oldid=14897"