|
|
| Line 1: |
Line 1: |
| '''Poky/Bitbake Performance:'''<br>
| | See https://wiki.yoctoproject.org/wiki/Profiling |
| Performance is an essential point for measuring the quality of a build system.<br>
| |
| We are continously working on improving poky/bitbake performance, including major parts of build time, disk footprint, and file parsing speed.<br>
| |
| This page provides some performance optimization results. Part of the optimization patches have been accepted in master tree, while some others are pending for review.<br>
| |
| <br>
| |
| The following optimization patches are mainly provided by:<br>
| |
| Richard Purdie <richard.purdie@linuxfoundation.org><br>
| |
| Qing He <qing.he@intel.com><br>
| |
| Dongxiao Xu <dongxiao.xu@intel.com><br>
| |
| <br>
| |
| '''Hardware and software configuration:'''<br>
| |
| CPU: 4-core * 2-threads Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz<br>
| |
| Memory: 4GB<br>
| |
| Harddisk: 1TB<br>
| |
| <br>
| |
| OS: Ubuntu 10.04 x86_64<br>
| |
| Kernel: 2.6.32-21<br>
| |
| <br>
| |
| local.conf:<br>
| |
| CONF_VERSION = "1"<br>
| |
| DL_DIR ?= "/sda1/sources/downloads" # Using a local download dir to avoid fetch.<br>
| |
| BB_NUMBER_THREADS = "9"<br>
| |
| PARALLEL_MAKE = "-j 9"<br>
| |
| MACHINE ?= "qemux86"<br>
| |
| DISTRO ?= "poky"<br>
| |
| BBMASK = ""<br>
| |
| EXTRA_IMAGE_FEATURES = "tools-debug tools-profile tools-testapps debug-tweaks"<br>
| |
| EXTRA_IMAGE_FEATURES_c7x0 = "tools-testapps debug-tweaks"<br>
| |
| EXTRA_IMAGE_FEATURES_mx31phy = "debug-tweaks"<br>
| |
| EXTRA_IMAGE_FEATURES_mx31ads = "tools-testapps debug-tweaks"<br>
| |
| PACKAGE_CLASSES ?= "package_rpm package_ipk"<br>
| |
| USER_CLASSES ?= "image-prelink"<br>
| |
| BBINCLUDELOGS = "yes"<br>
| |
| ENABLE_BINARY_LOCALE_GENERATION = "1"<br>
| |
| NO32LIBS = "1"<br>
| |
| <br>
| |
| '''Source code and commits:'''<br>
| |
| Green-3.3.1: http://autobuilder.yoctoproject.org/downloads/poky/poky-green-3.3.1.tar.bz2<br>
| |
| Yocto-0.9: http://autobuilder.yoctoproject.org/downloads/poky/poky-laverne-4.0.tar.bz2<br>
| |
| 01/07/2011 build: commit b22e345e05efcc3f66278af8f09fb083afe32b68<br>
| |
| 02/14/2011 build: commit 6cb8fd6def4912e4aa76330649ba42a9ed2694fd<br>
| |
| <br>
| |
| Other detail commit number for optimization will be annotated in the table note. See following.<br>
| |
| <br>
| |
| | |
| '''Performance results:'''<br>
| |
| {| cellpadding="10" cellspacing="0" border="1"
| |
| |+Table 1: File Parsing Time Optmization Results
| |
| |align="center"|File parsing speed
| |
| |align="center"|BB files
| |
| |align="center"|Parsing time (s)
| |
| |align="center"|Standardized BB files
| |
| |align="center"|Standardized time (s)
| |
| |align="center"|Performance compare
| |
| |-
| |
| |align="center"|Green-3.3.1
| |
| |align="center"|925
| |
| |align="center"|16.5
| |
| |align="center"|770
| |
| |align="center"|13.7
| |
| |align="center"|100%
| |
| |-
| |
| |align="center"|Yocto-0.9
| |
| |align="center"|844
| |
| |align="center"|44.8
| |
| |align="center"|770
| |
| |align="center"|40.9
| |
| |align="center"|298%
| |
| |-
| |
| |align="center"|Optimization A (rm distro variables)
| |
| |align="center"|757
| |
| |align="center"|29.6
| |
| |align="center"|770
| |
| |align="center"|30.1
| |
| |align="center"|220%
| |
| |-
| |
| |align="center"|01/07/2011 build
| |
| |align="center"|770
| |
| |align="center"|16.6
| |
| |align="center"|770
| |
| |align="center"|16.6
| |
| |align="center"|121%
| |
| |-
| |
| |align="center"|Optimization B ("??=" re-implementation)
| |
| |align="center"|770
| |
| |align="center"|15.6
| |
| |align="center"|770
| |
| |align="center"|15.6
| |
| |align="center"|114%
| |
| |}
| |
| Note:<br>
| |
| Optimization A: commit: 53aff7d6775eb1c2c8f419f325b91c062d85eed5<br>
| |
| 01/07/2011 build: Latest master with parallel parsing mechanism<br>
| |
| Optimization B: "??=" re-implementation. (Pending for review)<br>
| |
| <br>
| |
| | |
| <br>
| |
| {| cellpadding="10" cellspacing="0" border="1"
| |
| |+Table 2: Build Time Optmization Results
| |
| |align="center"|Build time (poky-image-minimal)
| |
| |align="center"|Recipe number
| |
| |align="center"|Build time (s)
| |
| |align="center"|Standardized recipe number
| |
| |align="center"|Standardized time (s)
| |
| |align="center"|Performance compare
| |
| |-
| |
| |align="center"|Green-3.3.1
| |
| |align="center"|88
| |
| |align="center"|32m57s
| |
| |align="center"|147
| |
| |align="center"|53m54s
| |
| |align="center"|100%
| |
| |-
| |
| |align="center"|Yocto-0.9
| |
| |align="center"|147
| |
| |align="center"|88m50s
| |
| |align="center"|147
| |
| |align="center"|88m50s
| |
| |align="center"|165%
| |
| |-
| |
| |align="center"|Exec (before pseudo wrapper)
| |
| |align="center"|147
| |
| |align="center"|114m6s
| |
| |align="center"|147
| |
| |align="center"|114m6s
| |
| |align="center"|212%
| |
| |-
| |
| |align="center"|Fork (after pseudo wrapper)
| |
| |align="center"|147
| |
| |align="center"|99m52s
| |
| |align="center"|147
| |
| |align="center"|99m52s
| |
| |align="center"|185%
| |
| |-
| |
| |align="center"|01/07/2011 build
| |
| |align="center"|147
| |
| |align="center"|134m39s
| |
| |align="center"|147
| |
| |align="center"|134m39s
| |
| |align="center"|250%
| |
| |-
| |
| |align="center"|pipe buffer 1024 (before pipe buffer fix)
| |
| |align="center"|170
| |
| |align="center"|101m8s
| |
| |align="center"|147
| |
| |align="center"|87m27s
| |
| |align="center"|162%
| |
| |-
| |
| |align="center"|pipe buffer 102400 (after pipe buffer fix)
| |
| |align="center"|170
| |
| |align="center"|97m32s
| |
| |align="center"|147
| |
| |align="center"|84m20s
| |
| |align="center"|156%
| |
| |-
| |
| |align="center"|02/14/2011 build
| |
| |align="center"|227
| |
| |align="center"|75m10s
| |
| |align="center"|147
| |
| |align="center"|48m40s
| |
| |align="center"|90%
| |
| |}
| |
| | |
| Note:<br>
| |
| Exec (before pseudo wrapper): commit 84263dbf43eba5cd99ce59062cef807a966e7312<br>
| |
| Fork (after pseudo wrapper): commit 995d4679d4bb6a28dd6fbdb1becc4231369311b7<br>
| |
| pipe buffer 1024 (before pipe buffer fix): commit 754047b6ec01df5f3159cce93b17b8493d0af5e1<br>
| |
| pipe buffer 102400 (after pipe buffer fix): commit 06c6db7929c75f576a395fb442abe447b833fc3b<br>
| |
| <br>
| |
| {| cellpadding="10" cellspacing="0" border="1"
| |
| |+Table 3: Disk Footprint Optmization Results (poky-image-minimal)
| |
| |align="center"|Disk space (poky-image-minimal)
| |
| |align="center"|Recipe number
| |
| |align="center"|Disk footprint (GB)
| |
| |align="center"|Standardized recipe number
| |
| |align="center"|Standardized disk footprint (GB)
| |
| |align="center"|Performance compare
| |
| |-
| |
| |align="center"|Green-3.3.1
| |
| |align="center"|88
| |
| |align="center"|7.6
| |
| |align="center"|147
| |
| |align="center"|12.7
| |
| |align="center"|100%
| |
| |-
| |
| |align="center"|Yocto-0.9
| |
| |align="center"|147
| |
| |align="center"|29
| |
| |align="center"|147
| |
| |align="center"|29
| |
| |align="center"|228%
| |
| |-
| |
| |align="center"|Optimization A (rm sstate-build-*)
| |
| |align="center"|147
| |
| |align="center"|24
| |
| |align="center"|147
| |
| |align="center"|24
| |
| |align="center"|189%
| |
| |-
| |
| |align="center"|01/07/2011 build
| |
| |align="center"|147
| |
| |align="center"|24
| |
| |align="center"|147
| |
| |align="center"|24
| |
| |align="center"|189%
| |
| |-
| |
| |align="center"|Optimization B (hard link files)
| |
| |align="center"|147
| |
| |align="center"|19
| |
| |align="center"|147
| |
| |align="center"|19
| |
| |align="center"|150%
| |
| |}
| |
| Note:<br>
| |
| Optimization A: commit: 7e7bb24b8512dd1452a11d8aeafc8dedcbc4d623).<br>
| |
| Optimization B: Using hardlink to replace copy between "package-->packages-split", "image-->sysroot-destdir", and "sysroot-destdir-->/tmp/sysroots" (Pending for review).<br>
| |
| <br>
| |
| | |
| There are some other data we may add later.<br>
| |
| 1) Using socket data accessing approach to avoid data re-parsing when fork/exec new bitbake processes.
| |
