Performance Test
Build Perfomance Data
Script used for running the builds: (dylan/master branch): scripts/contrib/build-perf-test.sh
Test systems:
System1: SandyBridge i7-2600 CPU @ 3.40GHz HT-enabled (4 cores/8 threads), 8GB RAM, 1 x 500 GB HDD, Ubuntu 12.04 x64 (with all updates available installed)
System settings are distro's default:
- FS mount options are Ubuntu's default: ext4 rw,relatime,user_xattr,barrier=1,data=ordered
- IO scheduler: cfq (default) and CPU governer: ondemand (default)
System2: IvyBridge i7-3770K CPU @ 3.50GHz HT-enabled (4 cores/8 threads), 32GB RAM, 1 x 1 TB HDD, Fedora 18 x64 (with all updates available installed)
System settings are distro's default:
- FS mount options are Fedora's default: ext4 rw,seclabel,relatime,data=ordered
- IO scheduler: cfq (default) and CPU governer: ondemand (default)
local.conf:
BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE = "qemux86"
DISTRO ?= "poky"
PACKAGE_CLASSES ?= "package_rpm"
EXTRA_IMAGE_FEATURES = "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS = "\
STOPTASKS,${TMPDIR},1G,100K \
STOPTASKS,${DL_DIR},1G,100K \
STOPTASKS,${SSTATE_DIR},1G,100K \
ABORT,${TMPDIR},100M,1K \
ABORT,${DL_DIR},100M,1K \
ABORT,${SSTATE_DIR},100M,1K"
CONF_VERSION = "1"
DL_DIR = "/data/yocto/poky/build-perf-test/downloads"
1.5 nightly build time results
See big table in the next section (1.4) for what each time means:
(thunderbird = System2, ubuntu12=System1 from above description)
| hostname | commit | time core-image-sato | time virtual/kernel | time core-image-sato rm_work | time core-image-sato -c rootfs | cache1 | cache2 | cache3 |
|---|---|---|---|---|---|---|---|---|
| thunderbird | 176a46b088135f2891983bb8540bc45389341d90 | 1:07:09 | 11:39.62 | 1:05:35 | 4:57.37 | 0:31.22 | 0:16.63 | 0:02.00 |
| thunderbird | 21089515bd18ba18e86ec8f72ff8141b9fb1dea8 | 1:06:27 | 11:26.78 | 1:05:48 | 4:53.59 | 0:31.22 | 0:17.96 | 0:02.20 |
| thunderbird | e8197722de1f6006ceae0d4266de91c1db7278e3 | 1:06:00 | 13:17.22 | 1:05:39 | 4:52.55 | 0:31.10 | 0:16.81 | 0:02.19 |
| thunderbird | 676fd3f394a0c0576b63c77ed2184bf5f42c29b6 | 1:07:43 | 11:42.89 | 1:06:16 | 4:51.57 | 0:31.40 | 0:16.90 | 0:02.19 |
| thunderbird | 676fd3f394a0c0576b63c77ed2184bf5f42c29b6 | 1:06:52 | 13:37.50 | 1:06:09 | 4:48.92 | 0:31.49 | 0:17.59 | 0:02.22 |
| thunderbird | 5cb59cc6910d8f3165528c4a71e29e4de897d242 | 1:07:24 | 11:47.08 | 1:08:16 | 4:56.51 | 0:31.55 | 0:17.29 | 0:02.21 |
| thunderbird | 73dae11852c656adbf5a2b0e44ac4c7e0078cd8d | 1:06:25 | 11:47.57 | 1:17:12 | 4:53.33 | 0:31.27 | 0:16.94 | 0:02.22 |
| thunderbird | 73dae11852c656adbf5a2b0e44ac4c7e0078cd8d | 1:06:37 | 11:35.24 | 1:20:20 | 4:57.05 | 0:31.27 | 0:16.94 | 0:02.21 |
| thunderbird | 14010b4a60072641bab62c761edffc51148ae6e9 | 1:06:40 | 17:00.83 | 1:17:01 | 5:13.92 | 0:31.54 | 0:16.87 | 0:02.21 |
| thunderbird | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:06:19 | 16:02.83 | 1:15:05 | 4:54.64 | 0:31.35 | 0:18.24 | 0:02.19 |
| thunderbird | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:06:36 | 11:50.66 | 1:18:42 | 4:56.03 | 0:31.22 | 0:16.85 | 0:02.19 |
| thunderbird | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:06:21 | 11:23.33 | 1:06:18 | 4:56.64 | 0:31.32 | 0:16.87 | 0:02.23 |
| hostname | commit | time core-image-sato | time virtual/kernel | time core-image-sato rm_work | time core-image-sato -c rootfs | cache1 | cache2 | cache3 |
|---|---|---|---|---|---|---|---|---|
| ubuntu12 | 14010b4a60072641bab62c761edffc51148ae6e9 | 1:22:59 | 12:44.92 | 1:11:48 | 4:58.42 | 0:27.81 | 0:15.54 | 0:02.20 |
| ubuntu12 | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:21:07 | 12:40.53 | 1:12:38 | 4:58.00 | 0:27.65 | 0:15.35 | 0:02.18 |
| ubuntu12 | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:20:10 | 12:28.66 | 1:12:24 | 5:00.78 | 0:27.98 | 0:15.64 | 0:02.20 |
| ubuntu12 | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:20:39 | 12:35.29 | 1:11:41 | 4:55.07 | 0:28.25 | 0:15.37 | 0:02.20 |
| ubuntu12 | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:20:13 | 12:52.21 | 1:13:21 | 5:01.17 | 0:27.83 | 0:15.47 | 0:02.25 |
1.4 Performance Data
Results:
Older results: Media: Results-feb.png
Compare two consecutive runs on System2: Media: Results_feb_2013.png
Historical/obsolete data
Poky/Bitbake Performance:
Performance is an essential point for measuring the quality of a build system.
This page is to track performance trend of poky build. Detailed development work is on https://wiki.yoctoproject.org/wiki/Performance.
Hardware and software configuration:
CPU: 4-core * 2-threads Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
Memory: 4GB
Harddisk: 1TB
OS: Ubuntu 11.10 x86_64
Kernel: 2.6.35-22
local.conf:
CONF_VERSION = "1"
DL_DIR ?= "/sda1/sources/downloads" # Using a local download dir to avoid fetch.
BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE ?= "qemux86"
DISTRO ?= "poky"
BBMASK = ""
EXTRA_IMAGE_FEATURES = "debug-tweaks"
PACKAGE_CLASSES ?= "package_rpm"
USER_CLASSES ?= "image-mklibs image-prelink"
PACKAGE_DEBUG_SPLIT_STYLE = '.debug'
BBINCLUDELOGS = "yes"
ENABLE_BINARY_LOCALE_GENERATION = "1"
NO32LIBS = "1"
Build Performance Trend
| Host | Yocto1.3_M1-build | Yocto1.2-M4-RC5-build | Yocto1.2-M4-RC4-build | Yocto1.2-M4-RC3-build | Yocto1.2-M4-RC2-build | Yocto1.2-M4-RC1-build | Yocto1.2-M3-RC1-build | Yocto1.2-M2-RC1-build | Yocto1.2-M1-RC2-build | Yocto1.1-M4-RC4-build | Yocto1.1-M4-RC3-build | Yocto1.1-M4-RC2-build | Yocto1.1-M2-RC3-build (both rpm and ipk built by default) |
| Ubuntu11.10 x86_64 | real 94m57s user 388m51.720s sys 48m34.660s |
real 111m48.083s user 381m29.410s sys 46m52.080s |
real 111m27.967s user 383m32.690s sys 46m41.830s |
real 110m24.635s user 381m29.740s sys 46m32.430s |
real 112m8.776s user 379m10.670s sys 47m19.270s |
real 130m48.838s user 370m56.000s sys 45m36.060s |
real 133m39.703s user 371m37.450s sys 46m57.510s |
real 124m39s user 373m16 sys 52m17s |
real 104m24s user 374m16s real 50m 37s |
real 119m31s user 355m3s sys 47m42s |
real 127m33s user 346m3s sys 48m5s |
real 124m54s user 342m57s sys 47m33s |
real 109m53s user 356m16s sys 48m16s |
