Performance Test
From Yocto Project
Jump to navigationJump to search
Build Perfomance Data
Script used for running the builds: (dylan/master branch): scripts/contrib/build-perf-test.sh
Test systems:
System1: SandyBridge i7-2600 CPU @ 3.40GHz HT-enabled (4 cores/8 threads), 8GB RAM, 1 x 500 GB HDD, Ubuntu 12.04 x64 (with all updates available installed)
System settings are distro's default:
- FS mount options are Ubuntu's default: ext4 rw,relatime,user_xattr,barrier=1,data=ordered
- IO scheduler: cfq (default) and CPU governer: ondemand (default)
System2: IvyBridge i7-3770K CPU @ 3.50GHz HT-enabled (4 cores/8 threads), 32GB RAM, 1 x 1 TB HDD, Fedora 18 x64 (with all updates available installed)
System settings are distro's default:
- FS mount options are Fedora's default: ext4 rw,seclabel,relatime,data=ordered
- IO scheduler: cfq (default) and CPU governer: ondemand (default)
local.conf:
BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE = "qemux86"
DISTRO ?= "poky"
PACKAGE_CLASSES ?= "package_rpm"
EXTRA_IMAGE_FEATURES = "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS = "\
STOPTASKS,${TMPDIR},1G,100K \
STOPTASKS,${DL_DIR},1G,100K \
STOPTASKS,${SSTATE_DIR},1G,100K \
ABORT,${TMPDIR},100M,1K \
ABORT,${DL_DIR},100M,1K \
ABORT,${SSTATE_DIR},100M,1K"
CONF_VERSION = "1"
DL_DIR = "/data/yocto/poky/build-perf-test/downloads"
1.5 nightly build time results
(System1 = ubuntu12, System2=thunderbird from above description)
| hostname | branch:commit | git describe | bitbake core-image-sato | bitbake virtual/kernel | bitbake core-image-sato (rm_work) | bitbake core-image-sato -c rootfs | bitbake -p (rm -rf cache/ tmp/cache/) | bitbake -p (rm -rf tmp/cache/) | bitbake -p | size of tmp dir | size of tmp dir (rm_work) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| ubuntu12 | (nobranch):789b2b7e0cca404ff8ba8ed5ba823d215e6ba5b4 | dylan-9.0.0 | 1:20:04 | 11:37.44 | 1:12:26 | 5:11.26 | 0:26.21 | 0:14.90 | 0:01.98 | 27G | 4.5G |
| ubuntu12 | master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1.5_M1.rc1 | 1:21:07 | 12:40.53 | 1:12:38 | 4:58.00 | 0:27.65 | 0:15.35 | 0:02.18 | 25G | 4.6G |
| ubuntu12 | master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1.5_M1.rc1 | 1:20:10 | 12:28.66 | 1:12:24 | 5:00.78 | 0:27.98 | 0:15.64 | 0:02.20 | 25G | 4.6G |
| ubuntu12 | master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1.5_M1.rc1 | 1:20:39 | 12:35.29 | 1:11:41 | 4:55.07 | 0:28.25 | 0:15.37 | 0:02.20 | 25G | 4.6G |
| ubuntu12 | master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1.5_M1.rc1 | 1:20:13 | 12:52.21 | 1:13:21 | 5:01.17 | 0:27.83 | 0:15.47 | 0:02.25 | 25G | 4.6G |
| ubuntu12 | master:7bf5c38e0f8bed9295f46773ade5336ec41044f6 | 1.5_M1.rc1-50-g7bf5c38 | 1:21:38 | 13:18.64 | 1:12:37 | 4:54.43 | 0:28.14 | 0:15.19 | 0:02.17 | 25G | 4.6G |
| ubuntu12 | master:7bf5c38e0f8bed9295f46773ade5336ec41044f6 | 1.5_M1.rc1-50-g7bf5c38 | 1:21:17 | 12:58.65 | 1:12:12 | 4:57.34 | 0:27.63 | 0:15.36 | 0:02.19 | 25G | 4.6G |
| ubuntu12 | master:7bf5c38e0f8bed9295f46773ade5336ec41044f6 | 1.5_M1.rc1-50-g7bf5c38 | 1:21:46 | 13:09.45 | 1:13:13 | 4:57.01 | 0:27.92 | 0:15.96 | 0:02.18 | 25G | 4.6G |
| ubuntu12 | master:7bf5c38e0f8bed9295f46773ade5336ec41044f6 | 1.5_M1.rc1-50-g7bf5c38 | 1:22:06 | 13:06.07 | 1:12:44 | 4:54.78 | 0:27.80 | 0:15.49 | 0:02.20 | 25G | 4.6G |
| ubuntu12 | master:5128f0a8465f574f520282499410446a156fc54d | 1.5_M1.rc1-52-g5128f0a | 1:24:33 | 13:31.94 | 1:13:06 | 4:51.96 | 0:28.59 | 0:15.22 | 0:02.19 | 25G | 4.6G |
| ubuntu12 | master:96b144a25598090b1e58eb5e33f216b5bfef7595 | 1.5_M1.rc1-54-g96b144a | 1:23:11 | 12:58.86 | 1:14:20 | 4:57.85 | 0:28.32 | 0:15.29 | 0:02.21 | 25G | 4.6G |
| ubuntu12 | master:69a6023502fb5830e2f103041985cb72e33806dd | 1.5_M1.rc1-86-g69a6023 | 1:21:36 | 13:10.87 | 1:14:05 | 5:00.99 | 0:28.18 | 0:15.69 | 0:02.26 | 25G | 4.7G |
| ubuntu12 | master:35ea19860bf731d68555b033db23ae9649181b6d | 1.5_M1.rc1-104-g35ea198 | 1:26:30 | 14:03.83 | 1:13:52 | 4:55.25 | 0:28.22 | 0:15.59 | 0:02.19 | 25G | 4.7G |
| ubuntu12 | master:b87429fdb45f82ebe4060fea8c4ecd6e4bc1ae52 | 1.5_M1.rc1-105-gb87429f | 1:23:46 | 12:56.53 | 1:14:29 | 4:58.23 | 0:28.41 | 0:15.58 | 0:02.22 | 25G | 4.7G |
| ubuntu12 | master:13bd8ad0214c645f67966c0a150644224522de18 | 1.5_M1.rc1-106-g13bd8ad | 1:21:20 | 13:07.61 | 1:13:53 | 5:00.12 | 0:28.06 | 0:15.48 | 0:02.19 | 25G | 4.7G |
| ubuntu12 | master:04af378874f38d1200bea2fa191beeae94232d6e | 1.5_M1.rc1-144-g04af378 | 1:24:50 | 13:20.98 | 1:15:34 | 5:05.14 | 0:28.03 | 0:15.53 | 0:02.21 | 25G | 4.7G |
| ubuntu12 | master:04af378874f38d1200bea2fa191beeae94232d6e | 1.5_M1.rc1-144-g04af378 | 1:23:58 | 13:23.70 | 1:24:32 | 4:57.00 | 0:27.79 | 0:15.52 | 0:02.24 | 25G | 4.7G |
| ubuntu12 | master:e023620773df82ba1551bec7fb921e4fef01322e | 1.5_M1.rc1-195-ge023620 | 1:21:46 | 13:37.46 | 1:14:50 | 4:46.60 | 0:28.18 | 0:15.76 | 0:02.20 | 25G | 4.7G |
| ubuntu12 | master:e023620773df82ba1551bec7fb921e4fef01322e | 1.5_M1.rc1-195-ge023620 | 1:22:22 | 13:32.35 | 1:16:55 | 4:50.60 | 0:28.15 | 0:15.55 | 0:02.18 | 25G | 4.7G |
| hostname | branch:commit | git describe | bitbake core-image-sato | bitbake virtual/kernel | bitbake core-image-sato (rm_work) | bitbake core-image-sato -c rootfs | bitbake -p (rm -rf cache/ tmp/cache/) | bitbake -p (rm -rf tmp/cache/) | bitbake -p | size of tmp dir | size of tmp dir (rm_work) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| thunderbird | master:176a46b088135f2891983bb8540bc45389341d90 | 1:07:09 | 11:39.62 | 1:05:35 | 4:57.37 | 0:31.22 | 0:16.63 | 0:02.00 | |||
| thunderbird | master:21089515bd18ba18e86ec8f72ff8141b9fb1dea8 | 1:06:27 | 11:26.78 | 1:05:48 | 4:53.59 | 0:31.22 | 0:17.96 | 0:02.20 | |||
| thunderbird | master:e8197722de1f6006ceae0d4266de91c1db7278e3 | 1:06:00 | 13:17.22 | 1:05:39 | 4:52.55 | 0:31.10 | 0:16.81 | 0:02.19 | |||
| thunderbird | master:676fd3f394a0c0576b63c77ed2184bf5f42c29b6 | 1:07:43 | 11:42.89 | 1:06:16 | 4:51.57 | 0:31.40 | 0:16.90 | 0:02.19 | |||
| thunderbird | master:676fd3f394a0c0576b63c77ed2184bf5f42c29b6 | 1:06:52 | 13:37.50 | 1:06:09 | 4:48.92 | 0:31.49 | 0:17.59 | 0:02.22 | |||
| thunderbird | master:5cb59cc6910d8f3165528c4a71e29e4de897d242 | 1:07:24 | 11:47.08 | 1:08:16 | 4:56.51 | 0:31.55 | 0:17.29 | 0:02.21 | |||
| thunderbird | master:73dae11852c656adbf5a2b0e44ac4c7e0078cd8d | 1:06:25 | 11:47.57 | 1:17:12 | 4:53.33 | 0:31.27 | 0:16.94 | 0:02.22 | |||
| thunderbird | master:73dae11852c656adbf5a2b0e44ac4c7e0078cd8d | 1:06:37 | 11:35.24 | 1:20:20 | 4:57.05 | 0:31.27 | 0:16.94 | 0:02.21 | |||
| thunderbird | master:14010b4a60072641bab62c761edffc51148ae6e9 | 1:06:40 | 17:00.83 | 1:17:01 | 5:13.92 | 0:31.54 | 0:16.87 | 0:02.21 | |||
| thunderbird | master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1.5_M1.rc1 | 1:06:19 | 16:02.83 | 1:15:05 | 4:54.64 | 0:31.35 | 0:18.24 | 0:02.19 | ||
| thunderbird | master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1.5_M1.rc1 | 1:06:36 | 11:50.66 | 1:18:42 | 4:56.03 | 0:31.22 | 0:16.85 | 0:02.19 | ||
| thunderbird | master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1.5_M1.rc1 | 1:06:21 | 11:23.33 | 1:06:18 | 4:56.64 | 0:31.32 | 0:16.87 | 0:02.23 | ||
| thunderbird | master:7bf5c38e0f8bed9295f46773ade5336ec41044f6 | 1.5_M1.rc1-54-g96b144a | 1:06:56 | 11:57.49 | 1:06:48 | 4:56.85 | 0:31.47 | 0:16.92 | 0:02.23 | ||
| thunderbird | master:96b144a25598090b1e58eb5e33f216b5bfef7595 | 1.5_M1.rc1-54-g96b144a | 1:06:48 | 11:37.51 | 1:07:05 | 6:46.44 | 0:32.94 | 0:18.59 | 0:02.23 | ||
| thunderbird | master:96b144a25598090b1e58eb5e33f216b5bfef7595 | 1.5_M1.rc1-54-g96b144a | 1:06:49 | 11:29.47 | 1:06:50 | 4:55.59 | 0:31.55 | 0:17.02 | 0:02.22 | ||
| thunderbird | master:35ea19860bf731d68555b033db23ae9649181b6d | 1.5_M1.rc1-104-g35ea198 | 1:06:48 | 12:07.82 | 1:06:42 | 5:21.74 | 0:31.75 | 0:17.24 | 0:02.23 | ||
| thunderbird | master:35ea19860bf731d68555b033db23ae9649181b6d | 1.5_M1.rc1-104-g35ea198 | 1:06:55 | 11:41.60 | 1:07:03 | 4:55.73 | 0:31.80 | 0:18.33 | 0:02.22 | ||
| thunderbird | master:13bd8ad0214c645f67966c0a150644224522de18 | 1.5_M1.rc1-106-g13bd8ad | 1:06:39 | 11:32.22 | 1:06:55 | 7:34.15 | 0:32.31 | 0:19.55 | 0:02.40 | 25G | 4.7G |
| thunderbird | master:13bd8ad0214c645f67966c0a150644224522de18 | 1.5_M1.rc1-106-g13bd8ad | 1:06:42 | 12:07.90 | 1:06:24 | 5:02.10 | 0:31.63 | 0:18.48 | 0:02.23 | 25G | 4.7G |
| thunderbird | master:04af378874f38d1200bea2fa191beeae94232d6e | 1.5_M1.rc1-144-g04af378 | 1:07:01 | 11:49.24 | 1:12:43 | 7:50.18 | 0:32.12 | 0:18.35 | 0:02.39 | 25G | 4.8G |
| thunderbird | master:04af378874f38d1200bea2fa191beeae94232d6e | 1.5_M1.rc1-144-g04af378 | 1:06:57 | 11:41.98 | 1:06:57 | 5:01.90 | 0:31.67 | 0:18.64 | 0:02.23 | 25G | 4.8G |
| thunderbird | master:e023620773df82ba1551bec7fb921e4fef01322e | 1.5_M1.rc1-195-ge023620 | 1:05:43 | 11:59.38 | 1:09:29 | 7:35.79 | 0:32.14 | 0:18.72 | 0:02.39 | 25G | 4.8G |
| thunderbird | master:e023620773df82ba1551bec7fb921e4fef01322e | 1.5_M1.rc1-195-ge023620 | 1:05:28 | 11:28.20 | 1:07:02 | 4:54.82 | 0:31.81 | 0:17.44 | 0:02.22 | 25G | 4.8G |
| thunderbird | master:e023620773df82ba1551bec7fb921e4fef01322e | 1.5_M1.rc1-195-ge023620 | 1:05:48 | 11:30.65 | 1:07:32 | 4:51.39 | 0:31.78 | 0:17.18 | 0:02.18 | 25G | 4.8G |
| thunderbird | master:e023620773df82ba1551bec7fb921e4fef01322e | 1.5_M1.rc1-195-ge023620 | 1:05:34 | 11:59.83 | 1:06:59 | 4:47.96 | 0:31.95 | 0:17.03 | 0:02.17 | 25G | 4.8G |
| thunderbird | master:1dd643b142c69ac9035e29bff11d02201638dc65 | 1.5_M1.rc1-224-g1dd643b | 1:05:20 | 12:13.68 | 1:18:36 | 6:59.89 | 0:31.91 | 0:17.11 | 0:02.20 | 25G | 4.8G |
| thunderbird | master:1dd643b142c69ac9035e29bff11d02201638dc65 | 1.5_M1.rc1-224-g1dd643b | 1:05:24 | 11:55.62 | 1:06:46 | 4:49.88 | 0:31.89 | 0:17.15 | 0:02.20 | 25G | 4.8G |
| thunderbird | master:590010a6525b0e1bc1de73e794764e23404591df | 1.5_M1.final-239-g590010a | 1:04:57 | 11:27.97 | 1:07:19 | 6:06.83 | 0:32.49 | 0:19.05 | 0:02.35 | 25G | 4.8G |
| thunderbird | master:590010a6525b0e1bc1de73e794764e23404591df | 1.5_M1.final-239-g590010a | 1:05:16 | 11:23.81 | 1:06:52 | 4:51.00 | 0:32.60 | 0:18.74 | 0:02.16 | 25G | 4.8G |
| thunderbird | master:e04e6dab0afa8f72bf001b0cc106cd13b7694a91 | 1.5_M1.final-245-ge04e6da | 1:03:51 | 11:51.15 | 1:05:13 | 5:03.39 | 0:31.98 | 0:19.33 | 0:02.30 | 25G | 4.8G |
| thunderbird | master:e04e6dab0afa8f72bf001b0cc106cd13b7694a91 | 1.5_M1.final-245-ge04e6da | 1:04:03 | 11:43.09 | 1:05:44 | 4:47.45 | 0:31.84 | 0:17.89 | 0:02.18 | 25G | 4.8G |
| thunderbird | master:78bcf871c65a87df8b40a0b09654d74434e4d01c | 1.5_M1.final-263-g78bcf87 | 1:04:45 | 11:50.15 | 1:07:27 | 6:43.08 | 0:32.15 | 0:18.81 | 0:02.32 | 25G | 4.8G |
| thunderbird | master:78bcf871c65a87df8b40a0b09654d74434e4d01c | 1.5_M1.final-263-g78bcf87 | 1:04:42 | 11:46.32 | 1:06:29 | 4:48.85 | 0:31.77 | 0:17.15 | 0:02.15 | 25G | 4.8G |
1.4.1 Performance Data
| hostname | branch:commit | git describe | bitbake core-image-sato | bitbake virtual/kernel | bitbake core-image-sato (rm_work) | bitbake core-image-sato -c rootfs | bitbake -p (rm -rf cache/ tmp/cache/) | bitbake -p (rm -rf tmp/cache/) | bitbake -p | size of tmp dir | size of tmp dir (rm_work) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| ubuntu12 | (nobranch):789b2b7e0cca404ff8ba8ed5ba823d215e6ba5b4 | dylan-9.0.0 | 1:20:04 | 11:37.44 | 1:12:26 | 5:11.26 | 0:26.21 | 0:14.90 | 0:01.98 | 27G | 4.5G |
| ubuntu12 | dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f | dylan-9.0.1.rc1 | 1:20:14 | 11:38.69 | 1:12:02 | 5:19.61 | 0:27.51 | 0:14.78 | 0:02.06 | 28G | 4.6G |
| ubuntu12 | dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f | dylan-9.0.1.rc1 | 1:23:01 | 12:14.09 | 1:11:38 | 5:12.68 | 0:27.39 | 0:14.71 | 0:01.98 | 27G | 4.6G |
| ubuntu12 | dylan:e4a0a4c5e419154a34710a1b6c28d4180c6304c3 | dylan-9.0.1.rc1-6-ge4a0a4c | 1:23:17 | 11:39.01 | 1:13:22 | 5:14.69 | 0:26.86 | 0:14.61 | 0:02.00 | 27G | 4.6G |
| ubuntu12 | dylan:e4a0a4c5e419154a34710a1b6c28d4180c6304c3 | dylan-9.0.1.rc1-6-ge4a0a4c | 1:22:23 | 11:34.47 | 1:12:44 | 5:13.63 | 0:26.98 | 0:14.96 | 0:02.00 | 27G | 4.6G |
| hostname | branch:commit | git describe | bitbake core-image-sato | bitbake virtual/kernel | bitbake core-image-sato (rm_work) | bitbake core-image-sato -c rootfs | bitbake -p (rm -rf cache/ tmp/cache/) | bitbake -p (rm -rf tmp/cache/) | bitbake -p | size of tmp dir | size of tmp dir (rm_work) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| thunderbird | dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f | dylan-9.0.1.rc1 | 1:05:52 | 11:04.48 | 1:05:23 | 5:16.83 | 0:30.80 | 0:17.16 | 0:02.01 | 28G | 4.7G |
| thunderbird | dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f | dylan-9.0.1.rc1 | 1:05:53 | 10:27.02 | 1:05:18 | 5:10.17 | 0:30.50 | 0:17.37 | 0:02.05 | 28G | 4.7G |
| thunderbird | dylan:e4a0a4c5e419154a34710a1b6c28d4180c6304c3 | dylan-9.0.1.rc1-6-ge4a0a4c | 1:05:45 | 11:15.34 | 1:05:44 | 5:08.59 | 0:30.48 | 0:16.33 | 0:02.00 | 28G | 4.7G |
| thunderbird | dylan:e4a0a4c5e419154a34710a1b6c28d4180c6304c3 | dylan-9.0.1.rc1-6-ge4a0a4c | 1:05:36 | 11:11.98 | 1:05:33 | 5:10.65 | 0:31.08 | 0:16.34 | 0:02.03 | 28G | 4.7G |
| thunderbird | dylan:e4a0a4c5e419154a34710a1b6c28d4180c6304c3 | dylan-9.0.1.rc1-6-ge4a0a4c | 1:05:45 | 11:17.05 | 1:05:22 | 5:16.18 | 0:30.81 | 0:17.55 | 0:02.01 | 28G | 4.7G |
1.4 Performance Data
Results:
Older results: Media: Results-feb.png
Compare two consecutive runs on System2: Media: Results_feb_2013.png
Historical/obsolete data
Poky/Bitbake Performance:
Performance is an essential point for measuring the quality of a build system.
This page is to track performance trend of poky build. Detailed development work is on https://wiki.yoctoproject.org/wiki/Performance.
Hardware and software configuration:
CPU: 4-core * 2-threads Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
Memory: 4GB
Harddisk: 1TB
OS: Ubuntu 11.10 x86_64
Kernel: 2.6.35-22
local.conf:
CONF_VERSION = "1"
DL_DIR ?= "/sda1/sources/downloads" # Using a local download dir to avoid fetch.
BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE ?= "qemux86"
DISTRO ?= "poky"
BBMASK = ""
EXTRA_IMAGE_FEATURES = "debug-tweaks"
PACKAGE_CLASSES ?= "package_rpm"
USER_CLASSES ?= "image-mklibs image-prelink"
PACKAGE_DEBUG_SPLIT_STYLE = '.debug'
BBINCLUDELOGS = "yes"
ENABLE_BINARY_LOCALE_GENERATION = "1"
NO32LIBS = "1"
Build Performance Trend
| Host | Yocto1.3_M1-build | Yocto1.2-M4-RC5-build | Yocto1.2-M4-RC4-build | Yocto1.2-M4-RC3-build | Yocto1.2-M4-RC2-build | Yocto1.2-M4-RC1-build | Yocto1.2-M3-RC1-build | Yocto1.2-M2-RC1-build | Yocto1.2-M1-RC2-build | Yocto1.1-M4-RC4-build | Yocto1.1-M4-RC3-build | Yocto1.1-M4-RC2-build | Yocto1.1-M2-RC3-build (both rpm and ipk built by default) |
| Ubuntu11.10 x86_64 | real 94m57s user 388m51.720s sys 48m34.660s |
real 111m48.083s user 381m29.410s sys 46m52.080s |
real 111m27.967s user 383m32.690s sys 46m41.830s |
real 110m24.635s user 381m29.740s sys 46m32.430s |
real 112m8.776s user 379m10.670s sys 47m19.270s |
real 130m48.838s user 370m56.000s sys 45m36.060s |
real 133m39.703s user 371m37.450s sys 46m57.510s |
real 124m39s user 373m16 sys 52m17s |
real 104m24s user 374m16s real 50m 37s |
real 119m31s user 355m3s sys 47m42s |
real 127m33s user 346m3s sys 48m5s |
real 124m54s user 342m57s sys 47m33s |
real 109m53s user 356m16s sys 48m16s |
