Performance Test: Difference between revisions
From Yocto Project
Jump to navigationJump to search
| Line 37: | Line 37: | ||
== 1.5 nightly build time results == | == 1.5 nightly build time results == | ||
(System1 = ubuntu12, System2=thunderbird from above description) | |||
<tab class=wikitable sep=comma head=top> | |||
hostname,commit,time core-image-sato,time virtual/kernel,time core-image-sato rm_work, time core-image-sato -c rootfs, cache1, cache2, cache3 | |||
ubuntu12,14010b4a60072641bab62c761edffc51148ae6e9,1:22:59,12:44.92,1:11:48,4:58.42,0:27.81,0:15.54,0:02.20 | |||
ubuntu12,a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1:21:07,12:40.53,1:12:38,4:58.00,0:27.65,0:15.35,0:02.18 | |||
ubuntu12,a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1:20:10,12:28.66,1:12:24,5:00.78,0:27.98,0:15.64,0:02.20 | |||
ubuntu12,a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1:20:39,12:35.29,1:11:41,4:55.07,0:28.25,0:15.37,0:02.20 | |||
ubuntu12,a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1:20:13,12:52.21,1:13:21,5:01.17,0:27.83,0:15.47,0:02.25 | |||
ubuntu12,master:7bf5c38e0f8bed9295f46773ade5336ec41044f6,1:21:38,13:18.64,1:12:37,4:54.43,0:28.14,0:15.19,0:02.17 | |||
ubuntu12,master:7bf5c38e0f8bed9295f46773ade5336ec41044f6,1:21:17,12:58.65,1:12:12,4:57.34,0:27.63,0:15.36,0:02.19 | |||
ubuntu12,master:7bf5c38e0f8bed9295f46773ade5336ec41044f6,1:21:46,13:09.45,1:13:13,4:57.01,0:27.92,0:15.96,0:02.18 | |||
ubuntu12,master:7bf5c38e0f8bed9295f46773ade5336ec41044f6,1:22:06,13:06.07,1:12:44,4:54.78,0:27.80,0:15.49,0:02.20 | |||
ubuntu12,master:5128f0a8465f574f520282499410446a156fc54d,1:24:33,13:31.94,1:13:06,4:51.96,0:28.59,0:15.22,0:02.19 | |||
ubuntu12,master:96b144a25598090b1e58eb5e33f216b5bfef7595,1:23:11,12:58.86,1:14:20,4:57.85,0:28.32,0:15.29,0:02.21 | |||
</tab> | |||
<tab class=wikitable sep=comma head=top> | <tab class=wikitable sep=comma head=top> | ||
| Line 59: | Line 72: | ||
thunderbird,master:96b144a25598090b1e58eb5e33f216b5bfef7595,1:06:48,11:37.51,1:07:05,6:46.44,0:32.94,0:18.59,0:02.23 | thunderbird,master:96b144a25598090b1e58eb5e33f216b5bfef7595,1:06:48,11:37.51,1:07:05,6:46.44,0:32.94,0:18.59,0:02.23 | ||
thunderbird,master:96b144a25598090b1e58eb5e33f216b5bfef7595,1:06:49,11:29.47,1:06:50,4:55.59,0:31.55,0:17.02,0:02.22 | thunderbird,master:96b144a25598090b1e58eb5e33f216b5bfef7595,1:06:49,11:29.47,1:06:50,4:55.59,0:31.55,0:17.02,0:02.22 | ||
</tab> | </tab> | ||
Revision as of 08:17, 13 June 2013
Build Perfomance Data
Script used for running the builds: (dylan/master branch): scripts/contrib/build-perf-test.sh
Test systems:
System1: SandyBridge i7-2600 CPU @ 3.40GHz HT-enabled (4 cores/8 threads), 8GB RAM, 1 x 500 GB HDD, Ubuntu 12.04 x64 (with all updates available installed)
System settings are distro's default:
- FS mount options are Ubuntu's default: ext4 rw,relatime,user_xattr,barrier=1,data=ordered
- IO scheduler: cfq (default) and CPU governer: ondemand (default)
System2: IvyBridge i7-3770K CPU @ 3.50GHz HT-enabled (4 cores/8 threads), 32GB RAM, 1 x 1 TB HDD, Fedora 18 x64 (with all updates available installed)
System settings are distro's default:
- FS mount options are Fedora's default: ext4 rw,seclabel,relatime,data=ordered
- IO scheduler: cfq (default) and CPU governer: ondemand (default)
local.conf:
BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE = "qemux86"
DISTRO ?= "poky"
PACKAGE_CLASSES ?= "package_rpm"
EXTRA_IMAGE_FEATURES = "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS = "\
STOPTASKS,${TMPDIR},1G,100K \
STOPTASKS,${DL_DIR},1G,100K \
STOPTASKS,${SSTATE_DIR},1G,100K \
ABORT,${TMPDIR},100M,1K \
ABORT,${DL_DIR},100M,1K \
ABORT,${SSTATE_DIR},100M,1K"
CONF_VERSION = "1"
DL_DIR = "/data/yocto/poky/build-perf-test/downloads"
1.5 nightly build time results
(System1 = ubuntu12, System2=thunderbird from above description)
| hostname | commit | time core-image-sato | time virtual/kernel | time core-image-sato rm_work | time core-image-sato -c rootfs | cache1 | cache2 | cache3 |
|---|---|---|---|---|---|---|---|---|
| ubuntu12 | 14010b4a60072641bab62c761edffc51148ae6e9 | 1:22:59 | 12:44.92 | 1:11:48 | 4:58.42 | 0:27.81 | 0:15.54 | 0:02.20 |
| ubuntu12 | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:21:07 | 12:40.53 | 1:12:38 | 4:58.00 | 0:27.65 | 0:15.35 | 0:02.18 |
| ubuntu12 | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:20:10 | 12:28.66 | 1:12:24 | 5:00.78 | 0:27.98 | 0:15.64 | 0:02.20 |
| ubuntu12 | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:20:39 | 12:35.29 | 1:11:41 | 4:55.07 | 0:28.25 | 0:15.37 | 0:02.20 |
| ubuntu12 | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:20:13 | 12:52.21 | 1:13:21 | 5:01.17 | 0:27.83 | 0:15.47 | 0:02.25 |
| ubuntu12 | master:7bf5c38e0f8bed9295f46773ade5336ec41044f6 | 1:21:38 | 13:18.64 | 1:12:37 | 4:54.43 | 0:28.14 | 0:15.19 | 0:02.17 |
| ubuntu12 | master:7bf5c38e0f8bed9295f46773ade5336ec41044f6 | 1:21:17 | 12:58.65 | 1:12:12 | 4:57.34 | 0:27.63 | 0:15.36 | 0:02.19 |
| ubuntu12 | master:7bf5c38e0f8bed9295f46773ade5336ec41044f6 | 1:21:46 | 13:09.45 | 1:13:13 | 4:57.01 | 0:27.92 | 0:15.96 | 0:02.18 |
| ubuntu12 | master:7bf5c38e0f8bed9295f46773ade5336ec41044f6 | 1:22:06 | 13:06.07 | 1:12:44 | 4:54.78 | 0:27.80 | 0:15.49 | 0:02.20 |
| ubuntu12 | master:5128f0a8465f574f520282499410446a156fc54d | 1:24:33 | 13:31.94 | 1:13:06 | 4:51.96 | 0:28.59 | 0:15.22 | 0:02.19 |
| ubuntu12 | master:96b144a25598090b1e58eb5e33f216b5bfef7595 | 1:23:11 | 12:58.86 | 1:14:20 | 4:57.85 | 0:28.32 | 0:15.29 | 0:02.21 |
| hostname | commit | time core-image-sato | time virtual/kernel | time core-image-sato rm_work | time core-image-sato -c rootfs | cache1 | cache2 | cache3 |
|---|---|---|---|---|---|---|---|---|
| thunderbird | 176a46b088135f2891983bb8540bc45389341d90 | 1:07:09 | 11:39.62 | 1:05:35 | 4:57.37 | 0:31.22 | 0:16.63 | 0:02.00 |
| thunderbird | 21089515bd18ba18e86ec8f72ff8141b9fb1dea8 | 1:06:27 | 11:26.78 | 1:05:48 | 4:53.59 | 0:31.22 | 0:17.96 | 0:02.20 |
| thunderbird | e8197722de1f6006ceae0d4266de91c1db7278e3 | 1:06:00 | 13:17.22 | 1:05:39 | 4:52.55 | 0:31.10 | 0:16.81 | 0:02.19 |
| thunderbird | 676fd3f394a0c0576b63c77ed2184bf5f42c29b6 | 1:07:43 | 11:42.89 | 1:06:16 | 4:51.57 | 0:31.40 | 0:16.90 | 0:02.19 |
| thunderbird | 676fd3f394a0c0576b63c77ed2184bf5f42c29b6 | 1:06:52 | 13:37.50 | 1:06:09 | 4:48.92 | 0:31.49 | 0:17.59 | 0:02.22 |
| thunderbird | 5cb59cc6910d8f3165528c4a71e29e4de897d242 | 1:07:24 | 11:47.08 | 1:08:16 | 4:56.51 | 0:31.55 | 0:17.29 | 0:02.21 |
| thunderbird | 73dae11852c656adbf5a2b0e44ac4c7e0078cd8d | 1:06:25 | 11:47.57 | 1:17:12 | 4:53.33 | 0:31.27 | 0:16.94 | 0:02.22 |
| thunderbird | 73dae11852c656adbf5a2b0e44ac4c7e0078cd8d | 1:06:37 | 11:35.24 | 1:20:20 | 4:57.05 | 0:31.27 | 0:16.94 | 0:02.21 |
| thunderbird | 14010b4a60072641bab62c761edffc51148ae6e9 | 1:06:40 | 17:00.83 | 1:17:01 | 5:13.92 | 0:31.54 | 0:16.87 | 0:02.21 |
| thunderbird | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:06:19 | 16:02.83 | 1:15:05 | 4:54.64 | 0:31.35 | 0:18.24 | 0:02.19 |
| thunderbird | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:06:36 | 11:50.66 | 1:18:42 | 4:56.03 | 0:31.22 | 0:16.85 | 0:02.19 |
| thunderbird | a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 | 1:06:21 | 11:23.33 | 1:06:18 | 4:56.64 | 0:31.32 | 0:16.87 | 0:02.23 |
| thunderbird | master:7bf5c38e0f8bed9295f46773ade5336ec41044f6 | 1:06:56 | 11:57.49 | 1:06:48 | 4:56.85 | 0:31.47 | 0:16.92 | 0:02.23 |
| thunderbird | master:96b144a25598090b1e58eb5e33f216b5bfef7595 | 1:06:48 | 11:37.51 | 1:07:05 | 6:46.44 | 0:32.94 | 0:18.59 | 0:02.23 |
| thunderbird | master:96b144a25598090b1e58eb5e33f216b5bfef7595 | 1:06:49 | 11:29.47 | 1:06:50 | 4:55.59 | 0:31.55 | 0:17.02 | 0:02.22 |
1.4 Performance Data
Results:
Older results: Media: Results-feb.png
Compare two consecutive runs on System2: Media: Results_feb_2013.png
Historical/obsolete data
Poky/Bitbake Performance:
Performance is an essential point for measuring the quality of a build system.
This page is to track performance trend of poky build. Detailed development work is on https://wiki.yoctoproject.org/wiki/Performance.
Hardware and software configuration:
CPU: 4-core * 2-threads Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
Memory: 4GB
Harddisk: 1TB
OS: Ubuntu 11.10 x86_64
Kernel: 2.6.35-22
local.conf:
CONF_VERSION = "1"
DL_DIR ?= "/sda1/sources/downloads" # Using a local download dir to avoid fetch.
BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE ?= "qemux86"
DISTRO ?= "poky"
BBMASK = ""
EXTRA_IMAGE_FEATURES = "debug-tweaks"
PACKAGE_CLASSES ?= "package_rpm"
USER_CLASSES ?= "image-mklibs image-prelink"
PACKAGE_DEBUG_SPLIT_STYLE = '.debug'
BBINCLUDELOGS = "yes"
ENABLE_BINARY_LOCALE_GENERATION = "1"
NO32LIBS = "1"
Build Performance Trend
| Host | Yocto1.3_M1-build | Yocto1.2-M4-RC5-build | Yocto1.2-M4-RC4-build | Yocto1.2-M4-RC3-build | Yocto1.2-M4-RC2-build | Yocto1.2-M4-RC1-build | Yocto1.2-M3-RC1-build | Yocto1.2-M2-RC1-build | Yocto1.2-M1-RC2-build | Yocto1.1-M4-RC4-build | Yocto1.1-M4-RC3-build | Yocto1.1-M4-RC2-build | Yocto1.1-M2-RC3-build (both rpm and ipk built by default) |
| Ubuntu11.10 x86_64 | real 94m57s user 388m51.720s sys 48m34.660s |
real 111m48.083s user 381m29.410s sys 46m52.080s |
real 111m27.967s user 383m32.690s sys 46m41.830s |
real 110m24.635s user 381m29.740s sys 46m32.430s |
real 112m8.776s user 379m10.670s sys 47m19.270s |
real 130m48.838s user 370m56.000s sys 45m36.060s |
real 133m39.703s user 371m37.450s sys 46m57.510s |
real 124m39s user 373m16 sys 52m17s |
real 104m24s user 374m16s real 50m 37s |
real 119m31s user 355m3s sys 47m42s |
real 127m33s user 346m3s sys 48m5s |
real 124m54s user 342m57s sys 47m33s |
real 109m53s user 356m16s sys 48m16s |
