Performance Test

Build Perfomance Data

Script used for running the builds: (dylan/master branch): scripts/contrib/build-perf-test.sh

Test systems:

System1: SandyBridge i7-2600 CPU @ 3.40GHz HT-enabled (4 cores/8 threads),  8GB RAM,  1 x 500 GB HDD, Ubuntu 12.04 x64 (with all updates available installed)
        System settings are distro's default:
              - FS mount options are Ubuntu's default: ext4 rw,relatime,user_xattr,barrier=1,data=ordered
              - IO scheduler: cfq (default) and CPU governer: ondemand (default)
System2: IvyBridge i7-3770K CPU @ 3.50GHz HT-enabled (4 cores/8 threads), 32GB RAM,  1 x 1 TB HDD, Fedora 18 x64 (with all updates available installed)
         System settings are distro's default:
              - FS mount options are Fedora's default: ext4 rw,seclabel,relatime,data=ordered
              - IO scheduler: cfq (default) and CPU governer: ondemand (default)

local.conf:

BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE = "qemux86"
DISTRO ?= "poky"
PACKAGE_CLASSES ?= "package_rpm"
EXTRA_IMAGE_FEATURES = "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS = "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    ABORT,${TMPDIR},100M,1K \
    ABORT,${DL_DIR},100M,1K \
    ABORT,${SSTATE_DIR},100M,1K" 
CONF_VERSION = "1"
DL_DIR = "/data/yocto/poky/build-perf-test/downloads"

1.5 nightly build time results

See big table in the next section (1.4) for what each time means:

(thunderbird = System2, ubuntu12=System1 from above description)

hostname	commit	time core-image-sato	time virtual/kernel	time core-image-sato rm_work	time core-image-sato -c rootfs	cache1	cache2	cache3
thunderbird	176a46b088135f2891983bb8540bc45389341d90	1:07:09	11:39.62	1:05:35	4:57.37	0:31.22	0:16.63	0:02.00
thunderbird	21089515bd18ba18e86ec8f72ff8141b9fb1dea8	1:06:27	11:26.78	1:05:48	4:53.59	0:31.22	0:17.96	0:02.20
thunderbird	e8197722de1f6006ceae0d4266de91c1db7278e3	1:06:00	13:17.22	1:05:39	4:52.55	0:31.10	0:16.81	0:02.19
thunderbird	676fd3f394a0c0576b63c77ed2184bf5f42c29b6	1:07:43	11:42.89	1:06:16	4:51.57	0:31.40	0:16.90	0:02.19
thunderbird	676fd3f394a0c0576b63c77ed2184bf5f42c29b6	1:06:52	13:37.50	1:06:09	4:48.92	0:31.49	0:17.59	0:02.22
thunderbird	5cb59cc6910d8f3165528c4a71e29e4de897d242	1:07:24	11:47.08	1:08:16	4:56.51	0:31.55	0:17.29	0:02.21
thunderbird	73dae11852c656adbf5a2b0e44ac4c7e0078cd8d	1:06:25	11:47.57	1:17:12	4:53.33	0:31.27	0:16.94	0:02.22
thunderbird	73dae11852c656adbf5a2b0e44ac4c7e0078cd8d	1:06:37	11:35.24	1:20:20	4:57.05	0:31.27	0:16.94	0:02.21
thunderbird	14010b4a60072641bab62c761edffc51148ae6e9	1:06:40	17:00.83	1:17:01	5:13.92	0:31.54	0:16.87	0:02.21
thunderbird	a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199	1:06:19	16:02.83	1:15:05	4:54.64	0:31.35	0:18.24	0:02.19
thunderbird	a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199	1:06:36	11:50.66	1:18:42	4:56.03	0:31.22	0:16.85	0:02.19
thunderbird	a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199	1:06:21	11:23.33	1:06:18	4:56.64	0:31.32	0:16.87	0:02.23

hostname	commit	time core-image-sato	time virtual/kernel	time core-image-sato rm_work	time core-image-sato -c rootfs	cache1	cache2	cache3
ubuntu12	14010b4a60072641bab62c761edffc51148ae6e9	1:22:59	12:44.92	1:11:48	4:58.42	0:27.81	0:15.54	0:02.20
ubuntu12	a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199	1:21:07	12:40.53	1:12:38	4:58.00	0:27.65	0:15.35	0:02.18
ubuntu12	a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199	1:20:10	12:28.66	1:12:24	5:00.78	0:27.98	0:15.64	0:02.20
ubuntu12	a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199	1:20:39	12:35.29	1:11:41	4:55.07	0:28.25	0:15.37	0:02.20
ubuntu12	a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199	1:20:13	12:52.21	1:13:21	5:01.17	0:27.83	0:15.47	0:02.25
ubuntu12	master:7bf5c38e0f8bed9295f46773ade5336ec41044f6	1:21:38	13:18.64	1:12:37	4:54.43	0:28.14	0:15.19	0:02.17
ubuntu12	master:7bf5c38e0f8bed9295f46773ade5336ec41044f6	1:21:17	12:58.65	1:12:12	4:57.34	0:27.63	0:15.36	0:02.19
ubuntu12	master:7bf5c38e0f8bed9295f46773ade5336ec41044f6	1:21:46	13:09.45	1:13:13	4:57.01	0:27.92	0:15.96	0:02.18

1.4 Performance Data

Results:

Older results: Media: Results-feb.png

Compare two consecutive runs on System2: Media: Results_feb_2013.png

Historical/obsolete data

Poky/Bitbake Performance:
Performance is an essential point for measuring the quality of a build system.
This page is to track performance trend of poky build. Detailed development work is on https://wiki.yoctoproject.org/wiki/Performance.

Hardware and software configuration:
CPU: 4-core * 2-threads Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
Memory: 4GB
Harddisk: 1TB

OS: Ubuntu 11.10 x86_64
Kernel: 2.6.35-22

local.conf:
CONF_VERSION = "1"
DL_DIR ?= "/sda1/sources/downloads" # Using a local download dir to avoid fetch.
BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE ?= "qemux86"
DISTRO ?= "poky"
BBMASK = ""
EXTRA_IMAGE_FEATURES = "debug-tweaks"
PACKAGE_CLASSES ?= "package_rpm"
USER_CLASSES ?= "image-mklibs image-prelink"
PACKAGE_DEBUG_SPLIT_STYLE = '.debug'
BBINCLUDELOGS = "yes"
ENABLE_BINARY_LOCALE_GENERATION = "1"
NO32LIBS = "1"

Build Performance Trend

Host

Yocto1.3_M1-build

Yocto1.2-M4-RC5-build

Yocto1.2-M4-RC4-build

Yocto1.2-M4-RC3-build

Yocto1.2-M4-RC2-build

Yocto1.2-M4-RC1-build

Yocto1.2-M3-RC1-build

Yocto1.2-M2-RC1-build

Yocto1.2-M1-RC2-build

Yocto1.1-M4-RC4-build

Yocto1.1-M4-RC3-build

Yocto1.1-M4-RC2-build

Yocto1.1-M2-RC3-build (both rpm and ipk built by default)

Ubuntu11.10 x86_64

real 94m57s user 388m51.720s sys 48m34.660s

real 111m48.083s user 381m29.410s sys 46m52.080s

real 111m27.967s user 383m32.690s sys 46m41.830s

real 110m24.635s user 381m29.740s sys 46m32.430s

real 112m8.776s user 379m10.670s sys 47m19.270s

real 130m48.838s user 370m56.000s sys 45m36.060s

real 133m39.703s user 371m37.450s sys 46m57.510s

real 124m39s user 373m16 sys 52m17s

real 104m24s user 374m16s real 50m 37s

real 119m31s user 355m3s sys 47m42s

real 127m33s user 346m3s sys 48m5s

real 124m54s user 342m57s sys 47m33s

real 109m53s user 356m16s sys 48m16s