Performance Test

Build Perfomance Data

Script used for running the builds: (dylan/master branch): scripts/contrib/build-perf-test.sh

Test systems:

System1: SandyBridge i7-2600 CPU @ 3.40GHz HT-enabled (4 cores/8 threads),  8GB RAM,  1 x 500 GB HDD, Ubuntu 12.04 x64 (with all updates available installed)
        System settings are distro's default:
              - FS mount options are Ubuntu's default: ext4 rw,relatime,user_xattr,barrier=1,data=ordered
              - IO scheduler: cfq (default) and CPU governer: ondemand (default)
System2: IvyBridge i7-3770K CPU @ 3.50GHz HT-enabled (4 cores/8 threads), 32GB RAM,  1 x 1 TB HDD, Fedora 18 x64 (with all updates available installed)
         System settings are distro's default:
              - FS mount options are Fedora's default: ext4 rw,seclabel,relatime,data=ordered
              - IO scheduler: cfq (default) and CPU governer: ondemand (default)

local.conf:

BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE = "qemux86"
DISTRO ?= "poky"
PACKAGE_CLASSES ?= "package_rpm"
EXTRA_IMAGE_FEATURES = "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS = "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    ABORT,${TMPDIR},100M,1K \
    ABORT,${DL_DIR},100M,1K \
    ABORT,${SSTATE_DIR},100M,1K" 
CONF_VERSION = "1"
DL_DIR = "/data/yocto/poky/build-perf-test/downloads"

1.5 nightly build time results

See the below table for what each time means:

(thunderbird = System2)

hostname,commit,time core-image-sato,time virtual/kernel,time core-image-sato rm_work, time core-image-sato -c rootfs, cache1, cache2, cache3

thunderbird,176a46b088135f2891983bb8540bc45389341d90,1:07:09,11:39.62,1:05:35,4:57.37,0:31.22,0:16.63,0:02.00

thunderbird,21089515bd18ba18e86ec8f72ff8141b9fb1dea8,1:06:27,11:26.78,1:05:48,4:53.59,0:31.22,0:17.96,0:02.20

thunderbird,e8197722de1f6006ceae0d4266de91c1db7278e3,1:06:00,13:17.22,1:05:39,4:52.55,0:31.10,0:16.81,0:02.19

thunderbird,676fd3f394a0c0576b63c77ed2184bf5f42c29b6,1:07:43,11:42.89,1:06:16,4:51.57,0:31.40,0:16.90,0:02.19

thunderbird,676fd3f394a0c0576b63c77ed2184bf5f42c29b6,1:07:06,11:43.42,1:06:55,4:52.46,0:31.48,0:17.64,0:02.22

thunderbird,676fd3f394a0c0576b63c77ed2184bf5f42c29b6,1:06:51,13:57.29,1:06:08,4:53.31,0:31.52,0:16.92,0:02.21

thunderbird,676fd3f394a0c0576b63c77ed2184bf5f42c29b6,1:06:58,13:58.48,1:05:59,4:47.42,0:31.33,0:16.99,0:02.19

thunderbird,676fd3f394a0c0576b63c77ed2184bf5f42c29b6,1:06:52,13:37.50,1:06:09,4:48.92,0:31.49,0:17.59,0:02.22

thunderbird,5cb59cc6910d8f3165528c4a71e29e4de897d242,1:07:24,11:47.08,1:08:16,4:56.51,0:31.55,0:17.29,0:02.21

1.4 Performance Data

Results:

Older results: Media: Results-feb.png

Compare two consecutive runs on System2: Media: Results_feb_2013.png

Historical/obsolete data

Poky/Bitbake Performance:
Performance is an essential point for measuring the quality of a build system.
This page is to track performance trend of poky build. Detailed development work is on https://wiki.yoctoproject.org/wiki/Performance.

Hardware and software configuration:
CPU: 4-core * 2-threads Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
Memory: 4GB
Harddisk: 1TB

OS: Ubuntu 11.10 x86_64
Kernel: 2.6.35-22

local.conf:
CONF_VERSION = "1"
DL_DIR ?= "/sda1/sources/downloads" # Using a local download dir to avoid fetch.
BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE ?= "qemux86"
DISTRO ?= "poky"
BBMASK = ""
EXTRA_IMAGE_FEATURES = "debug-tweaks"
PACKAGE_CLASSES ?= "package_rpm"
USER_CLASSES ?= "image-mklibs image-prelink"
PACKAGE_DEBUG_SPLIT_STYLE = '.debug'
BBINCLUDELOGS = "yes"
ENABLE_BINARY_LOCALE_GENERATION = "1"
NO32LIBS = "1"

Build Performance Trend

Host

Yocto1.3_M1-build

Yocto1.2-M4-RC5-build

Yocto1.2-M4-RC4-build

Yocto1.2-M4-RC3-build

Yocto1.2-M4-RC2-build

Yocto1.2-M4-RC1-build

Yocto1.2-M3-RC1-build

Yocto1.2-M2-RC1-build

Yocto1.2-M1-RC2-build

Yocto1.1-M4-RC4-build

Yocto1.1-M4-RC3-build

Yocto1.1-M4-RC2-build

Yocto1.1-M2-RC3-build (both rpm and ipk built by default)

Ubuntu11.10 x86_64

real 94m57s user 388m51.720s sys 48m34.660s

real 111m48.083s user 381m29.410s sys 46m52.080s

real 111m27.967s user 383m32.690s sys 46m41.830s

real 110m24.635s user 381m29.740s sys 46m32.430s

real 112m8.776s user 379m10.670s sys 47m19.270s

real 130m48.838s user 370m56.000s sys 45m36.060s

real 133m39.703s user 371m37.450s sys 46m57.510s

real 124m39s user 373m16 sys 52m17s

real 104m24s user 374m16s real 50m 37s

real 119m31s user 355m3s sys 47m42s

real 127m33s user 346m3s sys 48m5s

real 124m54s user 342m57s sys 47m33s

real 109m53s user 356m16s sys 48m16s