Performance Test: Difference between revisions

From Yocto Project
Jump to navigationJump to search
Line 66: Line 66:
ubuntu12,a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1:20:39,12:35.29,1:11:41,4:55.07,0:28.25,0:15.37,0:02.20
ubuntu12,a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1:20:39,12:35.29,1:11:41,4:55.07,0:28.25,0:15.37,0:02.20
ubuntu12,a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1:20:13,12:52.21,1:13:21,5:01.17,0:27.83,0:15.47,0:02.25
ubuntu12,a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1:20:13,12:52.21,1:13:21,5:01.17,0:27.83,0:15.47,0:02.25
ubuntu12,master:7bf5c38e0f8bed9295f46773ade5336ec41044f6,1:21:38,13:18.64,1:12:37,4:54.43,0:28.14,0:15.19,0:02.17
ubuntu12,master:7bf5c38e0f8bed9295f46773ade5336ec41044f6,1:21:17,12:58.65,1:12:12,4:57.34,0:27.63,0:15.36,0:02.19
ubuntu12,master:7bf5c38e0f8bed9295f46773ade5336ec41044f6,1:21:46,13:09.45,1:13:13,4:57.01,0:27.92,0:15.96,0:02.18
</tab>
</tab>



Revision as of 07:26, 10 June 2013

Build Perfomance Data

Script used for running the builds: (dylan/master branch): scripts/contrib/build-perf-test.sh

Test systems:

System1: SandyBridge i7-2600 CPU @ 3.40GHz HT-enabled (4 cores/8 threads),  8GB RAM,  1 x 500 GB HDD, Ubuntu 12.04 x64 (with all updates available installed)
        System settings are distro's default:
              - FS mount options are Ubuntu's default: ext4 rw,relatime,user_xattr,barrier=1,data=ordered
              - IO scheduler: cfq (default) and CPU governer: ondemand (default)
System2: IvyBridge i7-3770K CPU @ 3.50GHz HT-enabled (4 cores/8 threads), 32GB RAM,  1 x 1 TB HDD, Fedora 18 x64 (with all updates available installed)
         System settings are distro's default:
              - FS mount options are Fedora's default: ext4 rw,seclabel,relatime,data=ordered
              - IO scheduler: cfq (default) and CPU governer: ondemand (default)


local.conf:

BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE = "qemux86"
DISTRO ?= "poky"
PACKAGE_CLASSES ?= "package_rpm"
EXTRA_IMAGE_FEATURES = "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS = "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    ABORT,${TMPDIR},100M,1K \
    ABORT,${DL_DIR},100M,1K \
    ABORT,${SSTATE_DIR},100M,1K" 
CONF_VERSION = "1"
DL_DIR = "/data/yocto/poky/build-perf-test/downloads"


1.5 nightly build time results

See big table in the next section (1.4) for what each time means:

(thunderbird = System2, ubuntu12=System1 from above description)


hostname commit time core-image-sato time virtual/kernel time core-image-sato rm_work time core-image-sato -c rootfs cache1 cache2 cache3
thunderbird 176a46b088135f2891983bb8540bc45389341d90 1:07:09 11:39.62 1:05:35 4:57.37 0:31.22 0:16.63 0:02.00
thunderbird 21089515bd18ba18e86ec8f72ff8141b9fb1dea8 1:06:27 11:26.78 1:05:48 4:53.59 0:31.22 0:17.96 0:02.20
thunderbird e8197722de1f6006ceae0d4266de91c1db7278e3 1:06:00 13:17.22 1:05:39 4:52.55 0:31.10 0:16.81 0:02.19
thunderbird 676fd3f394a0c0576b63c77ed2184bf5f42c29b6 1:07:43 11:42.89 1:06:16 4:51.57 0:31.40 0:16.90 0:02.19
thunderbird 676fd3f394a0c0576b63c77ed2184bf5f42c29b6 1:06:52 13:37.50 1:06:09 4:48.92 0:31.49 0:17.59 0:02.22
thunderbird 5cb59cc6910d8f3165528c4a71e29e4de897d242 1:07:24 11:47.08 1:08:16 4:56.51 0:31.55 0:17.29 0:02.21
thunderbird 73dae11852c656adbf5a2b0e44ac4c7e0078cd8d 1:06:25 11:47.57 1:17:12 4:53.33 0:31.27 0:16.94 0:02.22
thunderbird 73dae11852c656adbf5a2b0e44ac4c7e0078cd8d 1:06:37 11:35.24 1:20:20 4:57.05 0:31.27 0:16.94 0:02.21
thunderbird 14010b4a60072641bab62c761edffc51148ae6e9 1:06:40 17:00.83 1:17:01 5:13.92 0:31.54 0:16.87 0:02.21
thunderbird a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 1:06:19 16:02.83 1:15:05 4:54.64 0:31.35 0:18.24 0:02.19
thunderbird a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 1:06:36 11:50.66 1:18:42 4:56.03 0:31.22 0:16.85 0:02.19
thunderbird a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 1:06:21 11:23.33 1:06:18 4:56.64 0:31.32 0:16.87 0:02.23


hostname commit time core-image-sato time virtual/kernel time core-image-sato rm_work time core-image-sato -c rootfs cache1 cache2 cache3
ubuntu12 14010b4a60072641bab62c761edffc51148ae6e9 1:22:59 12:44.92 1:11:48 4:58.42 0:27.81 0:15.54 0:02.20
ubuntu12 a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 1:21:07 12:40.53 1:12:38 4:58.00 0:27.65 0:15.35 0:02.18
ubuntu12 a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 1:20:10 12:28.66 1:12:24 5:00.78 0:27.98 0:15.64 0:02.20
ubuntu12 a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 1:20:39 12:35.29 1:11:41 4:55.07 0:28.25 0:15.37 0:02.20
ubuntu12 a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 1:20:13 12:52.21 1:13:21 5:01.17 0:27.83 0:15.47 0:02.25
ubuntu12 master:7bf5c38e0f8bed9295f46773ade5336ec41044f6 1:21:38 13:18.64 1:12:37 4:54.43 0:28.14 0:15.19 0:02.17
ubuntu12 master:7bf5c38e0f8bed9295f46773ade5336ec41044f6 1:21:17 12:58.65 1:12:12 4:57.34 0:27.63 0:15.36 0:02.19
ubuntu12 master:7bf5c38e0f8bed9295f46773ade5336ec41044f6 1:21:46 13:09.45 1:13:13 4:57.01 0:27.92 0:15.96 0:02.18

1.4 Performance Data

Results:

Results1.png


Results2.png


Older results: Media: Results-feb.png

Compare two consecutive runs on System2: Media: Results_feb_2013.png


Historical/obsolete data


Poky/Bitbake Performance:
Performance is an essential point for measuring the quality of a build system.
This page is to track performance trend of poky build. Detailed development work is on https://wiki.yoctoproject.org/wiki/Performance.

Hardware and software configuration:
CPU: 4-core * 2-threads Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
Memory: 4GB
Harddisk: 1TB

OS: Ubuntu 11.10 x86_64
Kernel: 2.6.35-22

local.conf:
CONF_VERSION = "1"
DL_DIR ?= "/sda1/sources/downloads" # Using a local download dir to avoid fetch.
BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE ?= "qemux86"
DISTRO ?= "poky"
BBMASK = ""
EXTRA_IMAGE_FEATURES = "debug-tweaks"
PACKAGE_CLASSES ?= "package_rpm"
USER_CLASSES ?= "image-mklibs image-prelink"
PACKAGE_DEBUG_SPLIT_STYLE = '.debug'
BBINCLUDELOGS = "yes"
ENABLE_BINARY_LOCALE_GENERATION = "1"
NO32LIBS = "1"

Build Performance Trend

Host Yocto1.3_M1-build Yocto1.2-M4-RC5-build Yocto1.2-M4-RC4-build Yocto1.2-M4-RC3-build Yocto1.2-M4-RC2-build Yocto1.2-M4-RC1-build Yocto1.2-M3-RC1-build Yocto1.2-M2-RC1-build Yocto1.2-M1-RC2-build Yocto1.1-M4-RC4-build Yocto1.1-M4-RC3-build Yocto1.1-M4-RC2-build Yocto1.1-M2-RC3-build (both rpm and ipk built by default)
Ubuntu11.10 x86_64 real 94m57s
user 388m51.720s
sys 48m34.660s
real 111m48.083s
user 381m29.410s
sys 46m52.080s
real 111m27.967s
user 383m32.690s
sys 46m41.830s
real 110m24.635s
user 381m29.740s
sys 46m32.430s
real 112m8.776s
user 379m10.670s
sys 47m19.270s
real 130m48.838s
user 370m56.000s
sys 45m36.060s
real 133m39.703s
user 371m37.450s
sys 46m57.510s
real 124m39s
user 373m16
sys 52m17s
real 104m24s
user 374m16s
real 50m 37s
real 119m31s
user 355m3s
sys 47m42s
real 127m33s
user 346m3s
sys 48m5s
real 124m54s
user 342m57s
sys 47m33s
real 109m53s
user 356m16s
sys 48m16s