Performance Test

From Yocto Project
Revision as of 17:01, 9 October 2013 by Stefans (talk | contribs)
Jump to navigationJump to search

Build Time Perfomance

Script used for running the builds: (dylan/master branch): scripts/contrib/build-perf-test.sh

Two identical test systems are used for the results below, one with Ubuntu 12.04 (uname -nr: ubuntu12 3.8.0-31-generic) and one with Fedora 19 (uname -nr: fedora19 3.11.3-201.fc19.x86_64) with all updates available installed. Extra packages installed (besides what's by the default in the distro) are the ones in the Quick Start guide.

All system/distro settings are distro's defaults, that includes:

- FS mount options: ext4 rw,relatime,data=ordered

- IO scheduler: cfq and CPU governer: ondemand

HW config is: SandyBridge i7-2600 CPU @ 3.40GHz HT-enabled (4 cores/8 threads), 8GB RAM, 1 x 500 GB HDD


local.conf it's the default one (the script doesn't touch it except for changing DL_DIR if necessary).

It however exports these in the enviroment:

BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE = "qemux86"

In the tables below, unless otherwise noted, sizes are in kilobytes.

Note: This page holds a re-run of milestone/RCs builds for the 1.5 cycle + 1.4.1/1.4.2 and for upcoming builds for 1.6. This page has the old weekly results ran during the 1.5 cycle. The results differ because the test sytems used there are a bit different than those here (so don't cross compare results).

1.4.1 & 1.4.2 Build Time results

hostname branch:commit git describe bitbake core-image-sato bitbake virtual/kernel bitbake core-image-sato (rm_work) bitbake core-image-sato -c rootfs bitbake -p (rm -rf cache/ tmp/cache/) bitbake -p (rm -rf tmp/cache/) bitbake -p size of tmp dir size of tmp dir (rm_work)
ubuntu12 dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f yocto-1.4.1 1:16:41 11:32.41 1:06:51 5:14.78 0:26.75 0:14.81 0:01.96 28213628 4743912
ubuntu12 dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f yocto-1.4.1 1:14:31 11:34.66 1:06:41 5:12.70 0:26.54 0:14.74 0:01.92 28213772 4743844
ubuntu12 dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f yocto-1.4.1 1:12:39 11:35.99 1:06:59 5:16.48 0:26.77 0:14.71 0:01.96 28217808 4743800
ubuntu12 dylan:d734ab491a30078d43dee5440c03acce2d251425 yocto-1.4.2 1:12:58 12:56.22 1:07:45 4:54.36 0:26.78 0:14.73 0:01.95 25349116 4767324
ubuntu12 dylan:d734ab491a30078d43dee5440c03acce2d251425 yocto-1.4.2 1:13:33 12:44.85 1:08:10 5:01.71 0:26.87 0:14.80 0:01.99 25349708 4767348
ubuntu12 dylan:d734ab491a30078d43dee5440c03acce2d251425 yocto-1.4.2 1:13:00 12:59.19 1:08:13 5:07.04 0:26.83 0:14.73 0:01.97 25363408 4767380


hostname branch:commit git describe bitbake core-image-sato bitbake virtual/kernel bitbake core-image-sato (rm_work) bitbake core-image-sato -c rootfs bitbake -p (rm -rf cache/ tmp/cache/) bitbake -p (rm -rf tmp/cache/) bitbake -p size of tmp dir size of tmp dir (rm_work)
fedora19 dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f yocto-1.4.1 1:14:55 10:11.74 1:10:35 4:06.95 0:34.06 0:17.79 0:01.99 28710092 4835464
fedora19 dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f yocto-1.4.1 1:12:34 10:09.63 1:10:30 4:03.92 0:33.53 0:17.84 0:01.92 28713992 4835460
fedora19 dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f yocto-1.4.1 1:17:15 10:11.67 1:11:47 4:45.40 0:33.90 0:17.31 0:01.97 28710884 4835484
fedora19 dylan:d734ab491a30078d43dee5440c03acce2d251425 yocto-1.4.2 1:14:50 10:53.97 1:15:47 4:19.85 0:34.18 0:17.92 0:01.92 25778860 4858892
fedora19 dylan:d734ab491a30078d43dee5440c03acce2d251425 yocto-1.4.2 1:14:58 10:43.16 1:12:00 4:07.15 0:33.93 0:17.88 0:01.98 25776164 4858920
fedora19 dylan:d734ab491a30078d43dee5440c03acce2d251425 yocto-1.4.2 1:16:11 11:23.14 1:12:31 4:08.25 0:33.64 0:18.03 0:01.95 25775976 4858876


1.5 Milestones results

hostname branch:commit git describe bitbake core-image-sato bitbake virtual/kernel bitbake core-image-sato (rm_work) bitbake core-image-sato -c rootfs bitbake -p (rm -rf cache/ tmp/cache/) bitbake -p (rm -rf tmp/cache/) bitbake -p size of tmp dir size of tmp dir (rm_work)
ubuntu12 master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 1.5_M1.final 1:14:16 12:53.53 1:08:28 5:11.74 0:27.39 0:15.28 0:02.18 25391176 4799632
ubuntu12 master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 1.5_M1.final 1:14:20 12:54.69 1:08:27 5:23.61 0:27.24 0:15.19 0:02.15 25390100 4799560
ubuntu12 master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 1.5_M1.final 1:13:14 12:42.64 1:08:49 5:05.26 0:27.49 0:15.16 0:02.14 25390404 4799588
ubuntu12 master:eaa5df34af42b6a37f6506847d0f3ef6ba0d298a 1.5_M1.final-412-geaa5df3 1:14:39 13:29.39 1:11:05 4:47.33 0:28.01 0:15.53 0:02.09 25316072 4827788
ubuntu12 master:eaa5df34af42b6a37f6506847d0f3ef6ba0d298a 1.5_M1.final-412-geaa5df3 1:15:39 13:39.82 1:11:14 5:00.98 0:28.02 0:15.74 0:02.11 25297644 4827824
ubuntu12 master:eaa5df34af42b6a37f6506847d0f3ef6ba0d298a 1.5_M1.final-412-geaa5df3 1:13:38 13:20.40 1:10:29 4:50.66 0:28.21 0:15.63 0:02.08 25296940 4823004
ubuntu12 master:9de0ad47462c13ac4a2d73e12e92be6c1b4e0415 1.5_M3 1:15:31 13:21.54 1:11:40 4:50.34 0:28.58 0:15.74 0:02.11 25440952 4838208
ubuntu12 master:9de0ad47462c13ac4a2d73e12e92be6c1b4e0415 1.5_M3 1:15:34 13:17.44 1:11:25 4:54.69 0:28.74 0:15.71 0:02.09 25421728 4833420
ubuntu12 master:9de0ad47462c13ac4a2d73e12e92be6c1b4e0415 1.5_M3 1:16:25 13:37.05 1:11:54 4:56.99 0:28.63 0:15.85 0:02.12 25378284 4833444
ubuntu12 master:5745e45b18e5099e94b4d5a73bc97dc6d4cdc91f 1.5_M4.rc2 1:17:58 13:44.95 1:12:30 5:03.64 0:28.95 0:15.97 0:02.32 26131176 4961052
ubuntu12 master:5745e45b18e5099e94b4d5a73bc97dc6d4cdc91f 1.5_M4.rc2 1:18:54 14:00.29 1:13:15 4:59.28 0:28.65 0:15.96 0:02.24 26192480 4966232
ubuntu12 master:5745e45b18e5099e94b4d5a73bc97dc6d4cdc91f 1.5_M4.rc2 1:18:56 14:17.80 1:14:11 4:59.10 0:28.81 0:16.12 0:02.25 26131008 4961248
ubuntu12 master:ed3ef0823fde89371a473d20c0127c0bd16d062b 1.5_M4.final 1:18:19 13:41.56 1:11:33 4:53.58 0:28.27 0:15.41 0:01.54 25688600 4898748
ubuntu12 master:ed3ef0823fde89371a473d20c0127c0bd16d062b 1.5_M4.final 1:15:25 13:59.10 1:11:41 4:51.22 0:28.39 0:15.36 0:01.56 25692980 4898576
ubuntu12 master:ed3ef0823fde89371a473d20c0127c0bd16d062b 1.5_M4.final 1:17:12 13:41.63 1:12:06 4:51.32 0:27.97 0:15.38 0:01.58 25688548 4903912
ubuntu12 master:566ca1e4769b4ec7e3f6c6a98bb817db2221cf82 1.5_M5.rc1 1:15:17 13:47.21 1:10:44 4:55.33 0:28.11 0:15.16 0:01.54 25726992 4897340
ubuntu12 master:566ca1e4769b4ec7e3f6c6a98bb817db2221cf82 1.5_M5.rc1 1:15:03 14:06.96 1:11:32 4:50.64 0:27.95 0:15.20 0:01.58 25704416 4902296
ubuntu12 master:566ca1e4769b4ec7e3f6c6a98bb817db2221cf82 1.5_M5.rc1 1:17:16 14:07.67 1:12:23 4:53.21 0:27.78 0:15.19 0:01.58 25686264 4902292
ubuntu12 master:853bc53cd58a621918f0e5ce662dba263d1befb4 1.5_M5.rc4 1:16:38 14:17.30 1:10:05 4:42.63 0:25.57 0:12.87 0:01.50 25770616 4925008
ubuntu12 master:853bc53cd58a621918f0e5ce662dba263d1befb4 1.5_M5.rc4 1:14:39 14:14.73 1:11:21 4:53.20 0:25.47 0:12.80 0:01.48 25770168 4924972
ubuntu12 master:4d8ab87d25ea6e59dc88375813ef88ee0ac813d0 1.5_M5.rc5 1:14:30 13:57.15 1:11:08 4:54.14 0:25.68 0:13.07 0:01.59 25763920 4920848
ubuntu12 master:4d8ab87d25ea6e59dc88375813ef88ee0ac813d0 1.5_M5.rc5 1:15:28 14:06.75 1:11:38 4:51.25 0:25.45 0:13.03 0:01.61 25761664 4921016
ubuntu12 master:4d8ab87d25ea6e59dc88375813ef88ee0ac813d0 1.5_M5.rc5 1:16:24 14:15.42 1:12:12 4:42.56 0:25.46 0:12.97 0:01.59 25780792 4915792
ubuntu12 master:a02280f99f05b1c21de2a78de9222a2f62a0dabc 1.5_M5.rc6 1:17:19 13:59.28 1:11:50 4:48.02 0:25.62 0:12.96 0:01.59 25759796 4919568
ubuntu12 master:a02280f99f05b1c21de2a78de9222a2f62a0dabc 1.5_M5.rc6 1:17:54 13:56.51 1:11:56 4:50.50 0:25.67 0:13.04 0:01.62 25763872 4919684
hostname branch:commit git describe bitbake core-image-sato bitbake virtual/kernel bitbake core-image-sato (rm_work) bitbake core-image-sato -c rootfs bitbake -p (rm -rf cache/ tmp/cache/) bitbake -p (rm -rf tmp/cache/) bitbake -p size of tmp dir size of tmp dir (rm_work)
fedora19 master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 1.5_M1.final 1:19:37 10:38.18 1:12:10 4:03.70 0:34.59 0:18.54 0:02.23 25801440 4887964
fedora19 master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199 1.5_M1.final 1:19:03 11:24.42 1:12:52 4:01.95 0:34.92 0:18.56 0:02.13 25796764 4887836
fedora19 master:eaa5df34af42b6a37f6506847d0f3ef6ba0d298a 1.5_M1.final-412-geaa5df3 1:20:02 11:43.17 1:13:52 3:58.63 0:35.54 0:19.14 0:02.13 25704568 4911492
fedora19 master:eaa5df34af42b6a37f6506847d0f3ef6ba0d298a 1.5_M1.final-412-geaa5df3 1:21:20 12:11.54 1:14:48 3:59.29 0:35.74 0:19.06 0:02.18 25710016 4911228
fedora19 master:9de0ad47462c13ac4a2d73e12e92be6c1b4e0415 1.5_M3 1:22:38 11:18.43 1:18:37 4:17.43 0:36.29 0:19.14 0:02.10 25783316 4921400
fedora19 master:9de0ad47462c13ac4a2d73e12e92be6c1b4e0415 1.5_M3 1:22:51 11:29.19 1:17:53 4:09.21 0:35.81 0:19.49 0:02.13 25782900 4921324
fedora19 master:5745e45b18e5099e94b4d5a73bc97dc6d4cdc91f 1.5_M4.rc2 1:23:20 12:42.28 1:16:41 4:08.14 0:35.09 0:18.98 0:02.29 26535116 5049040
fedora19 master:5745e45b18e5099e94b4d5a73bc97dc6d4cdc91f 1.5_M4.rc2 1:23:58 12:29.62 1:17:23 4:07.62 0:34.85 0:19.07 0:02.33 26540144 5048920
fedora19 master:ed3ef0823fde89371a473d20c0127c0bd16d062b 1.5_M4.final 1:21:45 12:37.91 1:15:53 4:05.91 0:34.19 0:18.32 0:01.60 26096196 4986476
fedora19 master:ed3ef0823fde89371a473d20c0127c0bd16d062b 1.5_M4.final 1:22:51 12:50.33 1:17:02 4:06.61 0:34.13 0:18.61 0:01.62 26093328 4986408
fedora19 master:566ca1e4769b4ec7e3f6c6a98bb817db2221cf82 1.5_M5.rc1 1:22:27 13:14.06 1:16:17 3:57.51 0:34.03 0:18.30 0:01.61 26090072 4984856
fedora19 master:566ca1e4769b4ec7e3f6c6a98bb817db2221cf82 1.5_M5.rc1 1:22:51 13:12.33 1:16:06 3:58.44 0:34.06 0:18.15 0:01.62 26090640 4985060
fedora19 master:853bc53cd58a621918f0e5ce662dba263d1befb4 1.5_M5.rc4 1:20:58 12:53.74 1:14:58 3:53.32 0:31.15 0:15.29 0:01.61 26176264 5008052
fedora19 master:853bc53cd58a621918f0e5ce662dba263d1befb4 1.5_M5.rc4 1:21:45 13:02.75 1:17:10 3:53.24 0:31.33 0:15.23 0:01.60 26177216 5007964
fedora19 master:4d8ab87d25ea6e59dc88375813ef88ee0ac813d0 1.5_M5.rc5 1:23:16 13:40.63 1:16:49 3:47.56 0:31.17 0:15.36 0:01.67 26168976 5003560
fedora19 master:4d8ab87d25ea6e59dc88375813ef88ee0ac813d0 1.5_M5.rc5 1:22:23 12:48.88 1:16:17 3:51.08 0:31.24 0:15.36 0:01.62 26169684 5003712
fedora19 master:a02280f99f05b1c21de2a78de9222a2f62a0dabc 1.5_M5.rc6 1:23:49 12:43.59 1:16:15 3:49.05 0:31.30 0:15.55 0:01.63 26165572 5007660
fedora19 master:a02280f99f05b1c21de2a78de9222a2f62a0dabc 1.5_M5.rc6 1:23:51 12:19.76 1:17:52 4:10.54 0:31.17 0:15.30 0:01.60 26164292 5007292