Performance Test: Difference between revisions

From Yocto Project
Jump to navigationJump to search
 
(14 intermediate revisions by the same user not shown)
Line 8: Line 8:
Two identical test systems are used for the results, with all updates available installed. Extra packages installed (besides what's by the default in the distro) are the ones in the Quick Start guide.
Two identical test systems are used for the results, with all updates available installed. Extra packages installed (besides what's by the default in the distro) are the ones in the Quick Start guide.


Romania Setup 1.6 - 2.1
Configuration of the GDC Setup 2.1 - Current master
 
* Ubuntu 12.04 (uname -nr: ubuntu12 3.8.0-31-generic) and
* Fedora 19 (uname -nr: fedora19 3.11.3-201.fc19.x86_64)
 
GDC Setup 2.1 - Current master
 
* Ubuntu 15.10 (Wily Werewolf)" -->  4.2.0-16-generic
* Ubuntu 15.10 (Wily Werewolf)" -->  4.2.0-16-generic
* Fedora release 23 (Twenty Three) --> 4.7.10-100.fc23.x86_64
* Fedora release 23 (Twenty Three) --> 4.7.10-100.fc23.x86_64
* HW configuration: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz, 16GB RAM,  1 x 600 GB SDD, 1 x 2 TB HDD


All system/distro settings are '''distro's defaults''', that includes:
All system/distro settings are '''distro's defaults''', that includes:
Line 23: Line 18:


- IO scheduler: cfq and CPU governer: ondemand
- IO scheduler: cfq and CPU governer: ondemand
Romania Setup 1.6 - 2.1
*HW config is: SandyBridge i7-2600 CPU @ 3.40GHz (4 cores/8 threads),  8GB RAM,  1 x 500 GB HDD
GDC Setup 2.1 - Current master
*HW config is: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz, 16GB RAM,  1 x 600 GB SDD, 1 x 2 TB HDD


local.conf it's the default one (the script doesn't touch it except for changing DL_DIR if necessary).
local.conf it's the default one (the script doesn't touch it except for changing DL_DIR if necessary).
Line 36: Line 25:
PARALLEL_MAKE = "-j 8"
PARALLEL_MAKE = "-j 8"
MACHINE = "qemux86"</nowiki>
MACHINE = "qemux86"</nowiki>
In the tables below, unless otherwise noted, sizes are in kilobytes.
Note:
*Master builds are run daily on whatever commit there is when the cron kicks in.
*This page holds a re-run of milestone/RCs builds only for the 1.5 (dora) cycle + 1.4.1/1.4.2 (dylan).
*[https://wiki.yoctoproject.org/wiki/Performance_Test_Weekly_1.5 This page] has the old weekly results ran during the 1.5 cycle. The results differ because the test sytems used there are a bit different than those here (so don't cross compare results).


FAQ:
FAQ:
Line 50: Line 32:




----
== Daily Test Results ==
 
Build performance test suite is run multiple times per day. Latest test reports for master branch can found here:
Ubuntu has the default system shell (/bin/sh -> /bin/dash) and Fedora has the default SELinux policy (enabled). (Some quick tests showed that switching to bash or disabling SELinux had an minimal impact on the total build time).
* [https://wiki.yoctoproject.org/charts/build_perf/latest/ypperf-ubuntu16_master_qemux86.html Ubuntu 16.04]
 
* [https://wiki.yoctoproject.org/charts/build_perf/latest/ypperf-ubuntu16-xeon_master_qemux86.html Ubuntu 16.04 (Intel® Xeon® HW)]
# sh -> dash (default)
* [https://wiki.yoctoproject.org/charts/build_perf/latest/ypperf-centos7_master_qemux86.html CentOS 7]
ubuntu12,master:ea92671d9823e3667d6ced7ac2af20f991da404d,poky-10.0.0.final-426-gea92671,1:14:02,12:03.51,1:09:57,4:49.58,0:26.57,0:13.32,0:01.72,25446936,4810304
* [https://wiki.yoctoproject.org/charts/build_perf/latest/ypperf-opensuse422_master_qemux86.html OpenSUSE Leap 42.2] (retired)
ubuntu12,master:ea92671d9823e3667d6ced7ac2af20f991da404d,poky-10.0.0.final-426-gea92671,1:14:10,12:06.96,1:10:22,4:52.66,0:26.31,0:13.35,0:01.66,25425188,4815440
* [https://wiki.yoctoproject.org/charts/build_perf/latest/ypperf-fedora25_master_qemux86.html Fedora 25] (retired)
ubuntu12,master:ea92671d9823e3667d6ced7ac2af20f991da404d,poky-10.0.0.final-426-gea92671,1:14:16,12:09.09,1:10:34,4:47.50,0:26.10,0:13.51,0:01.57,25425852,4810240
* [https://wiki.yoctoproject.org/charts/build_perf/latest/ypperf01_master_qemux86.html Fedora 23] (retired)
# sh -> bash
* [https://wiki.yoctoproject.org/charts/build_perf/latest/ypperf02_master_qemux86.html Ubuntu 15.10] (retired)
ubuntu12,master:ea92671d9823e3667d6ced7ac2af20f991da404d,poky-10.0.0.final-426-gea92671,1:12:56,12:03.28,1:10:25,4:57.96,0:26.19,0:13.36,0:01.59,25426624,4810376
ubuntu12,master:ea92671d9823e3667d6ced7ac2af20f991da404d,poky-10.0.0.final-426-gea92671,1:13:36,12:04.68,1:10:30,5:04.22,0:26.15,0:13.34,0:01.64,25425332,4815348
ubuntu12,master:ea92671d9823e3667d6ced7ac2af20f991da404d,poky-10.0.0.final-426-gea92671,1:14:21,12:08.91,1:10:41,5:01.82,0:26.39,0:13.39,0:01.83,25426632,4815256
# selinux=1 (default)
fedora19,master:ea92671d9823e3667d6ced7ac2af20f991da404d,poky-10.0.0.final-426-gea92671,1:15:22,10:40.98,1:13:21,4:05.22,0:32.10,0:15.63,0:01.63,25831032,4898552
fedora19,master:ea92671d9823e3667d6ced7ac2af20f991da404d,poky-10.0.0.final-426-gea92671,1:16:28,10:40.84,1:14:15,4:04.72,0:31.64,0:15.79,0:01.63,25831752,4898616
fedora19,master:ea92671d9823e3667d6ced7ac2af20f991da404d,poky-10.0.0.final-426-gea92671,1:17:04,10:32.56,1:14:18,4:04.11,0:31.74,0:15.75,0:01.59,25831972,4898692
# selinux=0
fedora19,master:ea92671d9823e3667d6ced7ac2af20f991da404d,poky-10.0.0.final-426-gea92671,1:16:22,10:14.86,1:13:09,3:55.77,0:31.72,0:15.58,0:01.60,25832104,4898672
fedora19,master:ea92671d9823e3667d6ced7ac2af20f991da404d,poky-10.0.0.final-426-gea92671,1:16:45,10:24.95,1:13:25,4:04.07,0:31.64,0:15.64,0:01.60,25831380,4898616
fedora19,master:ea92671d9823e3667d6ced7ac2af20f991da404d,poky-10.0.0.final-426-gea92671,1:17:28,10:16.52,1:12:42,4:02.91,0:31.70,0:15.53,0:01.59,25834384,4898748
 
----


== Milestone Test results ==
== Milestone Test results ==
The results are split in tow different phases as below
Romania Setup from 1.6 to 2.1 releases
* [https://wiki.yoctoproject.org/charts/perf_milestone/performance_test.html RO Charts]
Guadalajara Setup from 2.1 M4 to actual master  
Guadalajara Setup from 2.1 M4 to actual master  


* [https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.html GDC Charts]
* [https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.html GDC Charts]


== 1.5 Results (Milestones + 1.5 release + 1.5.1 + 1.5.2) ==
== Old Test results ==


<tab class=wikitable sep=comma head=top>
Milestone test results from the retired Romania Setup (Yocto 1.6 – 2.1) can be found from here:
hostname,branch:commit,git describe,bitbake core-image-sato,bitbake virtual/kernel,bitbake core-image-sato (rm_work),bitbake core-image-sato -c rootfs,bitbake -p (rm -rf cache/ tmp/cache/), bitbake -p (rm -rf tmp/cache/),bitbake -p,size of tmp dir, size of tmp dir (rm_work)
ubuntu12,master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1.5_M1.final,1:14:16,12:53.53,1:08:28,5:11.74,0:27.39,0:15.28,0:02.18,25391176,4799632
ubuntu12,master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1.5_M1.final,1:14:20,12:54.69,1:08:27,5:23.61,0:27.24,0:15.19,0:02.15,25390100,4799560
ubuntu12,master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1.5_M1.final,1:13:14,12:42.64,1:08:49,5:05.26,0:27.49,0:15.16,0:02.14,25390404,4799588
-
ubuntu12,master:eaa5df34af42b6a37f6506847d0f3ef6ba0d298a,1.5_M1.final-412-geaa5df3,1:14:39,13:29.39,1:11:05,4:47.33,0:28.01,0:15.53,0:02.09,25316072,4827788
ubuntu12,master:eaa5df34af42b6a37f6506847d0f3ef6ba0d298a,1.5_M1.final-412-geaa5df3,1:15:39,13:39.82,1:11:14,5:00.98,0:28.02,0:15.74,0:02.11,25297644,4827824
ubuntu12,master:eaa5df34af42b6a37f6506847d0f3ef6ba0d298a,1.5_M1.final-412-geaa5df3,1:13:38,13:20.40,1:10:29,4:50.66,0:28.21,0:15.63,0:02.08,25296940,4823004
-
ubuntu12,master:9de0ad47462c13ac4a2d73e12e92be6c1b4e0415,1.5_M3,1:15:31,13:21.54,1:11:40,4:50.34,0:28.58,0:15.74,0:02.11,25440952,4838208
ubuntu12,master:9de0ad47462c13ac4a2d73e12e92be6c1b4e0415,1.5_M3,1:15:34,13:17.44,1:11:25,4:54.69,0:28.74,0:15.71,0:02.09,25421728,4833420
ubuntu12,master:9de0ad47462c13ac4a2d73e12e92be6c1b4e0415,1.5_M3,1:16:25,13:37.05,1:11:54,4:56.99,0:28.63,0:15.85,0:02.12,25378284,4833444
-
ubuntu12,master:5745e45b18e5099e94b4d5a73bc97dc6d4cdc91f,1.5_M4.rc2,1:17:58,13:44.95,1:12:30,5:03.64,0:28.95,0:15.97,0:02.32,26131176,4961052
ubuntu12,master:5745e45b18e5099e94b4d5a73bc97dc6d4cdc91f,1.5_M4.rc2,1:18:54,14:00.29,1:13:15,4:59.28,0:28.65,0:15.96,0:02.24,26192480,4966232
ubuntu12,master:5745e45b18e5099e94b4d5a73bc97dc6d4cdc91f,1.5_M4.rc2,1:18:56,14:17.80,1:14:11,4:59.10,0:28.81,0:16.12,0:02.25,26131008,4961248
-
ubuntu12,master:ed3ef0823fde89371a473d20c0127c0bd16d062b,1.5_M4.final,1:18:19,13:41.56,1:11:33,4:53.58,0:28.27,0:15.41,0:01.54,25688600,4898748
ubuntu12,master:ed3ef0823fde89371a473d20c0127c0bd16d062b,1.5_M4.final,1:15:25,13:59.10,1:11:41,4:51.22,0:28.39,0:15.36,0:01.56,25692980,4898576
ubuntu12,master:ed3ef0823fde89371a473d20c0127c0bd16d062b,1.5_M4.final,1:17:12,13:41.63,1:12:06,4:51.32,0:27.97,0:15.38,0:01.58,25688548,4903912
-
ubuntu12,master:566ca1e4769b4ec7e3f6c6a98bb817db2221cf82,1.5_M5.rc1,1:15:17,13:47.21,1:10:44,4:55.33,0:28.11,0:15.16,0:01.54,25726992,4897340
ubuntu12,master:566ca1e4769b4ec7e3f6c6a98bb817db2221cf82,1.5_M5.rc1,1:15:03,14:06.96,1:11:32,4:50.64,0:27.95,0:15.20,0:01.58,25704416,4902296
ubuntu12,master:566ca1e4769b4ec7e3f6c6a98bb817db2221cf82,1.5_M5.rc1,1:17:16,14:07.67,1:12:23,4:53.21,0:27.78,0:15.19,0:01.58,25686264,4902292
-
ubuntu12,master:853bc53cd58a621918f0e5ce662dba263d1befb4,1.5_M5.rc4,1:16:38,14:17.30,1:10:05,4:42.63,0:25.57,0:12.87,0:01.50,25770616,4925008
ubuntu12,master:853bc53cd58a621918f0e5ce662dba263d1befb4,1.5_M5.rc4,1:14:39,14:14.73,1:11:21,4:53.20,0:25.47,0:12.80,0:01.48,25770168,4924972
ubuntu12,master:853bc53cd58a621918f0e5ce662dba263d1befb4,1.5_M5.rc4,1:17:48,14:19.40,1:11:51,4:53.53,0:25.39,0:12.81,0:01.49,25773428,4924908
-
ubuntu12,master:4d8ab87d25ea6e59dc88375813ef88ee0ac813d0,1.5_M5.rc5,1:14:30,13:57.15,1:11:08,4:54.14,0:25.68,0:13.07,0:01.59,25763920,4920848
ubuntu12,master:4d8ab87d25ea6e59dc88375813ef88ee0ac813d0,1.5_M5.rc5,1:15:28,14:06.75,1:11:38,4:51.25,0:25.45,0:13.03,0:01.61,25761664,4921016
ubuntu12,master:4d8ab87d25ea6e59dc88375813ef88ee0ac813d0,1.5_M5.rc5,1:16:24,14:15.42,1:12:12,4:42.56,0:25.46,0:12.97,0:01.59,25780792,4915792
-
ubuntu12,master:a02280f99f05b1c21de2a78de9222a2f62a0dabc,1.5_M5.rc6,1:17:19,13:59.28,1:11:50,4:48.02,0:25.62,0:12.96,0:01.59,25759796,4919568
ubuntu12,master:a02280f99f05b1c21de2a78de9222a2f62a0dabc,1.5_M5.rc6,1:16:54,13:56.51,1:11:56,4:50.50,0:25.67,0:13.04,0:01.62,25763872,4919684
ubuntu12,master:a02280f99f05b1c21de2a78de9222a2f62a0dabc,1.5_M5.rc6,1:14:44,14:02.52,1:10:37,4:44.39,0:25.65,0:13.03,0:01.59,25760268,4919680
-
ubuntu12,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:13:29,14:16.93,1:09:47,4:44.19,0:25.66,0:13.02,0:01.61,25761072,4924448
ubuntu12,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:13:12,13:42.57,1:10:12,4:42.28,0:25.77,0:13.09,0:01.58,25762964,4919732
ubuntu12,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:13:39,13:45.03,1:10:07,4:46.75,0:25.81,0:13.02,0:01.57,25759624,4919564
ubuntu12,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:14:26,14:03.31,1:09:05,4:50.65,0:25.79,0:13.03,0:01.60,25778360,4924472
ubuntu12,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:12:19,14:10.16,1:08:58,4:45.38,0:25.65,0:12.98,0:01.64,25760492,4924412
ubuntu12,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:13:06,14:11.91,1:09:51,4:47.82,0:25.72,0:13.10,0:01.65,25759352,4924528
ubuntu12,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:13:00,14:17.81,1:08:35,4:41.08,0:25.58,0:12.91,0:01.58,25760964,4924396
ubuntu12,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:12:34,14:14.87,1:10:27,4:44.05,0:25.90,0:13.05,0:01.58,25758828,4919492
ubuntu12,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:12:39,13:58.01,1:09:06,4:45.94,0:25.72,0:13.04,0:01.58,25758484,4919364
ubuntu12,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:12:47,14:08.33,1:09:08,4:47.11,0:25.53,0:13.02,0:01.56,25779068,4924396
ubuntu12,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:13:52,14:07.68,1:09:08,4:46.40,0:25.73,0:13.04,0:01.56,25758288,4919468
-
ubuntu12,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:12:57,14:00.27,1:09:06,4:42.51,0:27.42,0:13.03,0:02.06,25759632,4919488
ubuntu12,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:12:04,13:59.75,1:08:42,4:44.67,0:25.89,0:12.96,0:01.61,25758952,4919432
ubuntu12,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:14:45,14:06.80,1:09:02,4:44.42,0:25.67,0:12.91,0:01.57,25758808,4919364
ubuntu12,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:12:22,14:00.67,1:08:50,4:49.61,0:25.64,0:12.94,0:01.56,25759184,4919348
ubuntu12,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:12:50,14:16.78,1:09:12,4:46.23,0:26.19,0:13.09,0:01.59,25758792,4919432
ubuntu12,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:12:52,14:10.55,1:09:17,4:49.04,0:25.65,0:12.99,0:01.56,25759960,4919576
ubuntu12,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:14:52,14:12.56,1:09:32,4:47.73,0:25.68,0:12.97,0:01.62,25758608,4919364
ubuntu12,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:16:23,13:57.25,1:09:12,4:52.86,0:25.69,0:13.12,0:01.57,25760508,4924572
-
ubuntu12,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:12:45,14:07.85,1:09:18,4:43.28,0:25.83,0:13.01,0:01.62,25899200,4934612
ubuntu12,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:12:49,14:18.62,1:09:48,4:41.94,0:25.83,0:13.03,0:01.63,25881828,4934720
ubuntu12,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:13:34,14:27.51,1:09:22,4:43.94,0:25.89,0:13.07,0:01.60,25878960,4934480
ubuntu12,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:12:54,14:14.40,1:10:03,4:42.89,0:25.70,0:12.97,0:01.57,25879656,4934536
ubuntu12,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:12:52,14:11.04,1:09:33,5:12.54,0:25.66,0:13.16,0:01.55,25879568,4934560
ubuntu12,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:12:37,14:03.64,1:09:26,4:51.42,0:25.85,0:12.91,0:01.59,25878848,4939788
ubuntu12,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:13:39,14:12.24,1:09:21,4:43.16,0:25.72,0:13.05,0:01.59,25879452,4934604
ubuntu12,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:12:38,14:21.97,1:09:28,4:41.82,0:25.73,0:13.07,0:01.57,25879820,4934672
-
ubuntu12,dora:6a92f7ede3be3edd58d7819aa611b0c5270756bf,dora-10.0.1.final-118-g6a92f7e,1:13:55,14:13.17,1:08:58,4:50.75,0:26.29,0:13.36,0:01.69,25712228,4929516
ubuntu12,dora:6a92f7ede3be3edd58d7819aa611b0c5270756bf,dora-10.0.1.final-118-g6a92f7e,1:13:51,14:38.41,1:09:08,4:48.95,0:26.51,0:13.48,0:01.88,25710964,4929512
ubuntu12,dora:6a92f7ede3be3edd58d7819aa611b0c5270756bf,dora-10.0.1.final-118-g6a92f7e,1:14:31,14:29.07,1:09:05,4:49.57,0:26.10,0:13.37,0:01.64,25711764,4929620
ubuntu12,dora:6a92f7ede3be3edd58d7819aa611b0c5270756bf,dora-10.0.1.final-118-g6a92f7e,1:15:18,14:23.33,1:09:18,4:54.65,0:26.24,0:13.36,0:01.64,25711120,4929432
</tab>


<tab class=wikitable sep=comma head=top>
* [https://wiki.yoctoproject.org/charts/perf_milestone/performance_test.html RO Charts]
hostname,branch:commit,git describe,bitbake core-image-sato,bitbake virtual/kernel,bitbake core-image-sato (rm_work),bitbake core-image-sato -c rootfs,bitbake -p (rm -rf cache/ tmp/cache/), bitbake -p (rm -rf tmp/cache/),bitbake -p,size of tmp dir, size of tmp dir (rm_work)
fedora19,master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1.5_M1.final,1:19:37,10:38.18,1:12:10,4:03.70,0:34.59,0:18.54,0:02.23,25801440,4887964
fedora19,master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1.5_M1.final,1:19:03,11:24.42,1:12:52,4:01.95,0:34.92,0:18.56,0:02.13,25796764,4887836
fedora19,master:a62aed41f2d8f874f7ae24d0e5be5dbc66ea2199,1.5_M1.final,1:17:50,10:59.62,1:12:27,4:18.09,0:34.47,0:18.48,0:02.11,25801880,4887780
-
fedora19,master:eaa5df34af42b6a37f6506847d0f3ef6ba0d298a,1.5_M1.final-412-geaa5df3,1:20:02,11:43.17,1:13:52,3:58.63,0:35.54,0:19.14,0:02.13,25704568,4911492
fedora19,master:eaa5df34af42b6a37f6506847d0f3ef6ba0d298a,1.5_M1.final-412-geaa5df3,1:21:20,12:11.54,1:14:48,3:59.29,0:35.74,0:19.06,0:02.18,25710016,4911228
fedora19,master:eaa5df34af42b6a37f6506847d0f3ef6ba0d298a,1.5_M1.final-412-geaa5df3,1:21:13,11:04.79,1:14:39,4:17.40,0:35.37,0:18.72,0:02.10,25704988,4911520
-
fedora19,master:9de0ad47462c13ac4a2d73e12e92be6c1b4e0415,1.5_M3,1:22:38,11:18.43,1:18:37,4:17.43,0:36.29,0:19.14,0:02.10,25783316,4921400
fedora19,master:9de0ad47462c13ac4a2d73e12e92be6c1b4e0415,1.5_M3,1:22:51,11:29.19,1:17:53,4:09.21,0:35.81,0:19.49,0:02.13,25782900,4921324
fedora19,master:9de0ad47462c13ac4a2d73e12e92be6c1b4e0415,1.5_M3,1:23:13,11:28.11,1:16:30,4:13.87,0:35.91,0:19.17,0:02.11,25785912,4921496
-
fedora19,master:5745e45b18e5099e94b4d5a73bc97dc6d4cdc91f,1.5_M4.rc2,1:23:20,12:42.28,1:16:41,4:08.14,0:35.09,0:18.98,0:02.29,26535116,5049040
fedora19,master:5745e45b18e5099e94b4d5a73bc97dc6d4cdc91f,1.5_M4.rc2,1:23:58,12:29.62,1:17:23,4:07.62,0:34.85,0:19.07,0:02.33,26540144,5048920
fedora19,master:5745e45b18e5099e94b4d5a73bc97dc6d4cdc91f,1.5_M4.rc2,1:25:03,11:49.90,1:18:40,4:15.76,0:34.56,0:18.93,0:02.30,26536564,5048832
-
fedora19,master:ed3ef0823fde89371a473d20c0127c0bd16d062b,1.5_M4.final,1:21:45,12:37.91,1:15:53,4:05.91,0:34.19,0:18.32,0:01.60,26096196,4986476
fedora19,master:ed3ef0823fde89371a473d20c0127c0bd16d062b,1.5_M4.final,1:22:51,12:50.33,1:17:02,4:06.61,0:34.13,0:18.61,0:01.62,26093328,4986408
fedora19,master:ed3ef0823fde89371a473d20c0127c0bd16d062b,1.5_M4.final,1:22:32,12:01.28,1:17:13,4:15.72,0:34.03,0:18.35,0:01.59,26094088,4986944
-
fedora19,master:566ca1e4769b4ec7e3f6c6a98bb817db2221cf82,1.5_M5.rc1,1:22:27,13:14.06,1:16:17,3:57.51,0:34.03,0:18.30,0:01.61,26090072,4984856
fedora19,master:566ca1e4769b4ec7e3f6c6a98bb817db2221cf82,1.5_M5.rc1,1:22:51,13:12.33,1:16:06,3:58.44,0:34.06,0:18.15,0:01.62,26090640,4985060
fedora19,master:566ca1e4769b4ec7e3f6c6a98bb817db2221cf82,1.5_M5.rc1,1:22:22,12:05.53,1:16:26,4:08.26,0:33.83,0:18.11,0:01.59,26090756,4984916
-
fedora19,master:853bc53cd58a621918f0e5ce662dba263d1befb4,1.5_M5.rc4,1:20:58,12:53.74,1:14:58,3:53.32,0:31.15,0:15.29,0:01.61,26176264,5008052
fedora19,master:853bc53cd58a621918f0e5ce662dba263d1befb4,1.5_M5.rc4,1:22:00,12:03.23,1:15:15,4:01.62,0:30.91,0:15.24,0:01.58,26177500,5007748
fedora19,master:853bc53cd58a621918f0e5ce662dba263d1befb4,1.5_M5.rc4,1:22:11,12:07.04,1:16:20,3:59.87,0:30.93,0:15.30,0:01.59,26181592,5008024
-
fedora19,master:4d8ab87d25ea6e59dc88375813ef88ee0ac813d0,1.5_M5.rc5,1:23:16,13:40.63,1:16:49,3:47.56,0:31.17,0:15.36,0:01.67,26168976,5003560
fedora19,master:4d8ab87d25ea6e59dc88375813ef88ee0ac813d0,1.5_M5.rc5,1:22:23,12:48.88,1:16:17,3:51.08,0:31.24,0:15.36,0:01.62,26169684,5003712
fedora19,master:4d8ab87d25ea6e59dc88375813ef88ee0ac813d0,1.5_M5.rc5,1:22:37,12:23.94,1:16:50,3:59.73,0:31.08,0:15.19,0:01.61,26167128,5003628
-
fedora19,master:a02280f99f05b1c21de2a78de9222a2f62a0dabc,1.5_M5.rc6,1:23:49,12:43.59,1:16:15,3:49.05,0:31.30,0:15.55,0:01.63,26165572,5007660
fedora19,master:a02280f99f05b1c21de2a78de9222a2f62a0dabc,1.5_M5.rc6,1:22:48,15:11.65,1:16:33,4:01.86,0:30.94,0:15.26,0:01.60,26165408,5007688
fedora19,master:a02280f99f05b1c21de2a78de9222a2f62a0dabc,1.5_M5.rc6,1:21:56,12:14.56,1:16:20,4:05.10,0:30.99,0:15.25,0:01.63,26164444,5007468
-
fedora19,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:16:54,11:52.82,1:12:45,4:02.22,0:31.04,0:15.21,0:01.60,26162736,5007468
fedora19,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:19:45,12:02.39,1:13:23,4:06.09,0:31.20,0:15.36,0:01.62,26165196,5007264
fedora19,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:19:18,11:57.40,1:14:25,4:05.45,0:31.07,0:15.25,0:01.61,26164232,5007452
fedora19,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:17:39,12:01.11,1:12:22,3:58.81,0:31.31,0:15.27,0:01.60,26164012,5007328
fedora19,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:17:45,11:54.60,1:12:47,4:02.61,0:31.12,0:15.34,0:01.59,26163676,5007224
fedora19,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:17:09,11:58.36,1:12:59,4:05.55,0:30.96,0:15.23,0:01.60,26163708,5007104
fedora19,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:16:55,11:59.64,1:13:23,4:09.24,0:31.00,0:15.26,0:01.63,26164080,5007216
fedora19,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:17:30,12:08.20,1:13:28,4:03.57,0:31.05,0:15.27,0:01.60,26163924,5007396
fedora19,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:17:45,12:09.95,1:13:18,4:03.98,0:31.09,0:15.12,0:01.60,26162440,5007220
fedora19,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:16:35,11:54.45,1:13:46,4:03.80,0:31.10,0:15.21,0:01.59,26163492,5007436
fedora19,dora:102bf5e0f640fe85068452a42b85077f1c81e0c9,dora-10.0.0,1:17:09,11:40.41,1:13:31,4:09.40,0:31.05,0:15.23,0:01.62,26163252,5007192
-
fedora19,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:17:03,11:59.04,1:13:05,4:05.67,0:31.13,0:15.32,0:01.60,26163708,5007140
fedora19,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:17:17,11:59.69,1:13:00,4:07.91,0:31.05,0:15.19,0:01.59,26163740,5007244
fedora19,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:17:27,12:07.22,1:12:53,4:02.08,0:31.03,0:15.25,0:01.59,26163356,5007356
fedora19,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:16:18,12:03.39,1:13:09,3:58.62,0:30.98,0:15.16,0:01.57,26163220,5007136
fedora19,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:17:27,11:48.97,1:13:03,4:05.68,0:31.36,0:15.21,0:01.59,26164544,5006968
fedora19,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:16:53,11:37.84,1:13:18,4:04.92,0:31.00,0:15.17,0:01.61,26165656,5007164
fedora19,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:17:18,11:58.27,1:13:08,4:03.75,0:31.10,0:15.30,0:01.57,26163624,5007312
fedora19,dora:faf8f8660fae26a14578738b642fdeb08bcb9d54,dora-10.0.0.final,1:18:37,11:42.27,1:12:56,4:05.39,0:30.96,0:15.22,0:01.60,26163188,5007108
-
fedora19,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:17:44,11:59.38,1:13:14,4:04.62,0:31.14,0:15.28,0:01.59,26285448,5022564
fedora19,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:17:43,11:58.18,1:13:29,4:00.26,0:30.96,0:15.33,0:01.60,26287488,5022536
fedora19,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:17:09,12:05.39,1:12:50,3:56.98,0:31.14,0:15.24,0:01.60,26284228,5022676
fedora19,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:17:38,11:56.33,1:13:00,4:02.62,0:31.01,0:15.19,0:01.60,26284388,5022524
fedora19,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:17:48,12:04.61,1:13:38,4:07.60,0:31.03,0:15.16,0:01.60,26284568,5022740
fedora19,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:17:52,12:08.54,1:13:08,4:13.37,0:31.01,0:15.48,0:01.60,26287384,5022572
fedora19,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:17:17,12:04.67,1:13:45,4:05.42,0:31.51,0:15.34,0:01.58,26284428,5022612
fedora19,dora:3cf2d232529c4fd6b58f87ddbf3df9d805e4180f,dora-10.0.0.final-538-g3cf2d23,1:18:17,12:10.28,1:13:41,4:07.05,0:31.07,0:15.27,0:01.60,26285060,5022508
-
fedora19,dora:6a92f7ede3be3edd58d7819aa611b0c5270756bf,dora-10.0.1.final-118-g6a92f7e,1:18:46,12:29.52,1:14:43,4:01.81,0:31.68,0:15.84,0:01.68,26113380,5016712
fedora19,dora:6a92f7ede3be3edd58d7819aa611b0c5270756bf,dora-10.0.1.final-118-g6a92f7e,1:18:27,12:09.93,1:13:29,4:08.12,0:31.69,0:16.46,0:02.01,26112896,5016836
fedora19,dora:6a92f7ede3be3edd58d7819aa611b0c5270756bf,dora-10.0.1.final-118-g6a92f7e,1:18:51,12:20.40,1:13:58,4:02.34,0:31.61,0:15.77,0:01.70,26114020,5016612
fedora19,dora:6a92f7ede3be3edd58d7819aa611b0c5270756bf,dora-10.0.1.final-118-g6a92f7e,1:19:21,12:32.61,1:13:35,4:03.69,0:31.70,0:15.83,0:01.75,26116744,5016724
</tab>


== 1.4.1 & 1.4.2 & 1.4.3 Build Time results ==
Raw test data from the retired Romania test setup has been moved to
* [[Performance_Test_Old_Results]]
* [[Performance_Test_Weekly_1.5]] has the old weekly results ran during the 1.5 cycle.


<tab class=wikitable sep=comma head=top>
Configuration for the Romania Setup 1.6 - 2.1:
hostname,branch:commit,git describe,bitbake core-image-sato,bitbake virtual/kernel,bitbake core-image-sato (rm_work),bitbake core-image-sato -c rootfs,bitbake -p (rm -rf cache/ tmp/cache/), bitbake -p (rm -rf tmp/cache/),bitbake -p,size of tmp dir, size of tmp dir (rm_work)
ubuntu12,dylan:789b2b7e0cca404ff8ba8ed5ba823d215e6ba5b4,dylan-9.0.0,1:15:10,11:51.45,1:06:52,5:15.66,0:26.08,0:14.64,0:01.95,28104788,4693596
ubuntu12,dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f,yocto-1.4.1,1:16:41,11:32.41,1:06:51,5:14.78,0:26.75,0:14.81,0:01.96,28213628,4743912
ubuntu12,dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f,yocto-1.4.1,1:14:31,11:34.66,1:06:41,5:12.70,0:26.54,0:14.74,0:01.92,28213772,4743844
ubuntu12,dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f,yocto-1.4.1,1:12:39,11:35.99,1:06:59,5:16.48,0:26.77,0:14.71,0:01.96,28217808,4743800
ubuntu12,dylan:d734ab491a30078d43dee5440c03acce2d251425,yocto-1.4.2,1:12:58,12:56.22,1:07:45,4:54.36,0:26.78,0:14.73,0:01.95,25349116,4767324
ubuntu12,dylan:d734ab491a30078d43dee5440c03acce2d251425,yocto-1.4.2,1:13:33,12:44.85,1:08:10,5:01.71,0:26.87,0:14.80,0:01.99,25349708,4767348
ubuntu12,dylan:d734ab491a30078d43dee5440c03acce2d251425,yocto-1.4.2,1:13:00,12:59.19,1:08:13,5:07.04,0:26.83,0:14.73,0:01.97,25363408,4767380
ubuntu12,dylan:7ab82abcad9ede847c42c01d2a4f4c2c18877e6f,yocto-1.4.2-64-g7ab82ab,1:09:23,12:58.26,1:07:54,4:57.79,0:26.79,0:14.88,0:01.96,25365532,4767572
ubuntu12,dylan:7ab82abcad9ede847c42c01d2a4f4c2c18877e6f,yocto-1.4.2-64-g7ab82ab,1:09:33,13:00.44,1:07:48,5:01.62,0:27.07,0:14.91,0:02.11,25366496,4767568
ubuntu12,dylan:7ab82abcad9ede847c42c01d2a4f4c2c18877e6f,yocto-1.4.2-64-g7ab82ab,1:12:25,13:05.14,1:09:17,4:58.21,0:26.99,0:14.87,0:01.95,25351352,4767496
</tab>


 
* Ubuntu 12.04 (uname -nr: ubuntu12 3.8.0-31-generic) and
<tab class=wikitable sep=comma head=top>
* Fedora 19 (uname -nr: fedora19 3.11.3-201.fc19.x86_64)
hostname,branch:commit,git describe,bitbake core-image-sato,bitbake virtual/kernel,bitbake core-image-sato (rm_work),bitbake core-image-sato -c rootfs,bitbake -p (rm -rf cache/ tmp/cache/), bitbake -p (rm -rf tmp/cache/),bitbake -p,size of tmp dir, size of tmp dir (rm_work)
* HW configuration: SandyBridge i7-2600 CPU @ 3.40GHz (4 cores/8 threads), 8GB RAM, 1 x 500 GB HDD
fedora19,dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f,yocto-1.4.1,1:14:55,10:11.74,1:10:35,4:06.95,0:34.06,0:17.79,0:01.99,28710092,4835464
fedora19,dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f,yocto-1.4.1,1:12:34,10:09.63,1:10:30,4:03.92,0:33.53,0:17.84,0:01.92,28713992,4835460
fedora19,dylan:73f103bf9b2cdf985464dc53bf4f1cfd71d4531f,yocto-1.4.1,1:17:15,10:11.67,1:11:47,4:45.40,0:33.90,0:17.31,0:01.97,28710884,4835484
fedora19,dylan:d734ab491a30078d43dee5440c03acce2d251425,yocto-1.4.2,1:14:50,10:53.97,1:15:47,4:19.85,0:34.18,0:17.92,0:01.92,25778860,4858892
fedora19,dylan:d734ab491a30078d43dee5440c03acce2d251425,yocto-1.4.2,1:14:58,10:43.16,1:12:00,4:07.15,0:33.93,0:17.88,0:01.98,25776164,4858920
fedora19,dylan:d734ab491a30078d43dee5440c03acce2d251425,yocto-1.4.2,1:16:11,11:23.14,1:12:31,4:08.25,0:33.64,0:18.03,0:01.95,25775976,4858876
fedora19,dylan:7ab82abcad9ede847c42c01d2a4f4c2c18877e6f,yocto-1.4.2-64-g7ab82ab,1:12:05,10:26.20,1:15:21,4:19.49,0:34.25,0:18.12,0:01.95,25774008,4858772
fedora19,dylan:7ab82abcad9ede847c42c01d2a4f4c2c18877e6f,yocto-1.4.2-64-g7ab82ab,1:11:50,10:30.01,1:11:48,4:16.69,0:33.99,0:18.03,0:01.94,25775640,4858692
fedora19,dylan:7ab82abcad9ede847c42c01d2a4f4c2c18877e6f,yocto-1.4.2-64-g7ab82ab,1:11:51,10:25.89,1:11:56,4:42.52,0:34.19,0:17.97,0:01.94,25776076,4858728
</tab>

Latest revision as of 14:17, 28 January 2019

Build time tracking

Script used for running the builds:

  • Before morty: poky/scripts/contrib/build-perf-test.sh
  • After morty: poky/scripts/oe-build-perf-test

Two identical test systems are used for the results, with all updates available installed. Extra packages installed (besides what's by the default in the distro) are the ones in the Quick Start guide.

Configuration of the GDC Setup 2.1 - Current master

  • Ubuntu 15.10 (Wily Werewolf)" --> 4.2.0-16-generic
  • Fedora release 23 (Twenty Three) --> 4.7.10-100.fc23.x86_64
  • HW configuration: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz, 16GB RAM, 1 x 600 GB SDD, 1 x 2 TB HDD

All system/distro settings are distro's defaults, that includes:

- FS mount options: ext4 rw,relatime,data=ordered

- IO scheduler: cfq and CPU governer: ondemand

local.conf it's the default one (the script doesn't touch it except for changing DL_DIR if necessary).

It however exports these in the enviroment: 

BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
MACHINE = "qemux86"

FAQ:

  • Q: Where is 1.5_M2? It's there but it's called: 1.5_M1.final-412-geaa5df3 The M2 build wasn't released hence no git tag
  • Q: What's with the different number of RCs? See Yocto_Project_Release_Process#Milestone_Releases
  • Q: Why are there more results for some commits? The script is ran from a cron job multiple times a day, sometimes master changes daily sometimes it doesn't, hence more results for one commit (ie weekends)


Daily Test Results

Build performance test suite is run multiple times per day. Latest test reports for master branch can found here:

Milestone Test results

Guadalajara Setup from 2.1 M4 to actual master

Old Test results

Milestone test results from the retired Romania Setup (Yocto 1.6 – 2.1) can be found from here:

Raw test data from the retired Romania test setup has been moved to

Configuration for the Romania Setup 1.6 - 2.1:

  • Ubuntu 12.04 (uname -nr: ubuntu12 3.8.0-31-generic) and
  • Fedora 19 (uname -nr: fedora19 3.11.3-201.fc19.x86_64)
  • HW configuration: SandyBridge i7-2600 CPU @ 3.40GHz (4 cores/8 threads), 8GB RAM, 1 x 500 GB HDD
Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Performance_Test&oldid=46678"