TipsAndTricks/InvestigatingBuildTime: Difference between revisions

From Yocto Project
Jump to navigationJump to search
No edit summary
No edit summary
 
(8 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Investigating Build Time ==
== Draft: Investigating Build Time ==


(Placeholder. Demonstrate use of buildstats and pybootchartgui)
(Placeholder. Demonstrate use of buildstats and pybootchartgui)


The <tt>buildstats</tt> data produce by the class with the same name if included in <tt>USER_CLASSES</tt> variable
'''Current status''': The intention is to use the scripts under scripts/contrib/bb-perf to parse
provides '''process statistics''' for every task. This data  can offer insight of the time spent by the builder (<tt>bitbake</tt>) when
buildstat data and produce useful plots.
 
 
The <tt>buildstats</tt> data produced by the class with the same name provides '''process statistics''' for every recipe task.
This data  can offer insight of the time spent by the builder (<tt>bitbake</tt>) when
constructing a particular target at process level. This article focus more on profiling than optimization, the former being
constructing a particular target at process level. This article focus more on profiling than optimization, the former being
the base for the latter.
the base for the latter.
Line 14: Line 18:
   $ . oe-init-build-env
   $ . oe-init-build-env
   $ bitbake world
   $ bitbake world
=== Build Machine ===
Build time depends on the build machine HW and for this particular article we use a Linux box with the following characteristics:


=== Buildstats ===
=== Buildstats ===


There are two set of process' stat data: time and IO stats. For the moment, the time stats to be used are the following:
buildstats.bbclass stores mainly two set of process' stats: time and IO stats. For the moment, we are currently focusing on the following
 
time data:


* /proc/[pid]/stat: utime, stime
* /proc/[pid]/stat: utime, stime
Line 30: Line 29:
* python resource module: Child rusage ru_utime, Child rusage ru_stime
* python resource module: Child rusage ru_utime, Child rusage ru_stime


and the following IO stats:
and the IO stats:


* /proc/[pid]/io: IO rchar, IO wchar
* /proc/[pid]/io: IO rchar, IO wchar
* /proc/[pid]/io: IO read_bytes, IO write_bytes
* /proc/[pid]/io: IO read_bytes, IO write_bytes


With this separation of buildstats, proceed running the following script (once the <code>bitbake world</code> command has finished)
With this data, the following script (once the <code>bitbake world</code> command has finished) is used
to plot the top N stat consumers
to either plot the top N recipes-task stat values


  $ ../scripts/contrib/bb-perf/buildstats-plot.sh -s <BUILDSTATS> -N $N | gnuplot -p
  $ ../scripts/contrib/bb-perf/buildstats-plot.sh -s <BUILDSTATS> -n $N | gnuplot -p


In case we want a more general picture, getting data per task type instead of recipe's task, the script needs <code>-S</code> (for Sum)
or sum stats values per task for all (world) recipes
argument


  $ ../scripts/contrib/bb-perf/buildstats-plot.sh -s <BUILDSTATS> -S | gnuplot -p
  $ ../scripts/contrib/bb-perf/buildstats-plot.sh -s <BUILDSTATS> -S | gnuplot -p


The former produces a more granular plot and second a more general idea where the type is spent per task set.


=== Results ===
=== Results ===


$ IMAGE='set terminal png fontscale 1 size 1280,720; set output "IO-write_bytes:IO-read_bytes.png"'
Preliminar results can be found [[MortyBuildstats|here]]
 
==== utime and stime ====
 
Time spent by a certain process can be obtain looking at the <tt>stime<</tt> and <tt>time</tt> corresponding to the amount of time the process
has been scheduled in kernel and user mode.
 
$ IMAGE='set terminal png fontscale 1 size 1280,720; set output "IO-write_bytes:IO-read_bytes.png"'
$ ../scripts/contrib/bb-perf/buildstats-plot.sh -s 'stime:utime' -N 20 > input.gpl; gnuplot -e "\'$IMAGE\'" input.gpl
$ ../scripts/contrib/bb-perf/buildstats-plot.sh -s 'stime:utime' -S | gnuplot -p

Latest revision as of 21:12, 28 February 2017

Draft: Investigating Build Time

(Placeholder. Demonstrate use of buildstats and pybootchartgui)

Current status: The intention is to use the scripts under scripts/contrib/bb-perf to parse buildstat data and produce useful plots.


The buildstats data produced by the class with the same name provides process statistics for every recipe task. This data can offer insight of the time spent by the builder (bitbake) when constructing a particular target at process level. This article focus more on profiling than optimization, the former being the base for the latter.

Setup

 $ cd poky
 $ git checkout -b morty origin/morty
 $ . oe-init-build-env
 $ bitbake world

Buildstats

buildstats.bbclass stores mainly two set of process' stats: time and IO stats. For the moment, we are currently focusing on the following time data:

  • /proc/[pid]/stat: utime, stime
  • /proc/[pid]/stat: cutime, cstime
  • python resource module: rusage ru_utime, rusage ru_stime
  • python resource module: Child rusage ru_utime, Child rusage ru_stime

and the IO stats:

  • /proc/[pid]/io: IO rchar, IO wchar
  • /proc/[pid]/io: IO read_bytes, IO write_bytes

With this data, the following script (once the bitbake world command has finished) is used to either plot the top N recipes-task stat values 

$ ../scripts/contrib/bb-perf/buildstats-plot.sh -s <BUILDSTATS> -n $N | gnuplot -p

or sum stats values per task for all (world) recipes 

$ ../scripts/contrib/bb-perf/buildstats-plot.sh -s <BUILDSTATS> -S | gnuplot -p

The former produces a more granular plot and second a more general idea where the type is spent per task set.

Results

Preliminar results can be found here

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=TipsAndTricks/InvestigatingBuildTime&oldid=24480"