Tracing and Profiling

Tracing and Profiling in Yocto

Yocto bundles a large number of tracing and profiling tools; many of them are available however only in 'sdk' images. So, in order to be able to access all of the tools described here, please first build and boot an 'sdk' image e.g.

$ bitbake poky-image-sdk-live

This page contains basic information and simple examples to help you get started with each of the tracing and profiling tools available in Yocto. There is no overall structure - each tool is simply listed in alphabetical order. If you'd like to see more involved real-world use cases that show how you might use these tools in concert, see Tracing and Profiling End-to-End Examples.

blktrace

blktrace is a tool for tracing and reporting low-level disk I/O. blktrace provides the tracing half of the equation; its output can be piped into the blkparse program, which renders the data in a human-readable form and does some basic analysis:

$ blktrace /dev/sda | blkparse -

systemtap

SystemTap is a system-wide script-based tracing and profiling tool.

For example, this probe from the SystemTap tutorial [1] simply prints a line every time any process on the system open()s a file. For each line, it prints the executable name of the program that opened the file, along with its pid, and the name of the file it opened (or tried to open), which it extracts from the open syscall's argstr.

probe syscall.open
{
        printf ("%s(%d) open (%s)\n", execname(), pid(), argstr)
}
probe timer.ms(4000) # after 4 seconds
{
        exit ()
}

Normally, to execute this probe, you'd simply install systemtap on the system you want to probe, and directly run the probe on that system e.g. assuming the name of the file containing the above text is trace_open.stp:

# stap trace_open.stp

What systemtap does under the covers to run this probe is 1) parse and convert the probe to an equivalent 'C' form, 2) compile the 'C' form into a kernel module, 3) insert the module into the kernel, which arms it, and 4) collect the data generated by the probe and display it to the user.

In order to accomplish steps 1 and 2, the 'stap' program needs access to the kernel build system that produced the kernel that the probed system is running. In the case of a typical embedded system (the 'target'), the kernel build system unfortunately isn't typically part of the image running on the target. It is normally available on the 'host' system that produced the target image however; in such cases, steps 1 and 2 are executed on the host system, and steps 3 and 4 are executed on the target system, using only the systemtap 'runtime'.

The systemtap support in Yocto assumes that only steps 3 and 4 are run on the target; it is possible to do everything on the target, but this section assumes only the typical embedded use-case.

So basically what you need to do in order to run a systemtap script on the target is to 1) on the host system, compile the probe into a kernel module that makes sense to the target, 2) copy the module onto the target system and 3) insert the module into the target kernel, which arms it, and 4) collect the data generated by the probe and display it to the user.

For many more examples, and documentation, see [2].