Tracing and Profiling - Revision history

Tzanussi: /* operf */

2013-01-27T23:40:52Z

operf

@@ Line 1,837: / Line 1,837: @@
 100.000 vmlinux-3.4.26-yocto-standard
-Display with callgraph:
+You can also, as always, display the results with callgraphs:
   root@crownbay:~# opreport --callgraph
@@ Line 1,864: / Line 1,864: @@
       50.0000  vmlinux-3.4.26-yocto-standard operf                    memset
       50.0000  vmlinux-3.4.26-yocto-standard operf                    memset [self]
 ==== OProfileUI - A GUI for OProfile ====

Tzanussi: /* operf */

2013-01-27T23:34:51Z

operf

@@ Line 1,836: / Line 1,836: @@
   	------------------
 100.000 vmlinux-3.4.26-yocto-standard
 ==== OProfileUI - A GUI for OProfile ====

Tzanussi: /* operf */

2013-01-27T23:31:39Z

operf

← Older revision		Revision as of 23:31, 27 January 2013
Line 1,798:		Line 1,798:
	Profiling done.		Profiling done.

			You can use 'opreport' to get the results just as you would with 'legacy oprofile':

			root@crownbay:~# opreport
			Using /home/root/oprofile_data/samples/ for samples directory.
			CPU: Intel Architectural Perfmon, speed 1.3e+06 MHz (estimated)
			Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (No unit mask) count 90000
			CPU_CLK_UNHALT...\|
			samples\| %\|
			------------------
			3431 99.2766 operf
			CPU_CLK_UNHALT...\|
			samples\| %\|
			------------------
			1822 53.1041 vmlinux-3.4.26-yocto-standard
			809 23.5791 operf
			680 19.8193 libc-2.16.so
			117 3.4101 libstdc++.so.6.0.17
			3 0.0874 emgd
			13 0.3762 dropbearmulti
			CPU_CLK_UNHALT...\|
			samples\| %\|
			------------------
			9 69.2308 vmlinux-3.4.26-yocto-standard
			3 23.0769 dropbearmulti
			1 7.6923 libc-2.16.so
			11 0.3183 matchbox-terminal
			CPU_CLK_UNHALT...\|
			samples\| %\|
			------------------
			6 54.5455 vmlinux-3.4.26-yocto-standard
			3 27.2727 libglib-2.0.so.0.3400.3
			1 9.0909 libc-2.16.so
			1 9.0909 librt-2.16.so
			1 0.0289 init.sysvinit
			CPU_CLK_UNHALT...\|
			samples\| %\|
			------------------
			1 100.000 vmlinux-3.4.26-yocto-standard

	==== OProfileUI - A GUI for OProfile ====		==== OProfileUI - A GUI for OProfile ====

Tzanussi: /* oprofile */

2013-01-27T23:28:45Z

oprofile

← Older revision		Revision as of 23:28, 27 January 2013
Line 1,776:		Line 1,776:

	root@crownbay:~# opcontrol --reset		root@crownbay:~# opcontrol --reset

			==== operf ====
			On systems with a kernel that supports it, you can more simply use the 'operf' command to record profiles. 'operf' also allows you to easily profile a single application by running it as the last part of the 'operf' command. You can tell if your kernel supports 'operf' by simply running it with no arguments:

			root@crownbay:~# operf
			usage: operf [ options ] [ --system-wide \| --pid <pid> \| [ command [ args ] ] ]
			See operf man page for details.

			If it doesn't support operf, you'll get a message that says the kernel doesn't support it. You can run a system-wide profile by using operf with the -s (system-wide) option:

			root@crownbay:~# operf -k /boot/vmlinux-`uname -r` -s -g
			operf: Profiler started
			operf: Press Ctl-c or 'kill -SIGINT 1222' to stop profiling
			^C* * * * WARNING: Profiling rate was throttled back by the kernel * * * *
			The number of samples actually recorded is less than expected, but is
			probably still statistically valid. Decreasing the sampling rate is the
			best option if you want to avoid throttling.

			See the /home/root/oprofile_data/samples/operf.log file for statistics about lost samples.

			Profiling done.


	==== OProfileUI - A GUI for OProfile ====		==== OProfileUI - A GUI for OProfile ====

Tzanussi: /* Collecting and viewing a userspace trace on the target (inside a shell) */

2013-01-08T21:55:48Z

Collecting and viewing a userspace trace on the target (inside a shell)

← Older revision		Revision as of 21:55, 8 January 2013
Line 2,003:		Line 2,003:

	===== Collecting and viewing a userspace trace on the target (inside a shell) =====		===== Collecting and viewing a userspace trace on the target (inside a shell) =====

			For lttng userspace tracing, you need to have a properly instrumented userspace program. For this example, we'll use the 'hello' test program generated by the lttng-ust build.

			The 'hello' test program isn't installed on the rootfs by the lttng-ust build, so we need to copy it over manually. First cd into the build directory that contains the hello executable:

			$ cd build/tmp/work/core2-poky-linux/lttng-ust/2.0.5-r0/git/tests/hello/.libs

			Copy that over to the target machine:

			$ scp hello root@192.168.1.20:

			You now have the instrumented lttng 'hello world' test program on the target, ready to test.

	First, from the target, ssh to the target:		First, from the target, ssh to the target:
Line 2,055:		Line 2,067:
	root@crownbay:~# lttng destroy		root@crownbay:~# lttng destroy
	Session auto-20190303-021943 destroyed at /home/root		Session auto-20190303-021943 destroyed at /home/root


	===== Manually copying a trace to the host and viewing it in Eclipse (i.e. using Eclipse without network support) =====		===== Manually copying a trace to the host and viewing it in Eclipse (i.e. using Eclipse without network support) =====

Tzanussi: /* Collecting and Viewing Traces */

2013-01-08T21:49:44Z

Collecting and Viewing Traces

@@ Line 2,001: / Line 2,001: @@
   drwxr-xr-x    5 root     root          1024 Oct 15 23:57 ..
   drwxrwx---    3 root     root          1024 Oct 15 23:21 auto-20121015-232120
 ===== Manually copying a trace to the host and viewing it in Eclipse (i.e. using Eclipse without network support) =====

Tzanussi: /* Using dynamic tracepoints */

2012-11-15T21:19:56Z

Using dynamic tracepoints

← Older revision		Revision as of 21:19, 15 November 2012
Line 763:		Line 763:
	perf record -e probe:do_fork -aR sleep 1		perf record -e probe:do_fork -aR sleep 1

	Adding a new tracepoint via 'perf probe' results in an event with all the expected files and format in /sys/kernel/debug/tracing/events, just the same as for static tracepoints:		Adding a new tracepoint via 'perf probe' results in an event with all the expected files and format in /sys/kernel/debug/tracing/events, just the same as for static tracepoints (as discussed in more detail in the [https://wiki.yoctoproject.org/wiki/Tracing_and_Profiling#The_.27trace_events.27_subsystem trace events subsystem section]:

	root@crownbay:/sys/kernel/debug/tracing/events/probe/do_fork# ls -al		root@crownbay:/sys/kernel/debug/tracing/events/probe/do_fork# ls -al

Tzanussi: /* Using dynamic tracepoints */

2012-11-15T21:17:06Z

Using dynamic tracepoints

← Older revision		Revision as of 21:17, 15 November 2012
Line 753:		Line 753:

	===== Using dynamic tracepoints =====		===== Using dynamic tracepoints =====
			perf isn't restricted to the fixed set of static tracepoints listed by 'perf list'. Users can also add their own 'dynamic' tracepoints anywhere in the kernel. For instance, suppose we want to define our own tracepoint on do_fork(). We can do that using the 'perf probe' perf subcommand:

	root@crownbay:~# perf probe do_fork		root@crownbay:~# perf probe do_fork
Line 849:		Line 850:
	'''Tying It Together''': ''Dynamic tracepoints are implemented under the covers by kprobes and uprobes. kprobes and''		'''Tying It Together''': ''Dynamic tracepoints are implemented under the covers by kprobes and uprobes. kprobes and''
	'' uprobes are also used by and in fact are the main focus of SystemTap.''		'' uprobes are also used by and in fact are the main focus of SystemTap.''

	==== Documentation ====		==== Documentation ====
	Online versions of the man pages for the commands discussed in this section can be found here:		Online versions of the man pages for the commands discussed in this section can be found here:

Tzanussi: /* Tracing Block I/O via 'ftrace' */

2012-11-15T15:32:25Z

Tracing Block I/O via 'ftrace'

← Older revision		Revision as of 15:32, 15 November 2012
Line 2,353:		Line 2,353:

	===== Tracing Block I/O via 'ftrace' =====		===== Tracing Block I/O via 'ftrace' =====
			It's also possible to trace block I/O using only [https://wiki.yoctoproject.org/wiki/Tracing_and_Profiling#The_.27trace_events.27_subsystem trace events subsystem], which can be useful for casual tracing if you don't want bother dealing with the userspace tools.

			To enable tracing for a given device, use /sys/block/xxx/trace/enable, where xxx is the device name. This for example enables tracing for /dev/sdc:

	root@crownbay:/sys/kernel/debug/tracing# echo 1 > /sys/block/sdc/trace/enable		root@crownbay:/sys/kernel/debug/tracing# echo 1 > /sys/block/sdc/trace/enable

			Once you've selected the device(s) you want to trace, selecting the 'blk' tracer will turn the blk tracer on:

	root@crownbay:/sys/kernel/debug/tracing# cat available_tracers		root@crownbay:/sys/kernel/debug/tracing# cat available_tracers
Line 2,360:		Line 2,365:

	root@crownbay:/sys/kernel/debug/tracing# echo blk > current_tracer		root@crownbay:/sys/kernel/debug/tracing# echo blk > current_tracer

			Execute the workload you're interested in:

	root@crownbay:/sys/kernel/debug/tracing# cat /media/sdc/testfile.txt		root@crownbay:/sys/kernel/debug/tracing# cat /media/sdc/testfile.txt

			And look at the output (note here that we're using 'trace_pipe' instead of trace to capture this trace - this allows us to wait around on the pipe for data to appear):

	root@crownbay:/sys/kernel/debug/tracing# cat trace_pipe		root@crownbay:/sys/kernel/debug/tracing# cat trace_pipe
Line 2,379:		Line 2,388:
	cat-3587 [001] d..1 3023.276497: 8,32 m N cfq3587 activate rq, drv=1		cat-3587 [001] d..1 3023.276497: 8,32 m N cfq3587 activate rq, drv=1
	cat-3587 [001] d..2 3023.276500: 8,32 D R 1699848 + 8 [cat]		cat-3587 [001] d..2 3023.276500: 8,32 D R 1699848 + 8 [cat]

			And this turns off tracing for the specified device:

	root@crownbay:/sys/kernel/debug/tracing# echo 0 > /sys/block/sdc/trace/enable		root@crownbay:/sys/kernel/debug/tracing# echo 0 > /sys/block/sdc/trace/enable

Tzanussi: /* perf */

2012-11-15T05:34:11Z

perf

← Older revision		Revision as of 05:34, 15 November 2012
Line 63:		Line 63:
	The 'perf' tool is the profiling and tracing tool that comes bundled with the Linux kernel.		The 'perf' tool is the profiling and tracing tool that comes bundled with the Linux kernel.

	Don't let the fact that it's part of the kernel fool you into thinking that it's only for tracing and profiling the kernel - you can indeed use it to trace and profile just the kernel , but you can also use it to profile specific applications separately (with or without kernel context), and you can also use it to trace and profile the kernel and all applications on the system simultaneously to gain a system-wide view of what's going on ~~in the system~~.		Don't let the fact that it's part of the kernel fool you into thinking that it's only for tracing and profiling the kernel - you can indeed use it to trace and profile just the kernel , but you can also use it to profile specific applications separately (with or without kernel context), and you can also use it to trace and profile the kernel and all applications on the system simultaneously to gain a system-wide view of what's going on.

	In many ways, it aims to be a superset of all the tracing and profiling tools available in Linux today, including all the other tools covered in this HOWTO. The past couple of years have seen perf subsume a lot of the functionality of those other tools, and at the same time those other tools have removed large portions of their previous functionality and replaced it with calls to the equivalent functionality now implemented by the perf subsystem. Extrapolation suggests that at some point those other tools will simply become completely redundant and go away; until then, we'll cover those other tools in these pages and in many cases show how the same things can be accomplished in perf and the other tools when it seems useful to do so.		In many ways, it aims to be a superset of all the tracing and profiling tools available in Linux today, including all the other tools covered in this HOWTO. The past couple of years have seen perf subsume a lot of the functionality of those other tools, and at the same time those other tools have removed large portions of their previous functionality and replaced it with calls to the equivalent functionality now implemented by the perf subsystem. Extrapolation suggests that at some point those other tools will simply become completely redundant and go away; until then, we'll cover those other tools in these pages and in many cases show how the same things can be accomplished in perf and the other tools when it seems useful to do so.