1 perf-script-python(1) 1 perf-script-python(1)
2 ==================== 2 ====================
3 3
4 NAME 4 NAME
5 ---- 5 ----
6 perf-script-python - Process trace data with a 6 perf-script-python - Process trace data with a Python script
7 7
8 SYNOPSIS 8 SYNOPSIS
9 -------- 9 --------
10 [verse] 10 [verse]
11 'perf script' [-s [Python]:script[.py] ] 11 'perf script' [-s [Python]:script[.py] ]
12 12
13 DESCRIPTION 13 DESCRIPTION
14 ----------- 14 -----------
15 15
16 This perf script option is used to process per 16 This perf script option is used to process perf script data using perf's
17 built-in Python interpreter. It reads and pro 17 built-in Python interpreter. It reads and processes the input file and
18 displays the results of the trace analysis imp 18 displays the results of the trace analysis implemented in the given
19 Python script, if any. 19 Python script, if any.
20 20
21 A QUICK EXAMPLE 21 A QUICK EXAMPLE
22 --------------- 22 ---------------
23 23
24 This section shows the process, start to finis 24 This section shows the process, start to finish, of creating a working
25 Python script that aggregates and extracts use 25 Python script that aggregates and extracts useful information from a
26 raw perf script stream. You can avoid reading 26 raw perf script stream. You can avoid reading the rest of this
27 document if an example is enough for you; the 27 document if an example is enough for you; the rest of the document
28 provides more details on each step and lists t 28 provides more details on each step and lists the library functions
29 available to script writers. 29 available to script writers.
30 30
31 This example actually details the steps that w 31 This example actually details the steps that were used to create the
32 'syscall-counts' script you see when you list 32 'syscall-counts' script you see when you list the available perf script
33 scripts via 'perf script -l'. As such, this s 33 scripts via 'perf script -l'. As such, this script also shows how to
34 integrate your script into the list of general 34 integrate your script into the list of general-purpose 'perf script'
35 scripts listed by that command. 35 scripts listed by that command.
36 36
37 The syscall-counts script is a simple script, 37 The syscall-counts script is a simple script, but demonstrates all the
38 basic ideas necessary to create a useful scrip 38 basic ideas necessary to create a useful script. Here's an example
39 of its output (syscall names are not yet suppo 39 of its output (syscall names are not yet supported, they will appear
40 as numbers): 40 as numbers):
41 41
42 ---- 42 ----
43 syscall events: 43 syscall events:
44 44
45 event 45 event count
46 ---------------------------------------- ---- 46 ---------------------------------------- -----------
47 sys_write 47 sys_write 455067
48 sys_getdents 48 sys_getdents 4072
49 sys_close 49 sys_close 3037
50 sys_swapoff 50 sys_swapoff 1769
51 sys_read 51 sys_read 923
52 sys_sched_setparam 52 sys_sched_setparam 826
53 sys_open 53 sys_open 331
54 sys_newfstat 54 sys_newfstat 326
55 sys_mmap 55 sys_mmap 217
56 sys_munmap 56 sys_munmap 216
57 sys_futex 57 sys_futex 141
58 sys_select 58 sys_select 102
59 sys_poll 59 sys_poll 84
60 sys_setitimer 60 sys_setitimer 12
61 sys_writev 61 sys_writev 8
62 15 62 15 8
63 sys_lseek 63 sys_lseek 7
64 sys_rt_sigprocmask 64 sys_rt_sigprocmask 6
65 sys_wait4 65 sys_wait4 3
66 sys_ioctl 66 sys_ioctl 3
67 sys_set_robust_list 67 sys_set_robust_list 1
68 sys_exit 68 sys_exit 1
69 56 69 56 1
70 sys_access 70 sys_access 1
71 ---- 71 ----
72 72
73 Basically our task is to keep a per-syscall ta 73 Basically our task is to keep a per-syscall tally that gets updated
74 every time a system call occurs in the system. 74 every time a system call occurs in the system. Our script will do
75 that, but first we need to record the data tha 75 that, but first we need to record the data that will be processed by
76 that script. Theoretically, there are a coupl 76 that script. Theoretically, there are a couple of ways we could do
77 that: 77 that:
78 78
79 - we could enable every event under the tracin 79 - we could enable every event under the tracing/events/syscalls
80 directory, but this is over 600 syscalls, we 80 directory, but this is over 600 syscalls, well beyond the number
81 allowable by perf. These individual syscall 81 allowable by perf. These individual syscall events will however be
82 useful if we want to later use the guidance 82 useful if we want to later use the guidance we get from the
83 general-purpose scripts to drill down and ge 83 general-purpose scripts to drill down and get more detail about
84 individual syscalls of interest. 84 individual syscalls of interest.
85 85
86 - we can enable the sys_enter and/or sys_exit 86 - we can enable the sys_enter and/or sys_exit syscalls found under
87 tracing/events/raw_syscalls. These are call 87 tracing/events/raw_syscalls. These are called for all syscalls; the
88 'id' field can be used to distinguish betwee 88 'id' field can be used to distinguish between individual syscall
89 numbers. 89 numbers.
90 90
91 For this script, we only need to know that a s 91 For this script, we only need to know that a syscall was entered; we
92 don't care how it exited, so we'll use 'perf r 92 don't care how it exited, so we'll use 'perf record' to record only
93 the sys_enter events: 93 the sys_enter events:
94 94
95 ---- 95 ----
96 # perf record -a -e raw_syscalls:sys_enter 96 # perf record -a -e raw_syscalls:sys_enter
97 97
98 ^C[ perf record: Woken up 1 times to write dat 98 ^C[ perf record: Woken up 1 times to write data ]
99 [ perf record: Captured and wrote 56.545 MB pe 99 [ perf record: Captured and wrote 56.545 MB perf.data (~2470503 samples) ]
100 ---- 100 ----
101 101
102 The options basically say to collect data for 102 The options basically say to collect data for every syscall event
103 system-wide and multiplex the per-cpu output i 103 system-wide and multiplex the per-cpu output into a single stream.
104 That single stream will be recorded in a file 104 That single stream will be recorded in a file in the current directory
105 called perf.data. 105 called perf.data.
106 106
107 Once we have a perf.data file containing our d 107 Once we have a perf.data file containing our data, we can use the -g
108 'perf script' option to generate a Python scri 108 'perf script' option to generate a Python script that will contain a
109 callback handler for each event type found in 109 callback handler for each event type found in the perf.data trace
110 stream (for more details, see the STARTER SCRI 110 stream (for more details, see the STARTER SCRIPTS section).
111 111
112 ---- 112 ----
113 # perf script -g python 113 # perf script -g python
114 generated Python script: perf-script.py 114 generated Python script: perf-script.py
115 115
116 The output file created also in the current di 116 The output file created also in the current directory is named
117 perf-script.py. Here's the file in its entire 117 perf-script.py. Here's the file in its entirety:
118 118
119 # perf script event handlers, generated by per 119 # perf script event handlers, generated by perf script -g python
120 # Licensed under the terms of the GNU GPL Lice 120 # Licensed under the terms of the GNU GPL License version 2
121 121
122 # The common_* event handler fields are the mo 122 # The common_* event handler fields are the most useful fields common to
123 # all events. They don't necessarily correspo 123 # all events. They don't necessarily correspond to the 'common_*' fields
124 # in the format files. Those fields not avail 124 # in the format files. Those fields not available as handler params can
125 # be retrieved using Python functions of the f 125 # be retrieved using Python functions of the form common_*(context).
126 # See the perf-script-python Documentation for 126 # See the perf-script-python Documentation for the list of available functions.
127 127
128 import os 128 import os
129 import sys 129 import sys
130 130
131 sys.path.append(os.environ['PERF_EXEC_PATH'] + 131 sys.path.append(os.environ['PERF_EXEC_PATH'] + \
132 '/scripts/python/Perf-Trace-Util/lib/P 132 '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
133 133
134 from perf_trace_context import * 134 from perf_trace_context import *
135 from Core import * 135 from Core import *
136 136
137 def trace_begin(): 137 def trace_begin():
138 print "in trace_begin" 138 print "in trace_begin"
139 139
140 def trace_end(): 140 def trace_end():
141 print "in trace_end" 141 print "in trace_end"
142 142
143 def raw_syscalls__sys_enter(event_name, contex 143 def raw_syscalls__sys_enter(event_name, context, common_cpu,
144 common_secs, common_nsecs, common_pid, 144 common_secs, common_nsecs, common_pid, common_comm,
145 id, args): 145 id, args):
146 print_header(event_name, commo 146 print_header(event_name, common_cpu, common_secs, common_nsecs,
147 common_pid, common_com 147 common_pid, common_comm)
148 148
149 print "id=%d, args=%s\n" % \ 149 print "id=%d, args=%s\n" % \
150 (id, args), 150 (id, args),
151 151
152 def trace_unhandled(event_name, context, event 152 def trace_unhandled(event_name, context, event_fields_dict):
153 print ' '.join(['%s=%s'%(k,str 153 print ' '.join(['%s=%s'%(k,str(v))for k,v in sorted(event_fields_dict.items())])
154 154
155 def print_header(event_name, cpu, secs, nsecs, 155 def print_header(event_name, cpu, secs, nsecs, pid, comm):
156 print "%-20s %5u %05u.%09u %8u %-20s " 156 print "%-20s %5u %05u.%09u %8u %-20s " % \
157 (event_name, cpu, secs, nsecs, pid, co 157 (event_name, cpu, secs, nsecs, pid, comm),
158 ---- 158 ----
159 159
160 At the top is a comment block followed by some 160 At the top is a comment block followed by some import statements and a
161 path append which every perf script script sho 161 path append which every perf script script should include.
162 162
163 Following that are a couple generated function 163 Following that are a couple generated functions, trace_begin() and
164 trace_end(), which are called at the beginning 164 trace_end(), which are called at the beginning and the end of the
165 script respectively (for more details, see the 165 script respectively (for more details, see the SCRIPT_LAYOUT section
166 below). 166 below).
167 167
168 Following those are the 'event handler' functi 168 Following those are the 'event handler' functions generated one for
169 every event in the 'perf record' output. The 169 every event in the 'perf record' output. The handler functions take
170 the form subsystem\__event_name, and contain n 170 the form subsystem\__event_name, and contain named parameters, one for
171 each field in the event; in this case, there's 171 each field in the event; in this case, there's only one event,
172 raw_syscalls__sys_enter(). (see the EVENT HAN 172 raw_syscalls__sys_enter(). (see the EVENT HANDLERS section below for
173 more info on event handlers). 173 more info on event handlers).
174 174
175 The final couple of functions are, like the be 175 The final couple of functions are, like the begin and end functions,
176 generated for every script. The first, trace_ 176 generated for every script. The first, trace_unhandled(), is called
177 every time the script finds an event in the pe 177 every time the script finds an event in the perf.data file that
178 doesn't correspond to any event handler in the 178 doesn't correspond to any event handler in the script. This could
179 mean either that the record step recorded even 179 mean either that the record step recorded event types that it wasn't
180 really interested in, or the script was run ag 180 really interested in, or the script was run against a trace file that
181 doesn't correspond to the script. 181 doesn't correspond to the script.
182 182
183 The script generated by -g option simply print 183 The script generated by -g option simply prints a line for each
184 event found in the trace stream i.e. it basica 184 event found in the trace stream i.e. it basically just dumps the event
185 and its parameter values to stdout. The print 185 and its parameter values to stdout. The print_header() function is
186 simply a utility function used for that purpos 186 simply a utility function used for that purpose. Let's rename the
187 script and run it to see the default output: 187 script and run it to see the default output:
188 188
189 ---- 189 ----
190 # mv perf-script.py syscall-counts.py 190 # mv perf-script.py syscall-counts.py
191 # perf script -s syscall-counts.py 191 # perf script -s syscall-counts.py
192 192
193 raw_syscalls__sys_enter 1 00840.847582083 193 raw_syscalls__sys_enter 1 00840.847582083 7506 perf id=1, args=
194 raw_syscalls__sys_enter 1 00840.847595764 194 raw_syscalls__sys_enter 1 00840.847595764 7506 perf id=1, args=
195 raw_syscalls__sys_enter 1 00840.847620860 195 raw_syscalls__sys_enter 1 00840.847620860 7506 perf id=1, args=
196 raw_syscalls__sys_enter 1 00840.847710478 196 raw_syscalls__sys_enter 1 00840.847710478 6533 npviewer.bin id=78, args=
197 raw_syscalls__sys_enter 1 00840.847719204 197 raw_syscalls__sys_enter 1 00840.847719204 6533 npviewer.bin id=142, args=
198 raw_syscalls__sys_enter 1 00840.847755445 198 raw_syscalls__sys_enter 1 00840.847755445 6533 npviewer.bin id=3, args=
199 raw_syscalls__sys_enter 1 00840.847775601 199 raw_syscalls__sys_enter 1 00840.847775601 6533 npviewer.bin id=3, args=
200 raw_syscalls__sys_enter 1 00840.847781820 200 raw_syscalls__sys_enter 1 00840.847781820 6533 npviewer.bin id=3, args=
201 . 201 .
202 . 202 .
203 . 203 .
204 ---- 204 ----
205 205
206 Of course, for this script, we're not interest 206 Of course, for this script, we're not interested in printing every
207 trace event, but rather aggregating it in a us 207 trace event, but rather aggregating it in a useful way. So we'll get
208 rid of everything to do with printing as well 208 rid of everything to do with printing as well as the trace_begin() and
209 trace_unhandled() functions, which we won't be 209 trace_unhandled() functions, which we won't be using. That leaves us
210 with this minimalistic skeleton: 210 with this minimalistic skeleton:
211 211
212 ---- 212 ----
213 import os 213 import os
214 import sys 214 import sys
215 215
216 sys.path.append(os.environ['PERF_EXEC_PATH'] + 216 sys.path.append(os.environ['PERF_EXEC_PATH'] + \
217 '/scripts/python/Perf-Trace-Util/lib/P 217 '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
218 218
219 from perf_trace_context import * 219 from perf_trace_context import *
220 from Core import * 220 from Core import *
221 221
222 def trace_end(): 222 def trace_end():
223 print "in trace_end" 223 print "in trace_end"
224 224
225 def raw_syscalls__sys_enter(event_name, contex 225 def raw_syscalls__sys_enter(event_name, context, common_cpu,
226 common_secs, common_nsecs, common_pid, 226 common_secs, common_nsecs, common_pid, common_comm,
227 id, args): 227 id, args):
228 ---- 228 ----
229 229
230 In trace_end(), we'll simply print the results 230 In trace_end(), we'll simply print the results, but first we need to
231 generate some results to print. To do that we 231 generate some results to print. To do that we need to have our
232 sys_enter() handler do the necessary tallying 232 sys_enter() handler do the necessary tallying until all events have
233 been counted. A hash table indexed by syscall 233 been counted. A hash table indexed by syscall id is a good way to
234 store that information; every time the sys_ent 234 store that information; every time the sys_enter() handler is called,
235 we simply increment a count associated with th 235 we simply increment a count associated with that hash entry indexed by
236 that syscall id: 236 that syscall id:
237 237
238 ---- 238 ----
239 syscalls = autodict() 239 syscalls = autodict()
240 240
241 try: 241 try:
242 syscalls[id] += 1 242 syscalls[id] += 1
243 except TypeError: 243 except TypeError:
244 syscalls[id] = 1 244 syscalls[id] = 1
245 ---- 245 ----
246 246
247 The syscalls 'autodict' object is a special ki 247 The syscalls 'autodict' object is a special kind of Python dictionary
248 (implemented in Core.py) that implements Perl' 248 (implemented in Core.py) that implements Perl's 'autovivifying' hashes
249 in Python i.e. with autovivifying hashes, you 249 in Python i.e. with autovivifying hashes, you can assign nested hash
250 values without having to go to the trouble of 250 values without having to go to the trouble of creating intermediate
251 levels if they don't exist e.g syscalls[comm][ 251 levels if they don't exist e.g syscalls[comm][pid][id] = 1 will create
252 the intermediate hash levels and finally assig 252 the intermediate hash levels and finally assign the value 1 to the
253 hash entry for 'id' (because the value being a 253 hash entry for 'id' (because the value being assigned isn't a hash
254 object itself, the initial value is assigned i 254 object itself, the initial value is assigned in the TypeError
255 exception. Well, there may be a better way to 255 exception. Well, there may be a better way to do this in Python but
256 that's what works for now). 256 that's what works for now).
257 257
258 Putting that code into the raw_syscalls__sys_e 258 Putting that code into the raw_syscalls__sys_enter() handler, we
259 effectively end up with a single-level diction 259 effectively end up with a single-level dictionary keyed on syscall id
260 and having the counts we've tallied as values. 260 and having the counts we've tallied as values.
261 261
262 The print_syscall_totals() function iterates o 262 The print_syscall_totals() function iterates over the entries in the
263 dictionary and displays a line for each entry 263 dictionary and displays a line for each entry containing the syscall
264 name (the dictionary keys contain the syscall 264 name (the dictionary keys contain the syscall ids, which are passed to
265 the Util function syscall_name(), which transl 265 the Util function syscall_name(), which translates the raw syscall
266 numbers to the corresponding syscall name stri 266 numbers to the corresponding syscall name strings). The output is
267 displayed after all the events in the trace ha 267 displayed after all the events in the trace have been processed, by
268 calling the print_syscall_totals() function fr 268 calling the print_syscall_totals() function from the trace_end()
269 handler called at the end of script processing 269 handler called at the end of script processing.
270 270
271 The final script producing the output shown ab 271 The final script producing the output shown above is shown in its
272 entirety below (syscall_name() helper is not y 272 entirety below (syscall_name() helper is not yet available, you can
273 only deal with id's for now): 273 only deal with id's for now):
274 274
275 ---- 275 ----
276 import os 276 import os
277 import sys 277 import sys
278 278
279 sys.path.append(os.environ['PERF_EXEC_PATH'] + 279 sys.path.append(os.environ['PERF_EXEC_PATH'] + \
280 '/scripts/python/Perf-Trace-Util/lib/P 280 '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
281 281
282 from perf_trace_context import * 282 from perf_trace_context import *
283 from Core import * 283 from Core import *
284 from Util import * 284 from Util import *
285 285
286 syscalls = autodict() 286 syscalls = autodict()
287 287
288 def trace_end(): 288 def trace_end():
289 print_syscall_totals() 289 print_syscall_totals()
290 290
291 def raw_syscalls__sys_enter(event_name, contex 291 def raw_syscalls__sys_enter(event_name, context, common_cpu,
292 common_secs, common_nsecs, common_pid, 292 common_secs, common_nsecs, common_pid, common_comm,
293 id, args): 293 id, args):
294 try: 294 try:
295 syscalls[id] += 1 295 syscalls[id] += 1
296 except TypeError: 296 except TypeError:
297 syscalls[id] = 1 297 syscalls[id] = 1
298 298
299 def print_syscall_totals(): 299 def print_syscall_totals():
300 if for_comm is not None: 300 if for_comm is not None:
301 print "\nsyscall events for %s:\n\ 301 print "\nsyscall events for %s:\n\n" % (for_comm),
302 else: 302 else:
303 print "\nsyscall events:\n\n", 303 print "\nsyscall events:\n\n",
304 304
305 print "%-40s %10s\n" % ("event", "count") 305 print "%-40s %10s\n" % ("event", "count"),
306 print "%-40s %10s\n" % ("---------------- 306 print "%-40s %10s\n" % ("----------------------------------------", \
307 "-----------" 307 "-----------"),
308 308
309 for id, val in sorted(syscalls.iteritems() 309 for id, val in sorted(syscalls.iteritems(), key = lambda(k, v): (v, k), \
310 reverse = Tr 310 reverse = True):
311 print "%-40s %10d\n" % (syscall_n 311 print "%-40s %10d\n" % (syscall_name(id), val),
312 ---- 312 ----
313 313
314 The script can be run just as before: 314 The script can be run just as before:
315 315
316 # perf script -s syscall-counts.py 316 # perf script -s syscall-counts.py
317 317
318 So those are the essential steps in writing an 318 So those are the essential steps in writing and running a script. The
319 process can be generalized to any tracepoint o 319 process can be generalized to any tracepoint or set of tracepoints
320 you're interested in - basically find the trac 320 you're interested in - basically find the tracepoint(s) you're
321 interested in by looking at the list of availa 321 interested in by looking at the list of available events shown by
322 'perf list' and/or look in /sys/kernel/tracing 322 'perf list' and/or look in /sys/kernel/tracing/events/ for
323 detailed event and field info, record the corr 323 detailed event and field info, record the corresponding trace data
324 using 'perf record', passing it the list of in 324 using 'perf record', passing it the list of interesting events,
325 generate a skeleton script using 'perf script 325 generate a skeleton script using 'perf script -g python' and modify the
326 code to aggregate and display it for your part 326 code to aggregate and display it for your particular needs.
327 327
328 After you've done that you may end up with a g 328 After you've done that you may end up with a general-purpose script
329 that you want to keep around and have availabl 329 that you want to keep around and have available for future use. By
330 writing a couple of very simple shell scripts 330 writing a couple of very simple shell scripts and putting them in the
331 right place, you can have your script listed a 331 right place, you can have your script listed alongside the other
332 scripts listed by the 'perf script -l' command 332 scripts listed by the 'perf script -l' command e.g.:
333 333
334 ---- 334 ----
335 # perf script -l 335 # perf script -l
336 List of available trace scripts: 336 List of available trace scripts:
337 wakeup-latency system- 337 wakeup-latency system-wide min/max/avg wakeup latency
338 rw-by-file <comm> r/w act 338 rw-by-file <comm> r/w activity for a program, by file
339 rw-by-pid system- 339 rw-by-pid system-wide r/w activity
340 ---- 340 ----
341 341
342 A nice side effect of doing this is that you a 342 A nice side effect of doing this is that you also then capture the
343 probably lengthy 'perf record' command needed 343 probably lengthy 'perf record' command needed to record the events for
344 the script. 344 the script.
345 345
346 To have the script appear as a 'built-in' scri 346 To have the script appear as a 'built-in' script, you write two simple
347 scripts, one for recording and one for 'report 347 scripts, one for recording and one for 'reporting'.
348 348
349 The 'record' script is a shell script with the 349 The 'record' script is a shell script with the same base name as your
350 script, but with -record appended. The shell 350 script, but with -record appended. The shell script should be put
351 into the perf/scripts/python/bin directory in 351 into the perf/scripts/python/bin directory in the kernel source tree.
352 In that script, you write the 'perf record' co 352 In that script, you write the 'perf record' command-line needed for
353 your script: 353 your script:
354 354
355 ---- 355 ----
356 # cat kernel-source/tools/perf/scripts/python/ 356 # cat kernel-source/tools/perf/scripts/python/bin/syscall-counts-record
357 357
358 #!/bin/bash 358 #!/bin/bash
359 perf record -a -e raw_syscalls:sys_enter 359 perf record -a -e raw_syscalls:sys_enter
360 ---- 360 ----
361 361
362 The 'report' script is also a shell script wit 362 The 'report' script is also a shell script with the same base name as
363 your script, but with -report appended. It sh 363 your script, but with -report appended. It should also be located in
364 the perf/scripts/python/bin directory. In tha 364 the perf/scripts/python/bin directory. In that script, you write the
365 'perf script -s' command-line needed for runni 365 'perf script -s' command-line needed for running your script:
366 366
367 ---- 367 ----
368 # cat kernel-source/tools/perf/scripts/python/ 368 # cat kernel-source/tools/perf/scripts/python/bin/syscall-counts-report
369 369
370 #!/bin/bash 370 #!/bin/bash
371 # description: system-wide syscall counts 371 # description: system-wide syscall counts
372 perf script -s ~/libexec/perf-core/scripts/pyt 372 perf script -s ~/libexec/perf-core/scripts/python/syscall-counts.py
373 ---- 373 ----
374 374
375 Note that the location of the Python script gi 375 Note that the location of the Python script given in the shell script
376 is in the libexec/perf-core/scripts/python dir 376 is in the libexec/perf-core/scripts/python directory - this is where
377 the script will be copied by 'make install' wh 377 the script will be copied by 'make install' when you install perf.
378 For the installation to install your script th 378 For the installation to install your script there, your script needs
379 to be located in the perf/scripts/python direc 379 to be located in the perf/scripts/python directory in the kernel
380 source tree: 380 source tree:
381 381
382 ---- 382 ----
383 # ls -al kernel-source/tools/perf/scripts/pyth 383 # ls -al kernel-source/tools/perf/scripts/python
384 total 32 384 total 32
385 drwxr-xr-x 4 trz trz 4096 2010-01-26 22:30 . 385 drwxr-xr-x 4 trz trz 4096 2010-01-26 22:30 .
386 drwxr-xr-x 4 trz trz 4096 2010-01-26 22:29 .. 386 drwxr-xr-x 4 trz trz 4096 2010-01-26 22:29 ..
387 drwxr-xr-x 2 trz trz 4096 2010-01-26 22:29 bin 387 drwxr-xr-x 2 trz trz 4096 2010-01-26 22:29 bin
388 -rw-r--r-- 1 trz trz 2548 2010-01-26 22:29 che 388 -rw-r--r-- 1 trz trz 2548 2010-01-26 22:29 check-perf-script.py
389 drwxr-xr-x 3 trz trz 4096 2010-01-26 22:49 Per 389 drwxr-xr-x 3 trz trz 4096 2010-01-26 22:49 Perf-Trace-Util
390 -rw-r--r-- 1 trz trz 1462 2010-01-26 22:30 sys 390 -rw-r--r-- 1 trz trz 1462 2010-01-26 22:30 syscall-counts.py
391 ---- 391 ----
392 392
393 Once you've done that (don't forget to do a ne 393 Once you've done that (don't forget to do a new 'make install',
394 otherwise your script won't show up at run-tim 394 otherwise your script won't show up at run-time), 'perf script -l'
395 should show a new entry for your script: 395 should show a new entry for your script:
396 396
397 ---- 397 ----
398 # perf script -l 398 # perf script -l
399 List of available trace scripts: 399 List of available trace scripts:
400 wakeup-latency system- 400 wakeup-latency system-wide min/max/avg wakeup latency
401 rw-by-file <comm> r/w act 401 rw-by-file <comm> r/w activity for a program, by file
402 rw-by-pid system- 402 rw-by-pid system-wide r/w activity
403 syscall-counts system- 403 syscall-counts system-wide syscall counts
404 ---- 404 ----
405 405
406 You can now perform the record step via 'perf 406 You can now perform the record step via 'perf script record':
407 407
408 # perf script record syscall-counts 408 # perf script record syscall-counts
409 409
410 and display the output using 'perf script repo 410 and display the output using 'perf script report':
411 411
412 # perf script report syscall-counts 412 # perf script report syscall-counts
413 413
414 STARTER SCRIPTS 414 STARTER SCRIPTS
415 --------------- 415 ---------------
416 416
417 You can quickly get started writing a script f 417 You can quickly get started writing a script for a particular set of
418 trace data by generating a skeleton script usi 418 trace data by generating a skeleton script using 'perf script -g
419 python' in the same directory as an existing p 419 python' in the same directory as an existing perf.data trace file.
420 That will generate a starter script containing 420 That will generate a starter script containing a handler for each of
421 the event types in the trace file; it simply p 421 the event types in the trace file; it simply prints every available
422 field for each event in the trace file. 422 field for each event in the trace file.
423 423
424 You can also look at the existing scripts in 424 You can also look at the existing scripts in
425 ~/libexec/perf-core/scripts/python for typical 425 ~/libexec/perf-core/scripts/python for typical examples showing how to
426 do basic things like aggregate event data, pri 426 do basic things like aggregate event data, print results, etc. Also,
427 the check-perf-script.py script, while not int 427 the check-perf-script.py script, while not interesting for its results,
428 attempts to exercise all of the main scripting 428 attempts to exercise all of the main scripting features.
429 429
430 EVENT HANDLERS 430 EVENT HANDLERS
431 -------------- 431 --------------
432 432
433 When perf script is invoked using a trace scri 433 When perf script is invoked using a trace script, a user-defined
434 'handler function' is called for each event in 434 'handler function' is called for each event in the trace. If there's
435 no handler function defined for a given event 435 no handler function defined for a given event type, the event is
436 ignored (or passed to a 'trace_unhandled' func 436 ignored (or passed to a 'trace_unhandled' function, see below) and the
437 next event is processed. 437 next event is processed.
438 438
439 Most of the event's field values are passed as 439 Most of the event's field values are passed as arguments to the
440 handler function; some of the less common ones 440 handler function; some of the less common ones aren't - those are
441 available as calls back into the perf executab 441 available as calls back into the perf executable (see below).
442 442
443 As an example, the following perf record comma 443 As an example, the following perf record command can be used to record
444 all sched_wakeup events in the system: 444 all sched_wakeup events in the system:
445 445
446 # perf record -a -e sched:sched_wakeup 446 # perf record -a -e sched:sched_wakeup
447 447
448 Traces meant to be processed using a script sh 448 Traces meant to be processed using a script should be recorded with
449 the above option: -a to enable system-wide col 449 the above option: -a to enable system-wide collection.
450 450
451 The format file for the sched_wakeup event def 451 The format file for the sched_wakeup event defines the following fields
452 (see /sys/kernel/tracing/events/sched/sched_wa 452 (see /sys/kernel/tracing/events/sched/sched_wakeup/format):
453 453
454 ---- 454 ----
455 format: 455 format:
456 field:unsigned short common_type; 456 field:unsigned short common_type;
457 field:unsigned char common_flags; 457 field:unsigned char common_flags;
458 field:unsigned char common_preempt_cou 458 field:unsigned char common_preempt_count;
459 field:int common_pid; 459 field:int common_pid;
460 460
461 field:char comm[TASK_COMM_LEN]; 461 field:char comm[TASK_COMM_LEN];
462 field:pid_t pid; 462 field:pid_t pid;
463 field:int prio; 463 field:int prio;
464 field:int success; 464 field:int success;
465 field:int target_cpu; 465 field:int target_cpu;
466 ---- 466 ----
467 467
468 The handler function for this event would be d 468 The handler function for this event would be defined as:
469 469
470 ---- 470 ----
471 def sched__sched_wakeup(event_name, context, c 471 def sched__sched_wakeup(event_name, context, common_cpu, common_secs,
472 common_nsecs, common_pid, common_comm, 472 common_nsecs, common_pid, common_comm,
473 comm, pid, prio, success, target_cpu): 473 comm, pid, prio, success, target_cpu):
474 pass 474 pass
475 ---- 475 ----
476 476
477 The handler function takes the form subsystem_ 477 The handler function takes the form subsystem__event_name.
478 478
479 The common_* arguments in the handler's argume 479 The common_* arguments in the handler's argument list are the set of
480 arguments passed to all event handlers; some o 480 arguments passed to all event handlers; some of the fields correspond
481 to the common_* fields in the format file, but 481 to the common_* fields in the format file, but some are synthesized,
482 and some of the common_* fields aren't common 482 and some of the common_* fields aren't common enough to to be passed
483 to every event as arguments but are available 483 to every event as arguments but are available as library functions.
484 484
485 Here's a brief description of each of the inva 485 Here's a brief description of each of the invariant event args:
486 486
487 event_name the name of the ev 487 event_name the name of the event as text
488 context an opaque 'cookie' 488 context an opaque 'cookie' used in calls back into perf
489 common_cpu the cpu the event 489 common_cpu the cpu the event occurred on
490 common_secs the secs portion o 490 common_secs the secs portion of the event timestamp
491 common_nsecs the nsecs portion 491 common_nsecs the nsecs portion of the event timestamp
492 common_pid the pid of the cur 492 common_pid the pid of the current task
493 common_comm the name of the cu 493 common_comm the name of the current process
494 494
495 All of the remaining fields in the event's for 495 All of the remaining fields in the event's format file have
496 counterparts as handler function arguments of 496 counterparts as handler function arguments of the same name, as can be
497 seen in the example above. 497 seen in the example above.
498 498
499 The above provides the basics needed to direct 499 The above provides the basics needed to directly access every field of
500 every event in a trace, which covers 90% of wh 500 every event in a trace, which covers 90% of what you need to know to
501 write a useful trace script. The sections bel 501 write a useful trace script. The sections below cover the rest.
502 502
503 SCRIPT LAYOUT 503 SCRIPT LAYOUT
504 ------------- 504 -------------
505 505
506 Every perf script Python script should start b 506 Every perf script Python script should start by setting up a Python
507 module search path and 'import'ing a few suppo 507 module search path and 'import'ing a few support modules (see module
508 descriptions below): 508 descriptions below):
509 509
510 ---- 510 ----
511 import os 511 import os
512 import sys 512 import sys
513 513
514 sys.path.append(os.environ['PERF_EXEC_PATH'] 514 sys.path.append(os.environ['PERF_EXEC_PATH'] + \
515 '/scripts/python/Perf-Trace-Util 515 '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
516 516
517 from perf_trace_context import * 517 from perf_trace_context import *
518 from Core import * 518 from Core import *
519 ---- 519 ----
520 520
521 The rest of the script can contain handler fun 521 The rest of the script can contain handler functions and support
522 functions in any order. 522 functions in any order.
523 523
524 Aside from the event handler functions discuss 524 Aside from the event handler functions discussed above, every script
525 can implement a set of optional functions: 525 can implement a set of optional functions:
526 526
527 *trace_begin*, if defined, is called before an 527 *trace_begin*, if defined, is called before any event is processed and
528 gives scripts a chance to do setup tasks: 528 gives scripts a chance to do setup tasks:
529 529
530 ---- 530 ----
531 def trace_begin(): 531 def trace_begin():
532 pass 532 pass
533 ---- 533 ----
534 534
535 *trace_end*, if defined, is called after all e 535 *trace_end*, if defined, is called after all events have been
536 processed and gives scripts a chance to do en 536 processed and gives scripts a chance to do end-of-script tasks, such
537 as display results: 537 as display results:
538 538
539 ---- 539 ----
540 def trace_end(): 540 def trace_end():
541 pass 541 pass
542 ---- 542 ----
543 543
544 *trace_unhandled*, if defined, is called after 544 *trace_unhandled*, if defined, is called after for any event that
545 doesn't have a handler explicitly defined for 545 doesn't have a handler explicitly defined for it. The standard set
546 of common arguments are passed into it: 546 of common arguments are passed into it:
547 547
548 ---- 548 ----
549 def trace_unhandled(event_name, context, event 549 def trace_unhandled(event_name, context, event_fields_dict):
550 pass 550 pass
551 ---- 551 ----
552 552
553 *process_event*, if defined, is called for any 553 *process_event*, if defined, is called for any non-tracepoint event
554 554
555 ---- 555 ----
556 def process_event(param_dict): 556 def process_event(param_dict):
557 pass 557 pass
558 ---- 558 ----
559 559
560 *context_switch*, if defined, is called for an 560 *context_switch*, if defined, is called for any context switch
561 561
562 ---- 562 ----
563 def context_switch(ts, cpu, pid, tid, np_pid, 563 def context_switch(ts, cpu, pid, tid, np_pid, np_tid, machine_pid, out, out_preempt, *x):
564 pass 564 pass
565 ---- 565 ----
566 566
567 *auxtrace_error*, if defined, is called for an 567 *auxtrace_error*, if defined, is called for any AUX area tracing error
568 568
569 ---- 569 ----
570 def auxtrace_error(typ, code, cpu, pid, tid, i 570 def auxtrace_error(typ, code, cpu, pid, tid, ip, ts, msg, cpumode, *x):
571 pass 571 pass
572 ---- 572 ----
573 573
574 The remaining sections provide descriptions of 574 The remaining sections provide descriptions of each of the available
575 built-in perf script Python modules and their 575 built-in perf script Python modules and their associated functions.
576 576
577 AVAILABLE MODULES AND FUNCTIONS 577 AVAILABLE MODULES AND FUNCTIONS
578 ------------------------------- 578 -------------------------------
579 579
580 The following sections describe the functions 580 The following sections describe the functions and variables available
581 via the various perf script Python modules. T 581 via the various perf script Python modules. To use the functions and
582 variables from the given module, add the corre 582 variables from the given module, add the corresponding 'from XXXX
583 import' line to your perf script script. 583 import' line to your perf script script.
584 584
585 Core.py Module 585 Core.py Module
586 ~~~~~~~~~~~~~~ 586 ~~~~~~~~~~~~~~
587 587
588 These functions provide some essential functio 588 These functions provide some essential functions to user scripts.
589 589
590 The *flag_str* and *symbol_str* functions prov 590 The *flag_str* and *symbol_str* functions provide human-readable
591 strings for flag and symbolic fields. These c 591 strings for flag and symbolic fields. These correspond to the strings
592 and values parsed from the 'print fmt' fields 592 and values parsed from the 'print fmt' fields of the event format
593 files: 593 files:
594 594
595 flag_str(event_name, field_name, field_value 595 flag_str(event_name, field_name, field_value) - returns the string representation corresponding to field_value for the flag field field_name of event event_name
596 symbol_str(event_name, field_name, field_val 596 symbol_str(event_name, field_name, field_value) - returns the string representation corresponding to field_value for the symbolic field field_name of event event_name
597 597
598 The *autodict* function returns a special kind 598 The *autodict* function returns a special kind of Python
599 dictionary that implements Perl's 'autovivifyi 599 dictionary that implements Perl's 'autovivifying' hashes in Python
600 i.e. with autovivifying hashes, you can assign 600 i.e. with autovivifying hashes, you can assign nested hash values
601 without having to go to the trouble of creatin 601 without having to go to the trouble of creating intermediate levels if
602 they don't exist. 602 they don't exist.
603 603
604 autodict() - returns an autovivifying dictio 604 autodict() - returns an autovivifying dictionary instance
605 605
606 606
607 perf_trace_context Module 607 perf_trace_context Module
608 ~~~~~~~~~~~~~~~~~~~~~~~~~ 608 ~~~~~~~~~~~~~~~~~~~~~~~~~
609 609
610 Some of the 'common' fields in the event forma 610 Some of the 'common' fields in the event format file aren't all that
611 common, but need to be made accessible to user 611 common, but need to be made accessible to user scripts nonetheless.
612 612
613 perf_trace_context defines a set of functions 613 perf_trace_context defines a set of functions that can be used to
614 access this data in the context of the current 614 access this data in the context of the current event. Each of these
615 functions expects a context variable, which is 615 functions expects a context variable, which is the same as the
616 context variable passed into every tracepoint 616 context variable passed into every tracepoint event handler as the second
617 argument. For non-tracepoint events, the conte 617 argument. For non-tracepoint events, the context variable is also present
618 as perf_trace_context.perf_script_context . 618 as perf_trace_context.perf_script_context .
619 619
620 common_pc(context) - returns common_preempt c 620 common_pc(context) - returns common_preempt count for the current event
621 common_flags(context) - returns common_flags 621 common_flags(context) - returns common_flags for the current event
622 common_lock_depth(context) - returns common_l 622 common_lock_depth(context) - returns common_lock_depth for the current event
623 perf_sample_insn(context) - returns the machi 623 perf_sample_insn(context) - returns the machine code instruction
624 perf_set_itrace_options(context, itrace_optio 624 perf_set_itrace_options(context, itrace_options) - set --itrace options if they have not been set already
625 perf_sample_srcline(context) - returns source 625 perf_sample_srcline(context) - returns source_file_name, line_number
626 perf_sample_srccode(context) - returns source 626 perf_sample_srccode(context) - returns source_file_name, line_number, source_line
627 627
628 628
629 Util.py Module 629 Util.py Module
630 ~~~~~~~~~~~~~~ 630 ~~~~~~~~~~~~~~
631 631
632 Various utility functions for use with perf sc 632 Various utility functions for use with perf script:
633 633
634 nsecs(secs, nsecs) - returns total nsecs giv 634 nsecs(secs, nsecs) - returns total nsecs given secs/nsecs pair
635 nsecs_secs(nsecs) - returns whole secs porti 635 nsecs_secs(nsecs) - returns whole secs portion given nsecs
636 nsecs_nsecs(nsecs) - returns nsecs remainder 636 nsecs_nsecs(nsecs) - returns nsecs remainder given nsecs
637 nsecs_str(nsecs) - returns printable string 637 nsecs_str(nsecs) - returns printable string in the form secs.nsecs
638 avg(total, n) - returns average given a sum 638 avg(total, n) - returns average given a sum and a total number of values
639 639
640 SUPPORTED FIELDS 640 SUPPORTED FIELDS
641 ---------------- 641 ----------------
642 642
643 Currently supported fields: 643 Currently supported fields:
644 644
645 ev_name, comm, id, stream_id, pid, tid, cpu, i 645 ev_name, comm, id, stream_id, pid, tid, cpu, ip, time, period, phys_addr,
646 addr, symbol, symoff, dso, time_enabled, time_ 646 addr, symbol, symoff, dso, time_enabled, time_running, values, callchain,
647 brstack, brstacksym, datasrc, datasrc_decode, 647 brstack, brstacksym, datasrc, datasrc_decode, iregs, uregs,
648 weight, transaction, raw_buf, attr, cpumode. 648 weight, transaction, raw_buf, attr, cpumode.
649 649
650 Fields that may also be present: 650 Fields that may also be present:
651 651
652 flags - sample flags 652 flags - sample flags
653 flags_disp - sample flags display 653 flags_disp - sample flags display
654 insn_cnt - instruction count for determining 654 insn_cnt - instruction count for determining instructions-per-cycle (IPC)
655 cyc_cnt - cycle count for determining IPC 655 cyc_cnt - cycle count for determining IPC
656 addr_correlates_sym - addr can correlate to a 656 addr_correlates_sym - addr can correlate to a symbol
657 addr_dso - addr dso 657 addr_dso - addr dso
658 addr_symbol - addr symbol 658 addr_symbol - addr symbol
659 addr_symoff - addr symbol offset 659 addr_symoff - addr symbol offset
660 660
661 Some fields have sub items: 661 Some fields have sub items:
662 662
663 brstack: 663 brstack:
664 from, to, from_dsoname, to_dsoname, mispre 664 from, to, from_dsoname, to_dsoname, mispred,
665 predicted, in_tx, abort, cycles. 665 predicted, in_tx, abort, cycles.
666 666
667 brstacksym: 667 brstacksym:
668 items: from, to, pred, in_tx, abort (conve 668 items: from, to, pred, in_tx, abort (converted string)
669 669
670 For example, 670 For example,
671 We can use this code to print brstack "from", 671 We can use this code to print brstack "from", "to", "cycles".
672 672
673 if 'brstack' in dict: 673 if 'brstack' in dict:
674 for entry in dict['brstack']: 674 for entry in dict['brstack']:
675 print "from %s, to %s, cycles 675 print "from %s, to %s, cycles %s" % (entry["from"], entry["to"], entry["cycles"])
676 676
677 SEE ALSO 677 SEE ALSO
678 -------- 678 --------
679 linkperf:perf-script[1] 679 linkperf:perf-script[1]
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.