1 ================================== 1 ==================================
2 Using the Linux Kernel Tracepoints 2 Using the Linux Kernel Tracepoints
3 ================================== 3 ==================================
4 4
5 :Author: Mathieu Desnoyers 5 :Author: Mathieu Desnoyers
6 6
7 7
8 This document introduces Linux Kernel Tracepoi 8 This document introduces Linux Kernel Tracepoints and their use. It
9 provides examples of how to insert tracepoints 9 provides examples of how to insert tracepoints in the kernel and
10 connect probe functions to them and provides s 10 connect probe functions to them and provides some examples of probe
11 functions. 11 functions.
12 12
13 13
14 Purpose of tracepoints 14 Purpose of tracepoints
15 ---------------------- 15 ----------------------
16 A tracepoint placed in code provides a hook to 16 A tracepoint placed in code provides a hook to call a function (probe)
17 that you can provide at runtime. A tracepoint 17 that you can provide at runtime. A tracepoint can be "on" (a probe is
18 connected to it) or "off" (no probe is attache 18 connected to it) or "off" (no probe is attached). When a tracepoint is
19 "off" it has no effect, except for adding a ti 19 "off" it has no effect, except for adding a tiny time penalty
20 (checking a condition for a branch) and space 20 (checking a condition for a branch) and space penalty (adding a few
21 bytes for the function call at the end of the 21 bytes for the function call at the end of the instrumented function
22 and adds a data structure in a separate sectio 22 and adds a data structure in a separate section). When a tracepoint
23 is "on", the function you provide is called ea 23 is "on", the function you provide is called each time the tracepoint
24 is executed, in the execution context of the c 24 is executed, in the execution context of the caller. When the function
25 provided ends its execution, it returns to the 25 provided ends its execution, it returns to the caller (continuing from
26 the tracepoint site). 26 the tracepoint site).
27 27
28 You can put tracepoints at important locations 28 You can put tracepoints at important locations in the code. They are
29 lightweight hooks that can pass an arbitrary n 29 lightweight hooks that can pass an arbitrary number of parameters,
30 whose prototypes are described in a tracepoint !! 30 which prototypes are described in a tracepoint declaration placed in a
31 header file. 31 header file.
32 32
33 They can be used for tracing and performance a 33 They can be used for tracing and performance accounting.
34 34
35 35
36 Usage 36 Usage
37 ----- 37 -----
38 Two elements are required for tracepoints : 38 Two elements are required for tracepoints :
39 39
40 - A tracepoint definition, placed in a header 40 - A tracepoint definition, placed in a header file.
41 - The tracepoint statement, in C code. 41 - The tracepoint statement, in C code.
42 42
43 In order to use tracepoints, you should includ 43 In order to use tracepoints, you should include linux/tracepoint.h.
44 44
45 In include/trace/events/subsys.h:: 45 In include/trace/events/subsys.h::
46 46
47 #undef TRACE_SYSTEM 47 #undef TRACE_SYSTEM
48 #define TRACE_SYSTEM subsys 48 #define TRACE_SYSTEM subsys
49 49
50 #if !defined(_TRACE_SUBSYS_H) || defin 50 #if !defined(_TRACE_SUBSYS_H) || defined(TRACE_HEADER_MULTI_READ)
51 #define _TRACE_SUBSYS_H 51 #define _TRACE_SUBSYS_H
52 52
53 #include <linux/tracepoint.h> 53 #include <linux/tracepoint.h>
54 54
55 DECLARE_TRACE(subsys_eventname, 55 DECLARE_TRACE(subsys_eventname,
56 TP_PROTO(int firstarg, struct 56 TP_PROTO(int firstarg, struct task_struct *p),
57 TP_ARGS(firstarg, p)); 57 TP_ARGS(firstarg, p));
58 58
59 #endif /* _TRACE_SUBSYS_H */ 59 #endif /* _TRACE_SUBSYS_H */
60 60
61 /* This part must be outside protectio 61 /* This part must be outside protection */
62 #include <trace/define_trace.h> 62 #include <trace/define_trace.h>
63 63
64 In subsys/file.c (where the tracing statement 64 In subsys/file.c (where the tracing statement must be added)::
65 65
66 #include <trace/events/subsys.h> 66 #include <trace/events/subsys.h>
67 67
68 #define CREATE_TRACE_POINTS 68 #define CREATE_TRACE_POINTS
69 DEFINE_TRACE(subsys_eventname); 69 DEFINE_TRACE(subsys_eventname);
70 70
71 void somefct(void) 71 void somefct(void)
72 { 72 {
73 ... 73 ...
74 trace_subsys_eventname(arg, ta 74 trace_subsys_eventname(arg, task);
75 ... 75 ...
76 } 76 }
77 77
78 Where : 78 Where :
79 - subsys_eventname is an identifier unique t 79 - subsys_eventname is an identifier unique to your event
80 80
81 - subsys is the name of your subsystem. 81 - subsys is the name of your subsystem.
82 - eventname is the name of the event to tr 82 - eventname is the name of the event to trace.
83 83
84 - `TP_PROTO(int firstarg, struct task_struct 84 - `TP_PROTO(int firstarg, struct task_struct *p)` is the prototype of the
85 function called by this tracepoint. 85 function called by this tracepoint.
86 86
87 - `TP_ARGS(firstarg, p)` are the parameters 87 - `TP_ARGS(firstarg, p)` are the parameters names, same as found in the
88 prototype. 88 prototype.
89 89
90 - if you use the header in multiple source f 90 - if you use the header in multiple source files, `#define CREATE_TRACE_POINTS`
91 should appear only in one source file. 91 should appear only in one source file.
92 92
93 Connecting a function (probe) to a tracepoint 93 Connecting a function (probe) to a tracepoint is done by providing a
94 probe (function to call) for the specific trac 94 probe (function to call) for the specific tracepoint through
95 register_trace_subsys_eventname(). Removing a 95 register_trace_subsys_eventname(). Removing a probe is done through
96 unregister_trace_subsys_eventname(); it will r 96 unregister_trace_subsys_eventname(); it will remove the probe.
97 97
98 tracepoint_synchronize_unregister() must be ca 98 tracepoint_synchronize_unregister() must be called before the end of
99 the module exit function to make sure there is 99 the module exit function to make sure there is no caller left using
100 the probe. This, and the fact that preemption 100 the probe. This, and the fact that preemption is disabled around the
101 probe call, make sure that probe removal and m 101 probe call, make sure that probe removal and module unload are safe.
102 102
103 The tracepoint mechanism supports inserting mu 103 The tracepoint mechanism supports inserting multiple instances of the
104 same tracepoint, but a single definition must 104 same tracepoint, but a single definition must be made of a given
105 tracepoint name over all the kernel to make su 105 tracepoint name over all the kernel to make sure no type conflict will
106 occur. Name mangling of the tracepoints is don 106 occur. Name mangling of the tracepoints is done using the prototypes
107 to make sure typing is correct. Verification o 107 to make sure typing is correct. Verification of probe type correctness
108 is done at the registration site by the compil 108 is done at the registration site by the compiler. Tracepoints can be
109 put in inline functions, inlined static functi 109 put in inline functions, inlined static functions, and unrolled loops
110 as well as regular functions. 110 as well as regular functions.
111 111
112 The naming scheme "subsys_event" is suggested 112 The naming scheme "subsys_event" is suggested here as a convention
113 intended to limit collisions. Tracepoint names 113 intended to limit collisions. Tracepoint names are global to the
114 kernel: they are considered as being the same 114 kernel: they are considered as being the same whether they are in the
115 core kernel image or in modules. 115 core kernel image or in modules.
116 116
117 If the tracepoint has to be used in kernel mod 117 If the tracepoint has to be used in kernel modules, an
118 EXPORT_TRACEPOINT_SYMBOL_GPL() or EXPORT_TRACE 118 EXPORT_TRACEPOINT_SYMBOL_GPL() or EXPORT_TRACEPOINT_SYMBOL() can be
119 used to export the defined tracepoints. 119 used to export the defined tracepoints.
120 120
121 If you need to do a bit of work for a tracepoi 121 If you need to do a bit of work for a tracepoint parameter, and
122 that work is only used for the tracepoint, tha 122 that work is only used for the tracepoint, that work can be encapsulated
123 within an if statement with the following:: 123 within an if statement with the following::
124 124
125 if (trace_foo_bar_enabled()) { 125 if (trace_foo_bar_enabled()) {
126 int i; 126 int i;
127 int tot = 0; 127 int tot = 0;
128 128
129 for (i = 0; i < count; i++) 129 for (i = 0; i < count; i++)
130 tot += calculate_nugge 130 tot += calculate_nuggets();
131 131
132 trace_foo_bar(tot); 132 trace_foo_bar(tot);
133 } 133 }
134 134
135 All trace_<tracepoint>() calls have a matching 135 All trace_<tracepoint>() calls have a matching trace_<tracepoint>_enabled()
136 function defined that returns true if the trac 136 function defined that returns true if the tracepoint is enabled and
137 false otherwise. The trace_<tracepoint>() shou 137 false otherwise. The trace_<tracepoint>() should always be within the
138 block of the if (trace_<tracepoint>_enabled()) 138 block of the if (trace_<tracepoint>_enabled()) to prevent races between
139 the tracepoint being enabled and the check bei 139 the tracepoint being enabled and the check being seen.
140 140
141 The advantage of using the trace_<tracepoint>_ 141 The advantage of using the trace_<tracepoint>_enabled() is that it uses
142 the static_key of the tracepoint to allow the 142 the static_key of the tracepoint to allow the if statement to be implemented
143 with jump labels and avoid conditional branche 143 with jump labels and avoid conditional branches.
144 144
145 .. note:: The convenience macro TRACE_EVENT pr 145 .. note:: The convenience macro TRACE_EVENT provides an alternative way to
146 define tracepoints. Check http://lwn.net 146 define tracepoints. Check http://lwn.net/Articles/379903,
147 http://lwn.net/Articles/381064 and http: 147 http://lwn.net/Articles/381064 and http://lwn.net/Articles/383362
148 for a series of articles with more detai 148 for a series of articles with more details.
149 <<
150 If you require calling a tracepoint from a hea <<
151 recommended to call one directly or to use the <<
152 function call, as tracepoints in header files <<
153 header is included from a file that has CREATE <<
154 well as the trace_<tracepoint>() is not that s <<
155 and can bloat the kernel if used by other inli <<
156 include tracepoint-defs.h and use tracepoint_e <<
157 <<
158 In a C file:: <<
159 <<
160 void do_trace_foo_bar_wrapper(args) <<
161 { <<
162 trace_foo_bar(args); <<
163 } <<
164 <<
165 In the header file:: <<
166 <<
167 DECLARE_TRACEPOINT(foo_bar); <<
168 <<
169 static inline void some_inline_functio <<
170 { <<
171 [..] <<
172 if (tracepoint_enabled(foo_bar <<
173 do_trace_foo_bar_wrapp <<
174 [..] <<
175 } <<
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.