1 Dynamic debug
2 +++++++++++++
3
4
5 Introduction
6 ============
7
8 Dynamic debug allows you to dynamically enable
9 debug-print code to obtain additional kernel i
10
11 If ``/proc/dynamic_debug/control`` exists, you
12 debug. You'll need root access (sudo su) to u
13
14 Dynamic debug provides:
15
16 * a Catalog of all *prdbgs* in your kernel.
17 ``cat /proc/dynamic_debug/control`` to see
18
19 * a Simple query/command language to alter *p
20 any combination of 0 or 1 of:
21
22 - source filename
23 - function name
24 - line number (including ranges of line num
25 - module name
26 - format string
27 - class name (as known/declared by each mod
28
29 NOTE: To actually get the debug-print output o
30 need to adjust the kernel ``loglevel=``, or us
31 Read about these kernel parameters in
32 Documentation/admin-guide/kernel-parameters.rs
33
34 Viewing Dynamic Debug Behaviour
35 ===============================
36
37 You can view the currently configured behaviou
38
39 :#> head -n7 /proc/dynamic_debug/control
40 # filename:lineno [module]function flags for
41 init/main.c:1179 [main]initcall_blacklist =_
42 init/main.c:1218 [main]initcall_blacklisted
43 init/main.c:1424 [main]run_init_process =_ "
44 init/main.c:1426 [main]run_init_process =_ "
45 init/main.c:1427 [main]run_init_process =_ "
46 init/main.c:1429 [main]run_init_process =_ "
47
48 The 3rd space-delimited column shows the curre
49 a ``=`` for easy use with grep/cut. ``=p`` sho
50
51 Controlling dynamic debug Behaviour
52 ===================================
53
54 The behaviour of *prdbg* sites are controlled
55 query/commands to the control file. Example::
56
57 # grease the interface
58 :#> alias ddcmd='echo $* > /proc/dynamic_deb
59
60 :#> ddcmd '-p; module main func run* +p'
61 :#> grep =p /proc/dynamic_debug/control
62 init/main.c:1424 [main]run_init_process =p "
63 init/main.c:1426 [main]run_init_process =p "
64 init/main.c:1427 [main]run_init_process =p "
65 init/main.c:1429 [main]run_init_process =p "
66
67 Error messages go to console/syslog::
68
69 :#> ddcmd mode foo +p
70 dyndbg: unknown keyword "mode"
71 dyndbg: query parse failed
72 bash: echo: write error: Invalid argument
73
74 If debugfs is also enabled and mounted, ``dyna
75 also under the mount-dir, typically ``/sys/ker
76
77 Command Language Reference
78 ==========================
79
80 At the basic lexical level, a command is a seq
81 by spaces or tabs. So these are all equivalen
82
83 :#> ddcmd file svcsock.c line 1603 +p
84 :#> ddcmd "file svcsock.c line 1603 +p"
85 :#> ddcmd ' file svcsock.c line 1603
86
87 Command submissions are bounded by a write() s
88 Multiple commands can be written together, sep
89
90 :#> ddcmd "func pnpacpi_get_resources +p; fu
91 :#> ddcmd <<"EOC"
92 func pnpacpi_get_resources +p
93 func pnp_assign_mem +p
94 EOC
95 :#> cat query-batch-file > /proc/dynamic_deb
96
97 You can also use wildcards in each query term.
98 ``*`` (matches zero or more characters) and ``
99 character). For example, you can match all usb
100
101 :#> ddcmd file "drivers/usb/*" +p # "" t
102
103 Syntactically, a command is pairs of keyword v
104 flags change or setting::
105
106 command ::= match-spec* flags-spec
107
108 The match-spec's select *prdbgs* from the cata
109 the flags-spec, all constraints are ANDed toge
110 is the same as keyword "*".
111
112
113 A match specification is a keyword, which sele
114 the callsite to be compared, and a value to co
115 keywords are:::
116
117 match-spec ::= 'func' string |
118 'file' string |
119 'module' string |
120 'format' string |
121 'class' string |
122 'line' line-range
123
124 line-range ::= lineno |
125 '-'lineno |
126 lineno'-' |
127 lineno'-'lineno
128
129 lineno ::= unsigned-int
130
131 .. note::
132
133 ``line-range`` cannot contain space, e.g.
134 "1-30" is valid range but "1 - 30" is not.
135
136
137 The meanings of each keyword are:
138
139 func
140 The given string is compared against the f
141 of each callsite. Example::
142
143 func svc_tcp_accept
144 func *recv* # in rfcomm, b
145
146 file
147 The given string is compared against eithe
148 pathname, or the basename of the source fi
149 Examples::
150
151 file svcsock.c
152 file kernel/freezer.c # ie column 1
153 file drivers/usb/* # all callsite
154 file inode.c:start_* # parse :tail
155 file inode.c:1-100 # parse :tail
156
157 module
158 The given string is compared against the m
159 of each callsite. The module name is the
160 seen in ``lsmod``, i.e. without the direct
161 suffix and with ``-`` changed to ``_``. E
162
163 module sunrpc
164 module nfsd
165 module drm* # both drm, drm_kms_he
166
167 format
168 The given string is searched for in the dy
169 string. Note that the string does not nee
170 entire format, only some part. Whitespace
171 special characters can be escaped using C
172 escape ``\ooo`` notation, e.g. the space c
173 Alternatively, the string can be enclosed
174 characters (``"``) or single quote charact
175 Examples::
176
177 format svcrdma: // many of the
178 format readahead // some pr_deb
179 format nfsd:\040SETATTR // one way to
180 format "nfsd: SETATTR" // a neater wa
181 format 'nfsd: SETATTR' // yet another
182
183 class
184 The given class_name is validated against
185 have declared a list of known class_names.
186 found for a module, callsite & class match
187 proceeds. Examples::
188
189 class DRM_UT_KMS # a DRM.debug
190 class JUNK # silent non-m
191 // class TLD_* # NOTICE: no w
192
193 line
194 The given line number or range of line num
195 against the line number of each ``pr_debug
196 line number matches the callsite line numb
197 range of line numbers matches any callsite
198 and last line number inclusive. An empty
199 the first line in the file, an empty last
200 last line number in the file. Examples::
201
202 line 1603 // exactly line 16
203 line 1600-1605 // the six lines f
204 line -1605 // the 1605 lines
205 line 1600- // all lines from
206
207 The flags specification comprises a change ope
208 by one or more flag characters. The change op
209 of the characters::
210
211 - remove the given flags
212 + add the given flags
213 = set the flags to the given flags
214
215 The flags are::
216
217 p enables the pr_debug() callsite.
218 _ enables no flags.
219
220 Decorator flags add to the message-prefix, i
221 t Include thread ID, or <intr>
222 m Include module name
223 f Include the function name
224 s Include the source file name
225 l Include line number
226
227 For ``print_hex_dump_debug()`` and ``print_hex
228 the ``p`` flag has meaning, other flags are ig
229
230 Note the regexp ``^[-+=][fslmpt_]+$`` matches
231 To clear all flags at once, use ``=_`` or ``-f
232
233
234 Debug messages during Boot Process
235 ==================================
236
237 To activate debug messages for core code and b
238 the boot process, even before userspace and de
239 ``dyndbg="QUERY"`` or ``module.dyndbg="QUERY"`
240 the syntax described above, but must not excee
241 bootloader may impose lower limits.
242
243 These ``dyndbg`` params are processed just aft
244 processed, as part of the early_initcall. Thu
245 messages in all code run after this early_init
246 parameter.
247
248 On an x86 system for example ACPI enablement i
249
250 dyndbg="file ec.c +p"
251
252 will show early Embedded Controller transactio
253 your machine (typically a laptop) has an Embed
254 PCI (or other devices) initialization also is
255 this boot parameter for debugging purposes.
256
257 If ``foo`` module is not built-in, ``foo.dyndb
258 boot time, without effect, but will be reproce
259 loaded later. Bare ``dyndbg=`` is only process
260
261
262 Debug Messages at Module Initialization Time
263 ============================================
264
265 When ``modprobe foo`` is called, modprobe scan
266 ``foo.params``, strips ``foo.``, and passes th
267 params given in modprobe args or ``/etc/modpro
268 in the following order:
269
270 1. parameters given via ``/etc/modprobe.d/*.co
271
272 options foo dyndbg=+pt
273 options foo dyndbg # defaults to +p
274
275 2. ``foo.dyndbg`` as given in boot args, ``foo
276
277 foo.dyndbg=" func bar +p; func buz +mp
278
279 3. args to modprobe::
280
281 modprobe foo dyndbg==pmf # override pr
282
283 These ``dyndbg`` queries are applied in order,
284 This allows boot args to override or modify th
285 (sensible, since 1 is system wide, 2 is kernel
286 modprobe args to override both.
287
288 In the ``foo.dyndbg="QUERY"`` form, the query
289 ``foo`` is extracted from the param-name, and
290 ``QUERY``, and only 1 match-spec of each type
291
292 The ``dyndbg`` option is a "fake" module param
293
294 - modules do not need to define it explicitly
295 - every module gets it tacitly, whether they u
296 - it doesn't appear in ``/sys/module/$module/p
297 To see it, grep the control file, or inspect
298
299 For ``CONFIG_DYNAMIC_DEBUG`` kernels, any sett
300 enabled by ``-DDEBUG`` flag during compilation
301 the debugfs interface if the debug messages ar
302
303 echo "module module_name -p" > /proc/dynami
304
305 Examples
306 ========
307
308 ::
309
310 // enable the message at line 1603 of file s
311 :#> ddcmd 'file svcsock.c line 1603 +p'
312
313 // enable all the messages in file svcsock.c
314 :#> ddcmd 'file svcsock.c +p'
315
316 // enable all the messages in the NFS server
317 :#> ddcmd 'module nfsd +p'
318
319 // enable all 12 messages in the function sv
320 :#> ddcmd 'func svc_process +p'
321
322 // disable all 12 messages in the function s
323 :#> ddcmd 'func svc_process -p'
324
325 // enable messages for NFS calls READ, READL
326 :#> ddcmd 'format "nfsd: READ" +p'
327
328 // enable messages in files of which the pat
329 :#> ddcmd 'file *usb* +p'
330
331 // enable all messages
332 :#> ddcmd '+p'
333
334 // add module, function to all enabled messa
335 :#> ddcmd '+mf'
336
337 // boot-args example, with newlines and comm
338 Kernel command line: ...
339 // see what's going on in dyndbg=value pro
340 dynamic_debug.verbose=3
341 // enable pr_debugs in the btrfs module (c
342 btrfs.dyndbg="+p"
343 // enable pr_debugs in all files under ini
344 // and the function parse_one, #cmt is str
345 dyndbg="file init/* +p #cmt ; func parse_o
346 // enable pr_debugs in 2 functions in a mo
347 pc87360.dyndbg="func pc87360_init_device +
348
349 Kernel Configuration
350 ====================
351
352 Dynamic Debug is enabled via kernel config ite
353
354 CONFIG_DYNAMIC_DEBUG=y # build catalo
355 CONFIG_DYNAMIC_DEBUG_CORE=y # enable mecha
356
357 If you do not want to enable dynamic debug glo
358 system), you may set ``CONFIG_DYNAMIC_DEBUG_CO
359 debug and add ``ccflags := -DDYNAMIC_DEBUG_MOD
360 modules which you'd like to dynamically debug
361
362
363 Kernel *prdbg* API
364 ==================
365
366 The following functions are cataloged and cont
367 debug is enabled::
368
369 pr_debug()
370 dev_dbg()
371 print_hex_dump_debug()
372 print_hex_dump_bytes()
373
374 Otherwise, they are off by default; ``ccflags
375 ``#define DEBUG`` in a source file will enable
376
377 If ``CONFIG_DYNAMIC_DEBUG`` is not set, ``prin
378 just a shortcut for ``print_hex_dump(KERN_DEBU
379
380 For ``print_hex_dump_debug()``/``print_hex_dum
381 its ``prefix_str`` argument, if it is constant
382 in case ``prefix_str`` is built dynamically.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.