1 .. SPDX-License-Identifier: GPL-2.0 1 .. SPDX-License-Identifier: GPL-2.0
2 2
3 ========== 3 ==========
4 Netconsole 4 Netconsole
5 ========== 5 ==========
6 6
7 7
8 started by Ingo Molnar <mingo@redhat.com>, 2001 8 started by Ingo Molnar <mingo@redhat.com>, 2001.09.17
9 9
10 2.6 port and netpoll api by Matt Mackall <mpm@s 10 2.6 port and netpoll api by Matt Mackall <mpm@selenic.com>, Sep 9 2003
11 11
12 IPv6 support by Cong Wang <xiyou.wangcong@gmail 12 IPv6 support by Cong Wang <xiyou.wangcong@gmail.com>, Jan 1 2013
13 13
14 Extended console support by Tejun Heo <tj@kerne 14 Extended console support by Tejun Heo <tj@kernel.org>, May 1 2015
15 15
16 Release prepend support by Breno Leitao <leitao 16 Release prepend support by Breno Leitao <leitao@debian.org>, Jul 7 2023
17 17
18 Userdata append support by Matthew Wood <thepac <<
19 <<
20 Please send bug reports to Matt Mackall <mpm@se 18 Please send bug reports to Matt Mackall <mpm@selenic.com>
21 Satyam Sharma <satyam.sharma@gmail.com>, and Co 19 Satyam Sharma <satyam.sharma@gmail.com>, and Cong Wang <xiyou.wangcong@gmail.com>
22 20
23 Introduction: 21 Introduction:
24 ============= 22 =============
25 23
26 This module logs kernel printk messages over U 24 This module logs kernel printk messages over UDP allowing debugging of
27 problem where disk logging fails and serial co 25 problem where disk logging fails and serial consoles are impractical.
28 26
29 It can be used either built-in or as a module. 27 It can be used either built-in or as a module. As a built-in,
30 netconsole initializes immediately after NIC c 28 netconsole initializes immediately after NIC cards and will bring up
31 the specified interface as soon as possible. W 29 the specified interface as soon as possible. While this doesn't allow
32 capture of early kernel panics, it does captur 30 capture of early kernel panics, it does capture most of the boot
33 process. 31 process.
34 32
35 Sender and receiver configuration: 33 Sender and receiver configuration:
36 ================================== 34 ==================================
37 35
38 It takes a string configuration parameter "net 36 It takes a string configuration parameter "netconsole" in the
39 following format:: 37 following format::
40 38
41 netconsole=[+][r][src-port]@[src-ip]/[<dev>],[ 39 netconsole=[+][r][src-port]@[src-ip]/[<dev>],[tgt-port]@<tgt-ip>/[tgt-macaddr]
42 40
43 where 41 where
44 + if present, enable exten 42 + if present, enable extended console support
45 r if present, prepend kern 43 r if present, prepend kernel version (release) to the message
46 src-port source for UDP packets ( 44 src-port source for UDP packets (defaults to 6665)
47 src-ip source IP to use (interf 45 src-ip source IP to use (interface address)
48 dev network interface (eth0) 46 dev network interface (eth0)
49 tgt-port port for logging agent ( 47 tgt-port port for logging agent (6666)
50 tgt-ip IP address for logging a 48 tgt-ip IP address for logging agent
51 tgt-macaddr ethernet MAC address for 49 tgt-macaddr ethernet MAC address for logging agent (broadcast)
52 50
53 Examples:: 51 Examples::
54 52
55 linux netconsole=4444@10.0.0.1/eth1,9353@10.0 53 linux netconsole=4444@10.0.0.1/eth1,9353@10.0.0.2/12:34:56:78:9a:bc
56 54
57 or:: 55 or::
58 56
59 insmod netconsole netconsole=@/,@10.0.0.2/ 57 insmod netconsole netconsole=@/,@10.0.0.2/
60 58
61 or using IPv6:: 59 or using IPv6::
62 60
63 insmod netconsole netconsole=@/,@fd00:1:2:3:: 61 insmod netconsole netconsole=@/,@fd00:1:2:3::1/
64 62
65 It also supports logging to multiple remote ag 63 It also supports logging to multiple remote agents by specifying
66 parameters for the multiple agents separated b 64 parameters for the multiple agents separated by semicolons and the
67 complete string enclosed in "quotes", thusly:: 65 complete string enclosed in "quotes", thusly::
68 66
69 modprobe netconsole netconsole="@/,@10.0.0.2/ 67 modprobe netconsole netconsole="@/,@10.0.0.2/;@/eth1,6892@10.0.0.3/"
70 68
71 Built-in netconsole starts immediately after t 69 Built-in netconsole starts immediately after the TCP stack is
72 initialized and attempts to bring up the suppl 70 initialized and attempts to bring up the supplied dev at the supplied
73 address. 71 address.
74 72
75 The remote host has several options to receive 73 The remote host has several options to receive the kernel messages,
76 for example: 74 for example:
77 75
78 1) syslogd 76 1) syslogd
79 77
80 2) netcat 78 2) netcat
81 79
82 On distributions using a BSD-based netcat v 80 On distributions using a BSD-based netcat version (e.g. Fedora,
83 openSUSE and Ubuntu) the listening port mus 81 openSUSE and Ubuntu) the listening port must be specified without
84 the -p switch:: 82 the -p switch::
85 83
86 nc -u -l -p <port>' / 'nc -u -l <port> 84 nc -u -l -p <port>' / 'nc -u -l <port>
87 85
88 or:: 86 or::
89 87
90 netcat -u -l -p <port>' / 'netcat -u - 88 netcat -u -l -p <port>' / 'netcat -u -l <port>
91 89
92 3) socat 90 3) socat
93 91
94 :: 92 ::
95 93
96 socat udp-recv:<port> - 94 socat udp-recv:<port> -
97 95
98 Dynamic reconfiguration: 96 Dynamic reconfiguration:
99 ======================== 97 ========================
100 98
101 Dynamic reconfigurability is a useful addition 99 Dynamic reconfigurability is a useful addition to netconsole that enables
102 remote logging targets to be dynamically added 100 remote logging targets to be dynamically added, removed, or have their
103 parameters reconfigured at runtime from a conf 101 parameters reconfigured at runtime from a configfs-based userspace interface.
>> 102 [ Note that the parameters of netconsole targets that were specified/created
>> 103 from the boot/module option are not exposed via this interface, and hence
>> 104 cannot be modified dynamically. ]
104 105
105 To include this feature, select CONFIG_NETCONS 106 To include this feature, select CONFIG_NETCONSOLE_DYNAMIC when building the
106 netconsole module (or kernel, if netconsole is 107 netconsole module (or kernel, if netconsole is built-in).
107 108
108 Some examples follow (where configfs is mounte 109 Some examples follow (where configfs is mounted at the /sys/kernel/config
109 mountpoint). 110 mountpoint).
110 111
111 To add a remote logging target (target names c 112 To add a remote logging target (target names can be arbitrary)::
112 113
113 cd /sys/kernel/config/netconsole/ 114 cd /sys/kernel/config/netconsole/
114 mkdir target1 115 mkdir target1
115 116
116 Note that newly created targets have default p 117 Note that newly created targets have default parameter values (as mentioned
117 above) and are disabled by default -- they mus 118 above) and are disabled by default -- they must first be enabled by writing
118 "1" to the "enabled" attribute (usually after 119 "1" to the "enabled" attribute (usually after setting parameters accordingly)
119 as described below. 120 as described below.
120 121
121 To remove a target:: 122 To remove a target::
122 123
123 rmdir /sys/kernel/config/netconsole/othertarg 124 rmdir /sys/kernel/config/netconsole/othertarget/
124 125
125 The interface exposes these parameters of a ne 126 The interface exposes these parameters of a netconsole target to userspace:
126 127
127 ============== ====================== 128 ============== ================================= ============
128 enabled Is this target current 129 enabled Is this target currently enabled? (read-write)
129 extended Extended mode enabled 130 extended Extended mode enabled (read-write)
130 release Prepend kernel release 131 release Prepend kernel release to message (read-write)
131 dev_name Local network interfac 132 dev_name Local network interface name (read-write)
132 local_port Source UDP port to use 133 local_port Source UDP port to use (read-write)
133 remote_port Remote agent's UDP por 134 remote_port Remote agent's UDP port (read-write)
134 local_ip Source IP address to u 135 local_ip Source IP address to use (read-write)
135 remote_ip Remote agent's IP addr 136 remote_ip Remote agent's IP address (read-write)
136 local_mac Local interface's MAC 137 local_mac Local interface's MAC address (read-only)
137 remote_mac Remote agent's MAC add 138 remote_mac Remote agent's MAC address (read-write)
138 ============== ====================== 139 ============== ================================= ============
139 140
140 The "enabled" attribute is also used to contro 141 The "enabled" attribute is also used to control whether the parameters of
141 a target can be updated or not -- you can modi 142 a target can be updated or not -- you can modify the parameters of only
142 disabled targets (i.e. if "enabled" is 0). 143 disabled targets (i.e. if "enabled" is 0).
143 144
144 To update a target's parameters:: 145 To update a target's parameters::
145 146
146 cat enabled # chec 147 cat enabled # check if enabled is 1
147 echo 0 > enabled # disa 148 echo 0 > enabled # disable the target (if required)
148 echo eth2 > dev_name # set 149 echo eth2 > dev_name # set local interface
149 echo 10.0.0.4 > remote_ip # upda 150 echo 10.0.0.4 > remote_ip # update some parameter
150 echo cb:a9:87:65:43:21 > remote_mac # upda 151 echo cb:a9:87:65:43:21 > remote_mac # update more parameters
151 echo 1 > enabled # enab 152 echo 1 > enabled # enable target again
152 153
153 You can also update the local interface dynami 154 You can also update the local interface dynamically. This is especially
154 useful if you want to use interfaces that have 155 useful if you want to use interfaces that have newly come up (and may not
155 have existed when netconsole was loaded / init 156 have existed when netconsole was loaded / initialized).
156 <<
157 Netconsole targets defined at boot time (or mo <<
158 `netconsole=` param are assigned the name `cmd <<
159 first target in the parameter is named `cmdlin <<
160 these targets by creating configfs directories <<
161 <<
162 Let's suppose you have two netconsole targets <<
163 <<
164 netconsole=4444@10.0.0.1/eth1,9353@10.0.0.2/1 <<
165 <<
166 You can modify these targets in runtime by cre <<
167 <<
168 mkdir cmdline0 <<
169 cat cmdline0/remote_ip <<
170 10.0.0.2 <<
171 <<
172 mkdir cmdline1 <<
173 cat cmdline1/remote_ip <<
174 10.0.0.3 <<
175 <<
176 Append User Data <<
177 ---------------- <<
178 <<
179 Custom user data can be appended to the end of <<
180 dynamic configuration enabled. User data entri <<
181 changing the "enabled" attribute of a target. <<
182 <<
183 Directories (keys) under `userdata` are limite <<
184 data in `userdata/<key>/value` are limited to <<
185 <<
186 cd /sys/kernel/config/netconsole && mkdir cmd <<
187 cd cmdline0 <<
188 mkdir userdata/foo <<
189 echo bar > userdata/foo/value <<
190 mkdir userdata/qux <<
191 echo baz > userdata/qux/value <<
192 <<
193 Messages will now include this additional user <<
194 <<
195 echo "This is a message" > /dev/kmsg <<
196 <<
197 Sends:: <<
198 <<
199 12,607,22085407756,-;This is a message <<
200 foo=bar <<
201 qux=baz <<
202 <<
203 Preview the userdata that will be appended wit <<
204 <<
205 cd /sys/kernel/config/netconsole/cmdline0/use <<
206 for f in `ls userdata`; do echo $f=$(cat user <<
207 <<
208 If a `userdata` entry is created but no data i <<
209 the entry will be omitted from netconsole mess <<
210 <<
211 cd /sys/kernel/config/netconsole && mkdir cmd <<
212 cd cmdline0 <<
213 mkdir userdata/foo <<
214 echo bar > userdata/foo/value <<
215 mkdir userdata/qux <<
216 <<
217 The `qux` key is omitted since it has no value <<
218 <<
219 echo "This is a message" > /dev/kmsg <<
220 12,607,22085407756,-;This is a message <<
221 foo=bar <<
222 <<
223 Delete `userdata` entries with `rmdir`:: <<
224 <<
225 rmdir /sys/kernel/config/netconsole/cmdline0/ <<
226 <<
227 .. warning:: <<
228 When writing strings to user data values, i <<
229 configfs store calls and this can cause con <<
230 <<
231 mkdir userdata/testing <<
232 printf "val1\nval2" > userdata/testing/va <<
233 # userdata store value is called twice, f <<
234 # so "val2" is stored, being the last val <<
235 cat userdata/testing/value <<
236 val2 <<
237 <<
238 It is recommended to not write user data va <<
239 157
240 Extended console: 158 Extended console:
241 ================= 159 =================
242 160
243 If '+' is prefixed to the configuration line o 161 If '+' is prefixed to the configuration line or "extended" config file
244 is set to 1, extended console support is enabl 162 is set to 1, extended console support is enabled. An example boot
245 param follows:: 163 param follows::
246 164
247 linux netconsole=+4444@10.0.0.1/eth1,9353@10. 165 linux netconsole=+4444@10.0.0.1/eth1,9353@10.0.0.2/12:34:56:78:9a:bc
248 166
249 Log messages are transmitted with extended met 167 Log messages are transmitted with extended metadata header in the
250 following format which is the same as /dev/kms 168 following format which is the same as /dev/kmsg::
251 169
252 <level>,<sequnum>,<timestamp>,<contflag>;<mes 170 <level>,<sequnum>,<timestamp>,<contflag>;<message text>
253 171
254 If 'r' (release) feature is enabled, the kerne 172 If 'r' (release) feature is enabled, the kernel release version is
255 prepended to the start of the message. Example 173 prepended to the start of the message. Example::
256 174
257 6.4.0,6,444,501151268,-;netconsole: network l 175 6.4.0,6,444,501151268,-;netconsole: network logging started
258 176
259 Non printable characters in <message text> are 177 Non printable characters in <message text> are escaped using "\xff"
260 notation. If the message contains optional dic 178 notation. If the message contains optional dictionary, verbatim
261 newline is used as the delimiter. 179 newline is used as the delimiter.
262 180
263 If a message doesn't fit in certain number of 181 If a message doesn't fit in certain number of bytes (currently 1000),
264 the message is split into multiple fragments b 182 the message is split into multiple fragments by netconsole. These
265 fragments are transmitted with "ncfrag" header 183 fragments are transmitted with "ncfrag" header field added::
266 184
267 ncfrag=<byte-offset>/<total-bytes> 185 ncfrag=<byte-offset>/<total-bytes>
268 186
269 For example, assuming a lot smaller chunk size 187 For example, assuming a lot smaller chunk size, a message "the first
270 chunk, the 2nd chunk." may be split as follows 188 chunk, the 2nd chunk." may be split as follows::
271 189
272 6,416,1758426,-,ncfrag=0/31;the first chunk, 190 6,416,1758426,-,ncfrag=0/31;the first chunk,
273 6,416,1758426,-,ncfrag=16/31; the 2nd chunk. 191 6,416,1758426,-,ncfrag=16/31; the 2nd chunk.
274 192
275 Miscellaneous notes: 193 Miscellaneous notes:
276 ==================== 194 ====================
277 195
278 .. Warning:: 196 .. Warning::
279 197
280 the default target ethernet setting uses th 198 the default target ethernet setting uses the broadcast
281 ethernet address to send packets, which can 199 ethernet address to send packets, which can cause increased load on
282 other systems on the same ethernet segment. 200 other systems on the same ethernet segment.
283 201
284 .. Tip:: 202 .. Tip::
285 203
286 some LAN switches may be configured to supp 204 some LAN switches may be configured to suppress ethernet broadcasts
287 so it is advised to explicitly specify the 205 so it is advised to explicitly specify the remote agents' MAC addresses
288 from the config parameters passed to netcon 206 from the config parameters passed to netconsole.
289 207
290 .. Tip:: 208 .. Tip::
291 209
292 to find out the MAC address of, say, 10.0.0 210 to find out the MAC address of, say, 10.0.0.2, you may try using::
293 211
294 ping -c 1 10.0.0.2 ; /sbin/arp -n | gr 212 ping -c 1 10.0.0.2 ; /sbin/arp -n | grep 10.0.0.2
295 213
296 .. Tip:: 214 .. Tip::
297 215
298 in case the remote logging agent is on a se 216 in case the remote logging agent is on a separate LAN subnet than
299 the sender, it is suggested to try specifyi 217 the sender, it is suggested to try specifying the MAC address of the
300 default gateway (you may use /sbin/route -n 218 default gateway (you may use /sbin/route -n to find it out) as the
301 remote MAC address instead. 219 remote MAC address instead.
302 220
303 .. note:: 221 .. note::
304 222
305 the network device (eth1 in the above case) 223 the network device (eth1 in the above case) can run any kind
306 of other network traffic, netconsole is not 224 of other network traffic, netconsole is not intrusive. Netconsole
307 might cause slight delays in other traffic 225 might cause slight delays in other traffic if the volume of kernel
308 messages is high, but should have no other 226 messages is high, but should have no other impact.
309 227
310 .. note:: 228 .. note::
311 229
312 if you find that the remote logging agent i 230 if you find that the remote logging agent is not receiving or
313 printing all messages from the sender, it i 231 printing all messages from the sender, it is likely that you have set
314 the "console_loglevel" parameter (on the se 232 the "console_loglevel" parameter (on the sender) to only send high
315 priority messages to the console. You can c 233 priority messages to the console. You can change this at runtime using::
316 234
317 dmesg -n 8 235 dmesg -n 8
318 236
319 or by specifying "debug" on the kernel comm 237 or by specifying "debug" on the kernel command line at boot, to send
320 all kernel messages to the console. A speci 238 all kernel messages to the console. A specific value for this parameter
321 can also be set using the "loglevel" kernel 239 can also be set using the "loglevel" kernel boot option. See the
322 dmesg(8) man page and Documentation/admin-g 240 dmesg(8) man page and Documentation/admin-guide/kernel-parameters.rst
323 for details. 241 for details.
324 242
325 Netconsole was designed to be as instantaneous 243 Netconsole was designed to be as instantaneous as possible, to
326 enable the logging of even the most critical k 244 enable the logging of even the most critical kernel bugs. It works
327 from IRQ contexts as well, and does not enable 245 from IRQ contexts as well, and does not enable interrupts while
328 sending packets. Due to these unique needs, co 246 sending packets. Due to these unique needs, configuration cannot
329 be more automatic, and some fundamental limita 247 be more automatic, and some fundamental limitations will remain:
330 only IP networks, UDP packets and ethernet dev 248 only IP networks, UDP packets and ethernet devices are supported.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.