1 .. SPDX-License-Identifier: BSD-3-Clause
2
3 =======================
4 Introduction to Netlink
5 =======================
6
7 Netlink is often described as an ioctl() repla
8 It aims to replace fixed-format C structures a
9 to ioctl() with a format which allows an easy
10 or extended the arguments.
11
12 To achieve this Netlink uses a minimal fixed-f
13 followed by multiple attributes in the TLV (ty
14
15 Unfortunately the protocol has evolved over th
16 and undocumented fashion, making it hard to co
17 To make the most practical sense this document
18 netlink as it is used today and dives into mor
19 in later sections.
20
21 Opening a socket
22 ================
23
24 Netlink communication happens over sockets, a
25 opened first:
26
27 .. code-block:: c
28
29 fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_GE
30
31 The use of sockets allows for a natural way of
32 in both directions (to and from the kernel). T
33 performed synchronously when applications send
34 a separate recv() system call is needed to rea
35
36 A very simplified flow of a Netlink "call" wil
37 something like:
38
39 .. code-block:: c
40
41 fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_GE
42
43 /* format the request */
44 send(fd, &request, sizeof(request));
45 n = recv(fd, &response, RSP_BUFFER_SIZE);
46 /* interpret the response */
47
48 Netlink also provides natural support for "dum
49 to user space all objects of a certain type (e
50 interfaces).
51
52 .. code-block:: c
53
54 fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_GE
55
56 /* format the dump request */
57 send(fd, &request, sizeof(request));
58 while (1) {
59 n = recv(fd, &buffer, RSP_BUFFER_SIZE);
60 /* one recv() call can read multiple messa
61 for (nl_msg in buffer) {
62 if (nl_msg.nlmsg_type == NLMSG_DONE)
63 goto dump_finished;
64 /* process the object */
65 }
66 }
67 dump_finished:
68
69 The first two arguments of the socket() call r
70 it is opening a Netlink socket, with all heade
71 (hence NETLINK, RAW). The last argument is the
72 This field used to identify the subsystem with
73 communicate.
74
75 Classic vs Generic Netlink
76 --------------------------
77
78 Initial implementation of Netlink depended on
79 of IDs to subsystems and provided little suppo
80 Let us refer to those protocols collectively a
81 The list of them is defined on top of the ``in
82 file, they include among others - general netw
83 iSCSI (NETLINK_ISCSI), and audit (NETLINK_AUDI
84
85 **Generic Netlink** (introduced in 2005) allow
86 subsystems (and subsystem ID allocation), intr
87 implementing the kernel side of the interface.
88
89 The following section describes how to use Gen
90 number of subsystems using Generic Netlink out
91 protocols by an order of magnitude. There are
92 more Classic Netlink protocols to the kernel.
93 Basic information on how communicating with co
94 the Linux kernel (or another of the 20 subsyst
95 Netlink) differs from Generic Netlink is provi
96
97 Generic Netlink
98 ===============
99
100 In addition to the Netlink fixed metadata head
101 defines its own fixed metadata header. (Simila
102 headers stack - Ethernet > IP > TCP we have Ne
103
104 A Netlink message always starts with struct nl
105 by a protocol-specific header. In case of Gene
106 header is struct genlmsghdr.
107
108 The practical meaning of the fields in case of
109
110 .. code-block:: c
111
112 struct nlmsghdr {
113 __u32 nlmsg_len; /* Length of m
114 __u16 nlmsg_type; /* Generic Net
115 __u16 nlmsg_flags; /* Flags - req
116 __u32 nlmsg_seq; /* Sequence nu
117 __u32 nlmsg_pid; /* Port ID, se
118 };
119 struct genlmsghdr {
120 __u8 cmd; /* Command, as
121 __u8 version; /* Irrelevant,
122 __u16 reserved; /* Reserved, s
123 };
124 /* TLV attributes follow... */
125
126 In Classic Netlink :c:member:`nlmsghdr.nlmsg_t
127 which operation within the subsystem the messa
128 (e.g. get information about a netdev). Generic
129 multiple subsystems in a single protocol so it
130 identify the subsystem, and :c:member:`genlmsg
131 the operation instead. (See :ref:`res_fam` for
132 information on how to find the Family ID of th
133 Note that the first 16 values (0 - 15) of this
134 control messages both in Classic Netlink and G
135 See :ref:`nl_msg_type` for more details.
136
137 There are 3 usual types of message exchanges o
138
139 - performing a single action (``do``);
140 - dumping information (``dump``);
141 - getting asynchronous notifications (``multi
142
143 Classic Netlink is very flexible and presumabl
144 of exchanges to happen, but in practice those
145 used.
146
147 Asynchronous notifications are sent by the ker
148 the user sockets which subscribed to them. ``d
149 are initiated by the user. :c:member:`nlmsghdr
150 be set as follows:
151
152 - for ``do``: ``NLM_F_REQUEST | NLM_F_ACK``
153 - for ``dump``: ``NLM_F_REQUEST | NLM_F_ACK |
154
155 :c:member:`nlmsghdr.nlmsg_seq` should be a set
156 increasing value. The value gets echoed back i
157 matter in practice, but setting it to an incre
158 message sent is considered good hygiene. The p
159 matching responses to requests. Asynchronous n
160 :c:member:`nlmsghdr.nlmsg_seq` of ``0``.
161
162 :c:member:`nlmsghdr.nlmsg_pid` is the Netlink
163 This field can be set to ``0`` when talking to
164 See :ref:`nlmsg_pid` for the (uncommon) uses o
165
166 The expected use for :c:member:`genlmsghdr.ver
167 versioning of the APIs provided by the subsyst
168 date made significant use of this field, so se
169 like a safe bet.
170
171 .. _nl_msg_type:
172
173 Netlink message types
174 ---------------------
175
176 As previously mentioned :c:member:`nlmsghdr.nl
177 protocol specific values but the first 16 iden
178 (first subsystem specific message type should
179 ``NLMSG_MIN_TYPE`` which is ``0x10``).
180
181 There are only 4 Netlink control messages defi
182
183 - ``NLMSG_NOOP`` - ignore the message, not us
184 - ``NLMSG_ERROR`` - carries the return code o
185 - ``NLMSG_DONE`` - marks the end of a dump;
186 - ``NLMSG_OVERRUN`` - socket buffer has overf
187
188 ``NLMSG_ERROR`` and ``NLMSG_DONE`` are of prac
189 They carry return codes for operations. Note t
190 the ``NLM_F_ACK`` flag is set on the request N
191 with ``NLMSG_ERROR`` if there is no error. To
192 this quirk it is recommended to always set ``N
193
194 The format of ``NLMSG_ERROR`` is described by
195
196 --------------------------------------------
197 | struct nlmsghdr - response header
198 --------------------------------------------
199 | int error
200 --------------------------------------------
201 | struct nlmsghdr - original request header
202 --------------------------------------------
203 | ** optionally (1) payload of the request
204 --------------------------------------------
205 | ** optionally (2) extended ACK
206 --------------------------------------------
207
208 There are two instances of struct nlmsghdr her
209 and second of the request. ``NLMSG_ERROR`` car
210 the request which led to the error. This could
211 to match requests to responses or re-parse the
212 logs.
213
214 The payload of the request is not echoed in me
215 (``error == 0``) or if ``NETLINK_CAP_ACK`` set
216 The latter is common
217 and perhaps recommended as having to read a co
218 from the kernel is rather wasteful. The absenc
219 is indicated by ``NLM_F_CAPPED`` in :c:member:
220
221 The second optional element of ``NLMSG_ERROR``
222 attributes. See :ref:`ext_ack` for more detail
223 of extended ACK is indicated by ``NLM_F_ACK_TL
224 :c:member:`nlmsghdr.nlmsg_flags`.
225
226 ``NLMSG_DONE`` is simpler, the request is neve
227 ACK attributes may be present::
228
229 --------------------------------------------
230 | struct nlmsghdr - response header
231 --------------------------------------------
232 | int error
233 --------------------------------------------
234 | ** optionally extended ACK
235 --------------------------------------------
236
237 Note that some implementations may issue custo
238 in reply to ``do`` action requests. In that ca
239 implementation-specific and may also be absent
240
241 .. _res_fam:
242
243 Resolving the Family ID
244 -----------------------
245
246 This section explains how to find the Family I
247 It also serves as an example of Generic Netlin
248
249 Generic Netlink is itself a subsystem exposed
250 To avoid a circular dependency Generic Netlink
251 Family ID (``GENL_ID_CTRL`` which is equal to
252 The Generic Netlink family implements a comman
253 about other families (``CTRL_CMD_GETFAMILY``).
254
255 To get information about the Generic Netlink f
256 ``"test1"`` we need to send a message on the p
257 socket. The message should target the Generic
258 ``do`` (2) call to ``CTRL_CMD_GETFAMILY`` (3).
259 call would make the kernel respond with inform
260 it knows about. Last but not least the name of
261 to be specified (4) as an attribute with the a
262
263 struct nlmsghdr:
264 __u32 nlmsg_len: 32
265 __u16 nlmsg_type: GENL_ID_CTRL
266 __u16 nlmsg_flags: NLM_F_REQUEST | NLM_F_
267 __u32 nlmsg_seq: 1
268 __u32 nlmsg_pid: 0
269
270 struct genlmsghdr:
271 __u8 cmd: CTRL_CMD_GETFAMILY
272 __u8 version: 2 /* or 1, doesn't mat
273 __u16 reserved: 0
274
275 struct nlattr:
276 __u16 nla_len: 10
277 __u16 nla_type: CTRL_ATTR_FAMILY_NAME
278 char data: test1\0
279
280 (padding:)
281 char data: \0\0
282
283 The length fields in Netlink (:c:member:`nlmsg
284 and :c:member:`nlattr.nla_len`) always *includ
285 Attribute headers in netlink must be aligned t
286 of the message, hence the extra ``\0\0`` after
287 The attribute lengths *exclude* the padding.
288
289 If the family is found kernel will reply with
290 with all the information about the family::
291
292 /* Message #1 - reply */
293 struct nlmsghdr:
294 __u32 nlmsg_len: 136
295 __u16 nlmsg_type: GENL_ID_CTRL
296 __u16 nlmsg_flags: 0
297 __u32 nlmsg_seq: 1 /* echoed from ou
298 __u32 nlmsg_pid: 5831 /* The PID of our
299
300 struct genlmsghdr:
301 __u8 cmd: CTRL_CMD_GETFAMILY
302 __u8 version: 2
303 __u16 reserved: 0
304
305 struct nlattr:
306 __u16 nla_len: 10
307 __u16 nla_type: CTRL_ATTR_FAMILY_NAME
308 char data: test1\0
309
310 (padding:)
311 data: \0\0
312
313 struct nlattr:
314 __u16 nla_len: 6
315 __u16 nla_type: CTRL_ATTR_FAMILY_ID
316 __u16: 123 /* The Family ID
317
318 (padding:)
319 char data: \0\0
320
321 struct nlattr:
322 __u16 nla_len: 9
323 __u16 nla_type: CTRL_ATTR_FAMILY_VERSI
324 __u16: 1
325
326 /* ... etc, more attributes will follow. */
327
328 And the error code (success) since ``NLM_F_ACK
329
330 /* Message #2 - the ACK */
331 struct nlmsghdr:
332 __u32 nlmsg_len: 36
333 __u16 nlmsg_type: NLMSG_ERROR
334 __u16 nlmsg_flags: NLM_F_CAPPED /* There
335 __u32 nlmsg_seq: 1 /* echoed from ou
336 __u32 nlmsg_pid: 5831 /* The PID of our
337
338 int error: 0
339
340 struct nlmsghdr: /* Copy of the request head
341 __u32 nlmsg_len: 32
342 __u16 nlmsg_type: GENL_ID_CTRL
343 __u16 nlmsg_flags: NLM_F_REQUEST | NLM_F_
344 __u32 nlmsg_seq: 1
345 __u32 nlmsg_pid: 0
346
347 The order of attributes (struct nlattr) is not
348 has to walk the attributes and parse them.
349
350 Note that Generic Netlink sockets are not asso
351 family. A socket can be used to exchange messa
352 families, selecting the recipient family on me
353 the :c:member:`nlmsghdr.nlmsg_type` field.
354
355 .. _ext_ack:
356
357 Extended ACK
358 ------------
359
360 Extended ACK controls reporting of additional
361 in ``NLMSG_ERROR`` and ``NLMSG_DONE`` messages
362 compatibility this feature has to be explicitl
363 the ``NETLINK_EXT_ACK`` setsockopt() to ``1``.
364
365 Types of extended ack attributes are defined i
366 The most commonly used attributes are ``NLMSGE
367 ``NLMSGERR_ATTR_OFFS`` and ``NLMSGERR_ATTR_MIS
368
369 ``NLMSGERR_ATTR_MSG`` carries a message in Eng
370 the encountered problem. These messages are fa
371 than what can be expressed thru standard UNIX
372
373 ``NLMSGERR_ATTR_OFFS`` points to the attribute
374
375 ``NLMSGERR_ATTR_MISS_TYPE`` and ``NLMSGERR_ATT
376 inform about a missing attribute.
377
378 Extended ACKs can be reported on errors as wel
379 The latter should be treated as a warning.
380
381 Extended ACKs greatly improve the usability of
382 always be enabled, appropriately parsed and re
383
384 Advanced topics
385 ===============
386
387 Dump consistency
388 ----------------
389
390 Some of the data structures kernel uses for st
391 it hard to provide an atomic snapshot of all t
392 (without impacting the fast-paths updating the
393
394 Kernel may set the ``NLM_F_DUMP_INTR`` flag on
395 (including the ``NLMSG_DONE`` message) if the
396 may be inconsistent (e.g. missing objects). Us
397 the dump if it sees the flag set.
398
399 Introspection
400 -------------
401
402 The basic introspection abilities are enabled
403 object as reported in :ref:`res_fam`. User can
404 the Generic Netlink family, including which op
405 by the kernel and what attributes the kernel u
406 Family information includes the highest ID of
407 a separate command (``CTRL_CMD_GETPOLICY``) pr
408 about supported attributes, including ranges o
409
410 Querying family information is useful in cases
411 to make sure that the kernel has support for a
412 a request.
413
414 .. _nlmsg_pid:
415
416 nlmsg_pid
417 ---------
418
419 :c:member:`nlmsghdr.nlmsg_pid` is the Netlink
420 It is referred to as Port ID, sometimes Proces
421 reasons if the application does not select (bi
422 kernel will automatically assign it the ID equ
423 (as reported by the getpid() system call).
424
425 Similarly to the bind() semantics of the TCP/I
426 of zero means "assign automatically", hence it
427 to leave the :c:member:`nlmsghdr.nlmsg_pid` fi
428
429 The field is still used today in rare cases wh
430 a unicast notification. User space application
431 its socket with a specific PID, it then commun
432 This way the kernel can reach the specific use
433
434 This sort of communication is utilized in UMH
435 scenarios when kernel needs to trigger user sp
436 space for a policy decision.
437
438 Multicast notifications
439 -----------------------
440
441 One of the strengths of Netlink is the ability
442 to user space. This is a unidirectional form o
443 user) and does not involve any control message
444 ``NLMSG_DONE``.
445
446 For example the Generic Netlink family itself
447 notifications about registered families. When
448 sockets subscribed to the notifications will g
449
450 struct nlmsghdr:
451 __u32 nlmsg_len: 136
452 __u16 nlmsg_type: GENL_ID_CTRL
453 __u16 nlmsg_flags: 0
454 __u32 nlmsg_seq: 0
455 __u32 nlmsg_pid: 0
456
457 struct genlmsghdr:
458 __u8 cmd: CTRL_CMD_NEWFAMILY
459 __u8 version: 2
460 __u16 reserved: 0
461
462 struct nlattr:
463 __u16 nla_len: 10
464 __u16 nla_type: CTRL_ATTR_FAMILY_NAME
465 char data: test1\0
466
467 (padding:)
468 data: \0\0
469
470 struct nlattr:
471 __u16 nla_len: 6
472 __u16 nla_type: CTRL_ATTR_FAMILY_ID
473 __u16: 123 /* The Family ID
474
475 (padding:)
476 char data: \0\0
477
478 struct nlattr:
479 __u16 nla_len: 9
480 __u16 nla_type: CTRL_ATTR_FAMILY_VERSI
481 __u16: 1
482
483 /* ... etc, more attributes will follow. */
484
485 The notification contains the same information
486 to the ``CTRL_CMD_GETFAMILY`` request.
487
488 The Netlink headers of the notification are mo
489 The :c:member:`nlmsghdr.nlmsg_seq` may be eith
490 increasing notification sequence number mainta
491
492 To receive notifications the user socket must
493 notification group. Much like the Family ID, t
494 multicast group is dynamic and can be found in
495 The ``CTRL_ATTR_MCAST_GROUPS`` attribute conta
496 (``CTRL_ATTR_MCAST_GRP_NAME``) and IDs (``CTRL
497 the groups family.
498
499 Once the Group ID is known a setsockopt() call
500
501 .. code-block:: c
502
503 unsigned int group_id;
504
505 /* .. find the group ID... */
506
507 setsockopt(fd, SOL_NETLINK, NETLINK_ADD_MEMB
508 &group_id, sizeof(group_id));
509
510 The socket will now receive notifications.
511
512 It is recommended to use separate sockets for
513 and sending requests to the kernel. The asynch
514 means that they may get mixed in with the resp
515 handling much harder.
516
517 Buffer sizing
518 -------------
519
520 Netlink sockets are datagram sockets rather th
521 meaning that each message must be received in
522 recv()/recvmsg() system call. If the buffer pr
523 short, the message will be truncated and the `
524 in struct msghdr (struct msghdr is the second
525 of the recvmsg() system call, *not* a Netlink
526
527 Upon truncation the remaining part of the mess
528
529 Netlink expects that the user buffer will be a
530 size of the CPU architecture, whichever is big
531 families may, however, require a larger buffer
532 for most efficient handling of dumps (larger b
533 objects and therefore fewer recvmsg() calls ar
534
535 .. _classic_netlink:
536
537 Classic Netlink
538 ===============
539
540 The main differences between Classic and Gener
541 allocation of subsystem identifiers and availa
542 In theory the protocol does not differ signifi
543 Classic Netlink experimented with concepts whi
544 Netlink (really, they usually only found use i
545 subsystem). This section is meant as an explai
546 with the explicit goal of giving the Generic N
547 users the confidence to ignore them when readi
548
549 Most of the concepts and examples here refer t
550 which covers much of the configuration of the
551 Real documentation of that family, deserves a
552
553 Families
554 --------
555
556 Netlink refers to subsystems as families. This
557 sockets and the concept of protocol families,
558 demultiplexing in ``NETLINK_ROUTE``.
559
560 Sadly every layer of encapsulation likes to re
561 as "families" making the term very confusing:
562
563 1. AF_NETLINK is a bona fide socket protocol
564 2. AF_NETLINK's documentation refers to what
565 header (struct nlmsghdr) in a message as a
566 3. Generic Netlink is a family for AF_NETLINK
567 struct nlmsghdr), yet it also calls its us
568
569 Note that the Generic Netlink Family IDs are i
570 and overlap with Classic Netlink protocol numb
571 has the Classic Netlink protocol ID of 21 whic
572 happily allocate to one of its families as wel
573
574 Strict checking
575 ---------------
576
577 The ``NETLINK_GET_STRICT_CHK`` socket option e
578 in ``NETLINK_ROUTE``. It was needed because hi
579 validate the fields of structures it didn't pr
580 to start using those fields later without risk
581 which initialized them incorrectly or not at a
582
583 ``NETLINK_GET_STRICT_CHK`` declares that the a
584 all fields correctly. It also opts into valida
585 contain trailing data and requests that kernel
586 type higher than largest attribute type known
587
588 ``NETLINK_GET_STRICT_CHK`` is not used outside
589
590 Unknown attributes
591 ------------------
592
593 Historically Netlink ignored all unknown attri
594 it would free the application from having to p
595 The application could make a request to change
596 parts of the request "stuck".
597
598 This is no longer the case for new Generic Net
599 in to strict checking. See enum netlink_valida
600 performed.
601
602 Fixed metadata and structures
603 -----------------------------
604
605 Classic Netlink made liberal use of fixed-form
606 the messages. Messages would commonly have a s
607 a considerable number of fields after struct n
608 common to put structures with multiple members
609 without breaking each member into an attribute
610
611 This has caused problems with validation and e
612 therefore using binary structures is actively
613 attributes.
614
615 Request types
616 -------------
617
618 ``NETLINK_ROUTE`` categorized requests into 4
619 and ``SET``. Each object can handle all or som
620 (objects being netdevs, routes, addresses, qdi
621 is defined by the 2 lowest bits of the message
622 new objects would always be allocated with a s
623
624 Each object would also have its own fixed meta
625 types (e.g. struct ifinfomsg for netdev reques
626 requests, struct tcmsg for qdisc requests).
627
628 Even though other protocols and Generic Netlin
629 the same verbs in their message names (``GET``
630 of request types did not find wider adoption.
631
632 Notification echo
633 -----------------
634
635 ``NLM_F_ECHO`` requests for notifications resu
636 to be queued onto the requesting socket. This
637 the impact of the request.
638
639 Note that this feature is not universally impl
640
641 Other request-type-specific flags
642 ---------------------------------
643
644 Classic Netlink defined various flags for its
645 and ``DEL`` requests in the upper byte of nlms
646 Since request types have not been generalized
647 flags are rarely used (and considered deprecat
648
649 For ``GET`` - ``NLM_F_ROOT`` and ``NLM_F_MATCH
650 ``NLM_F_DUMP``, and not used separately. ``NLM
651
652 For ``DEL`` - ``NLM_F_NONREC`` is only used by
653 only by FDB some operations.
654
655 The flags for ``NEW`` are used most commonly i
656 the meaning is not crystal clear. The followin
657 best guess of the intention of the authors, an
658 stray from it in one way or another. ``NLM_F_R
659 an existing object, if no matching object exis
660 ``NLM_F_EXCL`` has the opposite semantics and
661 existed.
662 ``NLM_F_CREATE`` asks for the object to be cre
663 exist, it can be combined with ``NLM_F_REPLACE
664
665 A comment in the main Netlink uAPI header stat
666
667 4.4BSD ADD NLM_F_CREATE|NLM_F_EXC
668 4.4BSD CHANGE NLM_F_REPLACE
669
670 True CHANGE NLM_F_CREATE|NLM_F_REP
671 Append NLM_F_CREATE
672 Check NLM_F_EXCL
673
674 which seems to indicate that those flags preda
675 ``NLM_F_REPLACE`` without ``NLM_F_CREATE`` was
676 of ``SET`` commands.
677 ``NLM_F_EXCL`` without ``NLM_F_CREATE`` was us
678 without creating it, presumably predating ``GE
679
680 ``NLM_F_APPEND`` indicates that if one key can
681 with it (e.g. multiple next-hop objects for a
682 added to the list rather than replacing the en
683
684 uAPI reference
685 ==============
686
687 .. kernel-doc:: include/uapi/linux/netlink.h
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.