1 .. SPDX-License-Identifier: GPL-2.0 1 .. SPDX-License-Identifier: GPL-2.0
2 2
3 ======================= 3 =======================
4 NFSv4 client identifier 4 NFSv4 client identifier
5 ======================= 5 =======================
6 6
7 This document explains how the NFSv4 protocol 7 This document explains how the NFSv4 protocol identifies client
8 instances in order to maintain file open and l 8 instances in order to maintain file open and lock state during
9 system restarts. A special identifier and prin 9 system restarts. A special identifier and principal are maintained
10 on each client. These can be set by administra 10 on each client. These can be set by administrators, scripts
11 provided by site administrators, or tools prov 11 provided by site administrators, or tools provided by Linux
12 distributors. 12 distributors.
13 13
14 There are risks if a client's NFSv4 identifier 14 There are risks if a client's NFSv4 identifier and its principal
15 are not chosen carefully. 15 are not chosen carefully.
16 16
17 17
18 Introduction 18 Introduction
19 ------------ 19 ------------
20 20
21 The NFSv4 protocol uses "lease-based file lock 21 The NFSv4 protocol uses "lease-based file locking". Leases help
22 NFSv4 servers provide file lock guarantees and 22 NFSv4 servers provide file lock guarantees and manage their
23 resources. 23 resources.
24 24
25 Simply put, an NFSv4 server creates a lease fo 25 Simply put, an NFSv4 server creates a lease for each NFSv4 client.
26 The server collects each client's file open an 26 The server collects each client's file open and lock state under
27 the lease for that client. 27 the lease for that client.
28 28
29 The client is responsible for periodically ren 29 The client is responsible for periodically renewing its leases.
30 While a lease remains valid, the server holdin 30 While a lease remains valid, the server holding that lease
31 guarantees the file locks the client has creat 31 guarantees the file locks the client has created remain in place.
32 32
33 If a client stops renewing its lease (for exam 33 If a client stops renewing its lease (for example, if it crashes),
34 the NFSv4 protocol allows the server to remove 34 the NFSv4 protocol allows the server to remove the client's open
35 and lock state after a certain period of time. 35 and lock state after a certain period of time. When a client
36 restarts, it indicates to servers that open an 36 restarts, it indicates to servers that open and lock state
37 associated with its previous leases is no long 37 associated with its previous leases is no longer valid and can be
38 destroyed immediately. 38 destroyed immediately.
39 39
40 In addition, each NFSv4 server manages a persi 40 In addition, each NFSv4 server manages a persistent list of client
41 leases. When the server restarts and clients a 41 leases. When the server restarts and clients attempt to recover
42 their state, the server uses this list to dist 42 their state, the server uses this list to distinguish amongst
43 clients that held state before the server rest 43 clients that held state before the server restarted and clients
44 sending fresh OPEN and LOCK requests. This ena 44 sending fresh OPEN and LOCK requests. This enables file locks to
45 persist safely across server restarts. 45 persist safely across server restarts.
46 46
47 NFSv4 client identifiers 47 NFSv4 client identifiers
48 ------------------------ 48 ------------------------
49 49
50 Each NFSv4 client presents an identifier to NF 50 Each NFSv4 client presents an identifier to NFSv4 servers so that
51 they can associate the client with its lease. 51 they can associate the client with its lease. Each client's
52 identifier consists of two elements: 52 identifier consists of two elements:
53 53
54 - co_ownerid: An arbitrary but fixed string. 54 - co_ownerid: An arbitrary but fixed string.
55 55
56 - boot verifier: A 64-bit incarnation verifi 56 - boot verifier: A 64-bit incarnation verifier that enables a
57 server to distinguish successive boot epoc 57 server to distinguish successive boot epochs of the same client.
58 58
59 The NFSv4.0 specification refers to these two 59 The NFSv4.0 specification refers to these two items as an
60 "nfs_client_id4". The NFSv4.1 specification re 60 "nfs_client_id4". The NFSv4.1 specification refers to these two
61 items as a "client_owner4". 61 items as a "client_owner4".
62 62
63 NFSv4 servers tie this identifier to the princ 63 NFSv4 servers tie this identifier to the principal and security
64 flavor that the client used when presenting it 64 flavor that the client used when presenting it. Servers use this
65 principal to authorize subsequent lease modifi 65 principal to authorize subsequent lease modification operations
66 sent by the client. Effectively this principal 66 sent by the client. Effectively this principal is a third element of
67 the identifier. 67 the identifier.
68 68
69 As part of the identity presented to servers, 69 As part of the identity presented to servers, a good
70 "co_ownerid" string has several important prop 70 "co_ownerid" string has several important properties:
71 71
72 - The "co_ownerid" string identifies the cli 72 - The "co_ownerid" string identifies the client during reboot
73 recovery, therefore the string is persiste 73 recovery, therefore the string is persistent across client
74 reboots. 74 reboots.
75 - The "co_ownerid" string helps servers dist 75 - The "co_ownerid" string helps servers distinguish the client
76 from others, therefore the string is globa 76 from others, therefore the string is globally unique. Note
77 that there is no central authority that as 77 that there is no central authority that assigns "co_ownerid"
78 strings. 78 strings.
79 - Because it often appears on the network in 79 - Because it often appears on the network in the clear, the
80 "co_ownerid" string does not reveal privat 80 "co_ownerid" string does not reveal private information about
81 the client itself. 81 the client itself.
82 - The content of the "co_ownerid" string is 82 - The content of the "co_ownerid" string is set and unchanging
83 before the client attempts NFSv4 mounts af 83 before the client attempts NFSv4 mounts after a restart.
84 - The NFSv4 protocol places a 1024-byte limi 84 - The NFSv4 protocol places a 1024-byte limit on the size of the
85 "co_ownerid" string. 85 "co_ownerid" string.
86 86
87 Protecting NFSv4 lease state 87 Protecting NFSv4 lease state
88 ---------------------------- 88 ----------------------------
89 89
90 NFSv4 servers utilize the "client_owner4" as d 90 NFSv4 servers utilize the "client_owner4" as described above to
91 assign a unique lease to each client. Under th 91 assign a unique lease to each client. Under this scheme, there are
92 circumstances where clients can interfere with 92 circumstances where clients can interfere with each other. This is
93 referred to as "lease stealing". 93 referred to as "lease stealing".
94 94
95 If distinct clients present the same "co_owner 95 If distinct clients present the same "co_ownerid" string and use
96 the same principal (for example, AUTH_SYS and 96 the same principal (for example, AUTH_SYS and UID 0), a server is
97 unable to tell that the clients are not the sa 97 unable to tell that the clients are not the same. Each distinct
98 client presents a different boot verifier, so 98 client presents a different boot verifier, so it appears to the
99 server as if there is one client that is reboo 99 server as if there is one client that is rebooting frequently.
100 Neither client can maintain open or lock state 100 Neither client can maintain open or lock state in this scenario.
101 101
102 If distinct clients present the same "co_owner 102 If distinct clients present the same "co_ownerid" string and use
103 distinct principals, the server is likely to a 103 distinct principals, the server is likely to allow the first client
104 to operate normally but reject subsequent clie 104 to operate normally but reject subsequent clients with the same
105 "co_ownerid" string. 105 "co_ownerid" string.
106 106
107 If a client's "co_ownerid" string or principal 107 If a client's "co_ownerid" string or principal are not stable,
108 state recovery after a server or client reboot 108 state recovery after a server or client reboot is not guaranteed.
109 If a client unexpectedly restarts but presents 109 If a client unexpectedly restarts but presents a different
110 "co_ownerid" string or principal to the server 110 "co_ownerid" string or principal to the server, the server orphans
111 the client's previous open and lock state. Thi 111 the client's previous open and lock state. This blocks access to
112 locked files until the server removes the orph 112 locked files until the server removes the orphaned state.
113 113
114 If the server restarts and a client presents a 114 If the server restarts and a client presents a changed "co_ownerid"
115 string or principal to the server, the server 115 string or principal to the server, the server will not allow the
116 client to reclaim its open and lock state, and 116 client to reclaim its open and lock state, and may give those locks
117 to other clients in the meantime. This is refe 117 to other clients in the meantime. This is referred to as "lock
118 stealing". 118 stealing".
119 119
120 Lease stealing and lock stealing increase the 120 Lease stealing and lock stealing increase the potential for denial
121 of service and in rare cases even data corrupt 121 of service and in rare cases even data corruption.
122 122
123 Selecting an appropriate client identifier 123 Selecting an appropriate client identifier
124 ------------------------------------------ 124 ------------------------------------------
125 125
126 By default, the Linux NFSv4 client implementat 126 By default, the Linux NFSv4 client implementation constructs its
127 "co_ownerid" string starting with the words "L 127 "co_ownerid" string starting with the words "Linux NFS" followed by
128 the client's UTS node name (the same node name 128 the client's UTS node name (the same node name, incidentally, that
129 is used as the "machine name" in an AUTH_SYS c 129 is used as the "machine name" in an AUTH_SYS credential). In small
130 deployments, this construction is usually adeq 130 deployments, this construction is usually adequate. Often, however,
131 the node name by itself is not adequately uniq 131 the node name by itself is not adequately unique, and can change
132 unexpectedly. Problematic situations include: 132 unexpectedly. Problematic situations include:
133 133
134 - NFS-root (diskless) clients, where the loc !! 134 - NFS-root (diskless) clients, where the local DCHP server (or
135 equivalent) does not provide a unique host 135 equivalent) does not provide a unique host name.
136 136
137 - "Containers" within a single Linux host. 137 - "Containers" within a single Linux host. If each container has
138 a separate network namespace, but does not 138 a separate network namespace, but does not use the UTS namespace
139 to provide a unique host name, then there 139 to provide a unique host name, then there can be multiple NFS
140 client instances with the same host name. 140 client instances with the same host name.
141 141
142 - Clients across multiple administrative dom 142 - Clients across multiple administrative domains that access a
143 common NFS server. If hostnames are not as 143 common NFS server. If hostnames are not assigned centrally
144 then uniqueness cannot be guaranteed unles 144 then uniqueness cannot be guaranteed unless a domain name is
145 included in the hostname. 145 included in the hostname.
146 146
147 Linux provides two mechanisms to add uniquenes 147 Linux provides two mechanisms to add uniqueness to its "co_ownerid"
148 string: 148 string:
149 149
150 nfs.nfs4_unique_id 150 nfs.nfs4_unique_id
151 This module parameter can set an arbitra 151 This module parameter can set an arbitrary uniquifier string
152 via the kernel command line, or when the 152 via the kernel command line, or when the "nfs" module is
153 loaded. 153 loaded.
154 154
155 /sys/fs/nfs/net/nfs_client/identifier 155 /sys/fs/nfs/net/nfs_client/identifier
156 This virtual file, available since Linux 156 This virtual file, available since Linux 5.3, is local to the
157 network namespace in which it is accesse 157 network namespace in which it is accessed and so can provide
158 distinction between network namespaces ( 158 distinction between network namespaces (containers) when the
159 hostname remains uniform. 159 hostname remains uniform.
160 160
161 Note that this file is empty on name-space cre 161 Note that this file is empty on name-space creation. If the
162 container system has access to some sort of pe 162 container system has access to some sort of per-container identity
163 then that uniquifier can be used. For example, 163 then that uniquifier can be used. For example, a uniquifier might
164 be formed at boot using the container's intern 164 be formed at boot using the container's internal identifier:
165 165
166 sha256sum /etc/machine-id | awk '{print $1 166 sha256sum /etc/machine-id | awk '{print $1}' \\
167 > /sys/fs/nfs/net/nfs_client/identifie 167 > /sys/fs/nfs/net/nfs_client/identifier
168 168
169 Security considerations 169 Security considerations
170 ----------------------- 170 -----------------------
171 171
172 The use of cryptographic security for lease ma 172 The use of cryptographic security for lease management operations
173 is strongly encouraged. 173 is strongly encouraged.
174 174
175 If NFS with Kerberos is not configured, a Linu 175 If NFS with Kerberos is not configured, a Linux NFSv4 client uses
176 AUTH_SYS and UID 0 as the principal part of it 176 AUTH_SYS and UID 0 as the principal part of its client identity.
177 This configuration is not only insecure, it in 177 This configuration is not only insecure, it increases the risk of
178 lease and lock stealing. However, it might be 178 lease and lock stealing. However, it might be the only choice for
179 client configurations that have no local persi 179 client configurations that have no local persistent storage.
180 "co_ownerid" string uniqueness and persistence 180 "co_ownerid" string uniqueness and persistence is critical in this
181 case. 181 case.
182 182
183 When a Kerberos keytab is present on a Linux N 183 When a Kerberos keytab is present on a Linux NFS client, the client
184 attempts to use one of the principals in that 184 attempts to use one of the principals in that keytab when
185 identifying itself to servers. The "sec=" moun 185 identifying itself to servers. The "sec=" mount option does not
186 control this behavior. Alternately, a single-u 186 control this behavior. Alternately, a single-user client with a
187 Kerberos principal can use that principal in p 187 Kerberos principal can use that principal in place of the client's
188 host principal. 188 host principal.
189 189
190 Using Kerberos for this purpose enables the cl 190 Using Kerberos for this purpose enables the client and server to
191 use the same lease for operations covered by a 191 use the same lease for operations covered by all "sec=" settings.
192 Additionally, the Linux NFS client uses the RP 192 Additionally, the Linux NFS client uses the RPCSEC_GSS security
193 flavor with Kerberos and the integrity QOS to 193 flavor with Kerberos and the integrity QOS to prevent in-transit
194 modification of lease modification requests. 194 modification of lease modification requests.
195 195
196 Additional notes 196 Additional notes
197 ---------------- 197 ----------------
198 The Linux NFSv4 client establishes a single le 198 The Linux NFSv4 client establishes a single lease on each NFSv4
199 server it accesses. NFSv4 mounts from a Linux 199 server it accesses. NFSv4 mounts from a Linux NFSv4 client of a
200 particular server then share that lease. 200 particular server then share that lease.
201 201
202 Once a client establishes open and lock state, 202 Once a client establishes open and lock state, the NFSv4 protocol
203 enables lease state to transition to other ser 203 enables lease state to transition to other servers, following data
204 that has been migrated. This hides data migrat 204 that has been migrated. This hides data migration completely from
205 running applications. The Linux NFSv4 client f 205 running applications. The Linux NFSv4 client facilitates state
206 migration by presenting the same "client_owner 206 migration by presenting the same "client_owner4" to all servers it
207 encounters. 207 encounters.
208 208
209 ======== 209 ========
210 See Also 210 See Also
211 ======== 211 ========
212 212
213 - nfs(5) 213 - nfs(5)
214 - kerberos(7) 214 - kerberos(7)
215 - RFC 7530 for the NFSv4.0 specification 215 - RFC 7530 for the NFSv4.0 specification
216 - RFC 8881 for the NFSv4.1 specification. 216 - RFC 8881 for the NFSv4.1 specification.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.