1 .. SPDX-License-Identifier: GPL-2.0 1 .. SPDX-License-Identifier: GPL-2.0
2 2
3 .. _fsverity: 3 .. _fsverity:
4 4
5 ============================================== 5 =======================================================
6 fs-verity: read-only file-based authenticity p 6 fs-verity: read-only file-based authenticity protection
7 ============================================== 7 =======================================================
8 8
9 Introduction 9 Introduction
10 ============ 10 ============
11 11
12 fs-verity (``fs/verity/``) is a support layer 12 fs-verity (``fs/verity/``) is a support layer that filesystems can
13 hook into to support transparent integrity and 13 hook into to support transparent integrity and authenticity protection
14 of read-only files. Currently, it is supporte 14 of read-only files. Currently, it is supported by the ext4, f2fs, and
15 btrfs filesystems. Like fscrypt, not too much 15 btrfs filesystems. Like fscrypt, not too much filesystem-specific
16 code is needed to support fs-verity. 16 code is needed to support fs-verity.
17 17
18 fs-verity is similar to `dm-verity 18 fs-verity is similar to `dm-verity
19 <https://www.kernel.org/doc/Documentation/devi 19 <https://www.kernel.org/doc/Documentation/device-mapper/verity.txt>`_
20 but works on files rather than block devices. 20 but works on files rather than block devices. On regular files on
21 filesystems supporting fs-verity, userspace ca 21 filesystems supporting fs-verity, userspace can execute an ioctl that
22 causes the filesystem to build a Merkle tree f 22 causes the filesystem to build a Merkle tree for the file and persist
23 it to a filesystem-specific location associate 23 it to a filesystem-specific location associated with the file.
24 24
25 After this, the file is made readonly, and all 25 After this, the file is made readonly, and all reads from the file are
26 automatically verified against the file's Merk 26 automatically verified against the file's Merkle tree. Reads of any
27 corrupted data, including mmap reads, will fai 27 corrupted data, including mmap reads, will fail.
28 28
29 Userspace can use another ioctl to retrieve th 29 Userspace can use another ioctl to retrieve the root hash (actually
30 the "fs-verity file digest", which is a hash t 30 the "fs-verity file digest", which is a hash that includes the Merkle
31 tree root hash) that fs-verity is enforcing fo 31 tree root hash) that fs-verity is enforcing for the file. This ioctl
32 executes in constant time, regardless of the f 32 executes in constant time, regardless of the file size.
33 33
34 fs-verity is essentially a way to hash a file 34 fs-verity is essentially a way to hash a file in constant time,
35 subject to the caveat that reads which would v 35 subject to the caveat that reads which would violate the hash will
36 fail at runtime. 36 fail at runtime.
37 37
38 Use cases 38 Use cases
39 ========= 39 =========
40 40
41 By itself, fs-verity only provides integrity p 41 By itself, fs-verity only provides integrity protection, i.e.
42 detection of accidental (non-malicious) corrup 42 detection of accidental (non-malicious) corruption.
43 43
44 However, because fs-verity makes retrieving th 44 However, because fs-verity makes retrieving the file hash extremely
45 efficient, it's primarily meant to be used as 45 efficient, it's primarily meant to be used as a tool to support
46 authentication (detection of malicious modific 46 authentication (detection of malicious modifications) or auditing
47 (logging file hashes before use). 47 (logging file hashes before use).
48 48
49 A standard file hash could be used instead of 49 A standard file hash could be used instead of fs-verity. However,
50 this is inefficient if the file is large and o 50 this is inefficient if the file is large and only a small portion may
51 be accessed. This is often the case for Andro 51 be accessed. This is often the case for Android application package
52 (APK) files, for example. These typically con 52 (APK) files, for example. These typically contain many translations,
53 classes, and other resources that are infreque 53 classes, and other resources that are infrequently or even never
54 accessed on a particular device. It would be 54 accessed on a particular device. It would be slow and wasteful to
55 read and hash the entire file before starting 55 read and hash the entire file before starting the application.
56 56
57 Unlike an ahead-of-time hash, fs-verity also r 57 Unlike an ahead-of-time hash, fs-verity also re-verifies data each
58 time it's paged in. This ensures that malicio 58 time it's paged in. This ensures that malicious disk firmware can't
59 undetectably change the contents of the file a 59 undetectably change the contents of the file at runtime.
60 60
61 fs-verity does not replace or obsolete dm-veri 61 fs-verity does not replace or obsolete dm-verity. dm-verity should
62 still be used on read-only filesystems. fs-ve 62 still be used on read-only filesystems. fs-verity is for files that
63 must live on a read-write filesystem because t 63 must live on a read-write filesystem because they are independently
64 updated and potentially user-installed, so dm- 64 updated and potentially user-installed, so dm-verity cannot be used.
65 65
66 fs-verity does not mandate a particular scheme 66 fs-verity does not mandate a particular scheme for authenticating its
67 file hashes. (Similarly, dm-verity does not m 67 file hashes. (Similarly, dm-verity does not mandate a particular
68 scheme for authenticating its block device roo 68 scheme for authenticating its block device root hashes.) Options for
69 authenticating fs-verity file hashes include: 69 authenticating fs-verity file hashes include:
70 70
71 - Trusted userspace code. Often, the userspac 71 - Trusted userspace code. Often, the userspace code that accesses
72 files can be trusted to authenticate them. 72 files can be trusted to authenticate them. Consider e.g. an
73 application that wants to authenticate data 73 application that wants to authenticate data files before using them,
74 or an application loader that is part of the 74 or an application loader that is part of the operating system (which
75 is already authenticated in a different way, 75 is already authenticated in a different way, such as by being loaded
76 from a read-only partition that uses dm-veri 76 from a read-only partition that uses dm-verity) and that wants to
77 authenticate applications before loading the 77 authenticate applications before loading them. In these cases, this
78 trusted userspace code can authenticate a fi 78 trusted userspace code can authenticate a file's contents by
79 retrieving its fs-verity digest using `FS_IO 79 retrieving its fs-verity digest using `FS_IOC_MEASURE_VERITY`_, then
80 verifying a signature of it using any usersp 80 verifying a signature of it using any userspace cryptographic
81 library that supports digital signatures. 81 library that supports digital signatures.
82 82
83 - Integrity Measurement Architecture (IMA). I 83 - Integrity Measurement Architecture (IMA). IMA supports fs-verity
84 file digests as an alternative to its tradit 84 file digests as an alternative to its traditional full file digests.
85 "IMA appraisal" enforces that files contain 85 "IMA appraisal" enforces that files contain a valid, matching
86 signature in their "security.ima" extended a 86 signature in their "security.ima" extended attribute, as controlled
87 by the IMA policy. For more information, se 87 by the IMA policy. For more information, see the IMA documentation.
88 88
89 - Trusted userspace code in combination with ` 89 - Trusted userspace code in combination with `Built-in signature
90 verification`_. This approach should be use 90 verification`_. This approach should be used only with great care.
91 91
92 User API 92 User API
93 ======== 93 ========
94 94
95 FS_IOC_ENABLE_VERITY 95 FS_IOC_ENABLE_VERITY
96 -------------------- 96 --------------------
97 97
98 The FS_IOC_ENABLE_VERITY ioctl enables fs-veri 98 The FS_IOC_ENABLE_VERITY ioctl enables fs-verity on a file. It takes
99 in a pointer to a struct fsverity_enable_arg, 99 in a pointer to a struct fsverity_enable_arg, defined as
100 follows:: 100 follows::
101 101
102 struct fsverity_enable_arg { 102 struct fsverity_enable_arg {
103 __u32 version; 103 __u32 version;
104 __u32 hash_algorithm; 104 __u32 hash_algorithm;
105 __u32 block_size; 105 __u32 block_size;
106 __u32 salt_size; 106 __u32 salt_size;
107 __u64 salt_ptr; 107 __u64 salt_ptr;
108 __u32 sig_size; 108 __u32 sig_size;
109 __u32 __reserved1; 109 __u32 __reserved1;
110 __u64 sig_ptr; 110 __u64 sig_ptr;
111 __u64 __reserved2[11]; 111 __u64 __reserved2[11];
112 }; 112 };
113 113
114 This structure contains the parameters of the 114 This structure contains the parameters of the Merkle tree to build for
115 the file. It must be initialized as follows: 115 the file. It must be initialized as follows:
116 116
117 - ``version`` must be 1. 117 - ``version`` must be 1.
118 - ``hash_algorithm`` must be the identifier fo 118 - ``hash_algorithm`` must be the identifier for the hash algorithm to
119 use for the Merkle tree, such as FS_VERITY_H 119 use for the Merkle tree, such as FS_VERITY_HASH_ALG_SHA256. See
120 ``include/uapi/linux/fsverity.h`` for the li 120 ``include/uapi/linux/fsverity.h`` for the list of possible values.
121 - ``block_size`` is the Merkle tree block size 121 - ``block_size`` is the Merkle tree block size, in bytes. In Linux
122 v6.3 and later, this can be any power of 2 b 122 v6.3 and later, this can be any power of 2 between (inclusively)
123 1024 and the minimum of the system page size 123 1024 and the minimum of the system page size and the filesystem
124 block size. In earlier versions, the page s 124 block size. In earlier versions, the page size was the only allowed
125 value. 125 value.
126 - ``salt_size`` is the size of the salt in byt 126 - ``salt_size`` is the size of the salt in bytes, or 0 if no salt is
127 provided. The salt is a value that is prepe 127 provided. The salt is a value that is prepended to every hashed
128 block; it can be used to personalize the has 128 block; it can be used to personalize the hashing for a particular
129 file or device. Currently the maximum salt 129 file or device. Currently the maximum salt size is 32 bytes.
130 - ``salt_ptr`` is the pointer to the salt, or 130 - ``salt_ptr`` is the pointer to the salt, or NULL if no salt is
131 provided. 131 provided.
132 - ``sig_size`` is the size of the builtin sign 132 - ``sig_size`` is the size of the builtin signature in bytes, or 0 if no
133 builtin signature is provided. Currently th 133 builtin signature is provided. Currently the builtin signature is
134 (somewhat arbitrarily) limited to 16128 byte 134 (somewhat arbitrarily) limited to 16128 bytes.
135 - ``sig_ptr`` is the pointer to the builtin s 135 - ``sig_ptr`` is the pointer to the builtin signature, or NULL if no
136 builtin signature is provided. A builtin si 136 builtin signature is provided. A builtin signature is only needed
137 if the `Built-in signature verification`_ fe 137 if the `Built-in signature verification`_ feature is being used. It
138 is not needed for IMA appraisal, and it is n 138 is not needed for IMA appraisal, and it is not needed if the file
139 signature is being handled entirely in users 139 signature is being handled entirely in userspace.
140 - All reserved fields must be zeroed. 140 - All reserved fields must be zeroed.
141 141
142 FS_IOC_ENABLE_VERITY causes the filesystem to 142 FS_IOC_ENABLE_VERITY causes the filesystem to build a Merkle tree for
143 the file and persist it to a filesystem-specif 143 the file and persist it to a filesystem-specific location associated
144 with the file, then mark the file as a verity 144 with the file, then mark the file as a verity file. This ioctl may
145 take a long time to execute on large files, an 145 take a long time to execute on large files, and it is interruptible by
146 fatal signals. 146 fatal signals.
147 147
148 FS_IOC_ENABLE_VERITY checks for write access t 148 FS_IOC_ENABLE_VERITY checks for write access to the inode. However,
149 it must be executed on an O_RDONLY file descri 149 it must be executed on an O_RDONLY file descriptor and no processes
150 can have the file open for writing. Attempts 150 can have the file open for writing. Attempts to open the file for
151 writing while this ioctl is executing will fai 151 writing while this ioctl is executing will fail with ETXTBSY. (This
152 is necessary to guarantee that no writable fil 152 is necessary to guarantee that no writable file descriptors will exist
153 after verity is enabled, and to guarantee that 153 after verity is enabled, and to guarantee that the file's contents are
154 stable while the Merkle tree is being built ov 154 stable while the Merkle tree is being built over it.)
155 155
156 On success, FS_IOC_ENABLE_VERITY returns 0, an 156 On success, FS_IOC_ENABLE_VERITY returns 0, and the file becomes a
157 verity file. On failure (including the case o 157 verity file. On failure (including the case of interruption by a
158 fatal signal), no changes are made to the file 158 fatal signal), no changes are made to the file.
159 159
160 FS_IOC_ENABLE_VERITY can fail with the followi 160 FS_IOC_ENABLE_VERITY can fail with the following errors:
161 161
162 - ``EACCES``: the process does not have write 162 - ``EACCES``: the process does not have write access to the file
163 - ``EBADMSG``: the builtin signature is malfor 163 - ``EBADMSG``: the builtin signature is malformed
164 - ``EBUSY``: this ioctl is already running on 164 - ``EBUSY``: this ioctl is already running on the file
165 - ``EEXIST``: the file already has verity enab 165 - ``EEXIST``: the file already has verity enabled
166 - ``EFAULT``: the caller provided inaccessible 166 - ``EFAULT``: the caller provided inaccessible memory
167 - ``EFBIG``: the file is too large to enable v 167 - ``EFBIG``: the file is too large to enable verity on
168 - ``EINTR``: the operation was interrupted by 168 - ``EINTR``: the operation was interrupted by a fatal signal
169 - ``EINVAL``: unsupported version, hash algori 169 - ``EINVAL``: unsupported version, hash algorithm, or block size; or
170 reserved bits are set; or the file descripto 170 reserved bits are set; or the file descriptor refers to neither a
171 regular file nor a directory. 171 regular file nor a directory.
172 - ``EISDIR``: the file descriptor refers to a 172 - ``EISDIR``: the file descriptor refers to a directory
173 - ``EKEYREJECTED``: the builtin signature does 173 - ``EKEYREJECTED``: the builtin signature doesn't match the file
174 - ``EMSGSIZE``: the salt or builtin signature 174 - ``EMSGSIZE``: the salt or builtin signature is too long
175 - ``ENOKEY``: the ".fs-verity" keyring doesn't 175 - ``ENOKEY``: the ".fs-verity" keyring doesn't contain the certificate
176 needed to verify the builtin signature 176 needed to verify the builtin signature
177 - ``ENOPKG``: fs-verity recognizes the hash al 177 - ``ENOPKG``: fs-verity recognizes the hash algorithm, but it's not
178 available in the kernel's crypto API as curr 178 available in the kernel's crypto API as currently configured (e.g.
179 for SHA-512, missing CONFIG_CRYPTO_SHA512). 179 for SHA-512, missing CONFIG_CRYPTO_SHA512).
180 - ``ENOTTY``: this type of filesystem does not 180 - ``ENOTTY``: this type of filesystem does not implement fs-verity
181 - ``EOPNOTSUPP``: the kernel was not configure 181 - ``EOPNOTSUPP``: the kernel was not configured with fs-verity
182 support; or the filesystem superblock has no 182 support; or the filesystem superblock has not had the 'verity'
183 feature enabled on it; or the filesystem doe 183 feature enabled on it; or the filesystem does not support fs-verity
184 on this file. (See `Filesystem support`_.) 184 on this file. (See `Filesystem support`_.)
185 - ``EPERM``: the file is append-only; or, a bu 185 - ``EPERM``: the file is append-only; or, a builtin signature is
186 required and one was not provided. 186 required and one was not provided.
187 - ``EROFS``: the filesystem is read-only 187 - ``EROFS``: the filesystem is read-only
188 - ``ETXTBSY``: someone has the file open for w 188 - ``ETXTBSY``: someone has the file open for writing. This can be the
189 caller's file descriptor, another open file 189 caller's file descriptor, another open file descriptor, or the file
190 reference held by a writable memory map. 190 reference held by a writable memory map.
191 191
192 FS_IOC_MEASURE_VERITY 192 FS_IOC_MEASURE_VERITY
193 --------------------- 193 ---------------------
194 194
195 The FS_IOC_MEASURE_VERITY ioctl retrieves the 195 The FS_IOC_MEASURE_VERITY ioctl retrieves the digest of a verity file.
196 The fs-verity file digest is a cryptographic d 196 The fs-verity file digest is a cryptographic digest that identifies
197 the file contents that are being enforced on r 197 the file contents that are being enforced on reads; it is computed via
198 a Merkle tree and is different from a traditio 198 a Merkle tree and is different from a traditional full-file digest.
199 199
200 This ioctl takes in a pointer to a variable-le 200 This ioctl takes in a pointer to a variable-length structure::
201 201
202 struct fsverity_digest { 202 struct fsverity_digest {
203 __u16 digest_algorithm; 203 __u16 digest_algorithm;
204 __u16 digest_size; /* input/output 204 __u16 digest_size; /* input/output */
205 __u8 digest[]; 205 __u8 digest[];
206 }; 206 };
207 207
208 ``digest_size`` is an input/output field. On 208 ``digest_size`` is an input/output field. On input, it must be
209 initialized to the number of bytes allocated f 209 initialized to the number of bytes allocated for the variable-length
210 ``digest`` field. 210 ``digest`` field.
211 211
212 On success, 0 is returned and the kernel fills 212 On success, 0 is returned and the kernel fills in the structure as
213 follows: 213 follows:
214 214
215 - ``digest_algorithm`` will be the hash algori 215 - ``digest_algorithm`` will be the hash algorithm used for the file
216 digest. It will match ``fsverity_enable_arg 216 digest. It will match ``fsverity_enable_arg::hash_algorithm``.
217 - ``digest_size`` will be the size of the dige 217 - ``digest_size`` will be the size of the digest in bytes, e.g. 32
218 for SHA-256. (This can be redundant with `` 218 for SHA-256. (This can be redundant with ``digest_algorithm``.)
219 - ``digest`` will be the actual bytes of the d 219 - ``digest`` will be the actual bytes of the digest.
220 220
221 FS_IOC_MEASURE_VERITY is guaranteed to execute 221 FS_IOC_MEASURE_VERITY is guaranteed to execute in constant time,
222 regardless of the size of the file. 222 regardless of the size of the file.
223 223
224 FS_IOC_MEASURE_VERITY can fail with the follow 224 FS_IOC_MEASURE_VERITY can fail with the following errors:
225 225
226 - ``EFAULT``: the caller provided inaccessible 226 - ``EFAULT``: the caller provided inaccessible memory
227 - ``ENODATA``: the file is not a verity file 227 - ``ENODATA``: the file is not a verity file
228 - ``ENOTTY``: this type of filesystem does not 228 - ``ENOTTY``: this type of filesystem does not implement fs-verity
229 - ``EOPNOTSUPP``: the kernel was not configure 229 - ``EOPNOTSUPP``: the kernel was not configured with fs-verity
230 support, or the filesystem superblock has no 230 support, or the filesystem superblock has not had the 'verity'
231 feature enabled on it. (See `Filesystem sup 231 feature enabled on it. (See `Filesystem support`_.)
232 - ``EOVERFLOW``: the digest is longer than the 232 - ``EOVERFLOW``: the digest is longer than the specified
233 ``digest_size`` bytes. Try providing a larg 233 ``digest_size`` bytes. Try providing a larger buffer.
234 234
235 FS_IOC_READ_VERITY_METADATA 235 FS_IOC_READ_VERITY_METADATA
236 --------------------------- 236 ---------------------------
237 237
238 The FS_IOC_READ_VERITY_METADATA ioctl reads ve 238 The FS_IOC_READ_VERITY_METADATA ioctl reads verity metadata from a
239 verity file. This ioctl is available since Li 239 verity file. This ioctl is available since Linux v5.12.
240 240
241 This ioctl allows writing a server program tha 241 This ioctl allows writing a server program that takes a verity file
242 and serves it to a client program, such that t 242 and serves it to a client program, such that the client can do its own
243 fs-verity compatible verification of the file. 243 fs-verity compatible verification of the file. This only makes sense
244 if the client doesn't trust the server and if 244 if the client doesn't trust the server and if the server needs to
245 provide the storage for the client. 245 provide the storage for the client.
246 246
247 This is a fairly specialized use case, and mos 247 This is a fairly specialized use case, and most fs-verity users won't
248 need this ioctl. 248 need this ioctl.
249 249
250 This ioctl takes in a pointer to the following 250 This ioctl takes in a pointer to the following structure::
251 251
252 #define FS_VERITY_METADATA_TYPE_MERKLE_TREE 252 #define FS_VERITY_METADATA_TYPE_MERKLE_TREE 1
253 #define FS_VERITY_METADATA_TYPE_DESCRIPTOR 253 #define FS_VERITY_METADATA_TYPE_DESCRIPTOR 2
254 #define FS_VERITY_METADATA_TYPE_SIGNATURE 254 #define FS_VERITY_METADATA_TYPE_SIGNATURE 3
255 255
256 struct fsverity_read_metadata_arg { 256 struct fsverity_read_metadata_arg {
257 __u64 metadata_type; 257 __u64 metadata_type;
258 __u64 offset; 258 __u64 offset;
259 __u64 length; 259 __u64 length;
260 __u64 buf_ptr; 260 __u64 buf_ptr;
261 __u64 __reserved; 261 __u64 __reserved;
262 }; 262 };
263 263
264 ``metadata_type`` specifies the type of metada 264 ``metadata_type`` specifies the type of metadata to read:
265 265
266 - ``FS_VERITY_METADATA_TYPE_MERKLE_TREE`` read 266 - ``FS_VERITY_METADATA_TYPE_MERKLE_TREE`` reads the blocks of the
267 Merkle tree. The blocks are returned in ord 267 Merkle tree. The blocks are returned in order from the root level
268 to the leaf level. Within each level, the b 268 to the leaf level. Within each level, the blocks are returned in
269 the same order that their hashes are themsel 269 the same order that their hashes are themselves hashed.
270 See `Merkle tree`_ for more information. 270 See `Merkle tree`_ for more information.
271 271
272 - ``FS_VERITY_METADATA_TYPE_DESCRIPTOR`` reads 272 - ``FS_VERITY_METADATA_TYPE_DESCRIPTOR`` reads the fs-verity
273 descriptor. See `fs-verity descriptor`_. 273 descriptor. See `fs-verity descriptor`_.
274 274
275 - ``FS_VERITY_METADATA_TYPE_SIGNATURE`` reads 275 - ``FS_VERITY_METADATA_TYPE_SIGNATURE`` reads the builtin signature
276 which was passed to FS_IOC_ENABLE_VERITY, if 276 which was passed to FS_IOC_ENABLE_VERITY, if any. See `Built-in
277 signature verification`_. 277 signature verification`_.
278 278
279 The semantics are similar to those of ``pread( 279 The semantics are similar to those of ``pread()``. ``offset``
280 specifies the offset in bytes into the metadat 280 specifies the offset in bytes into the metadata item to read from, and
281 ``length`` specifies the maximum number of byt 281 ``length`` specifies the maximum number of bytes to read from the
282 metadata item. ``buf_ptr`` is the pointer to 282 metadata item. ``buf_ptr`` is the pointer to the buffer to read into,
283 cast to a 64-bit integer. ``__reserved`` must 283 cast to a 64-bit integer. ``__reserved`` must be 0. On success, the
284 number of bytes read is returned. 0 is return 284 number of bytes read is returned. 0 is returned at the end of the
285 metadata item. The returned length may be les 285 metadata item. The returned length may be less than ``length``, for
286 example if the ioctl is interrupted. 286 example if the ioctl is interrupted.
287 287
288 The metadata returned by FS_IOC_READ_VERITY_ME 288 The metadata returned by FS_IOC_READ_VERITY_METADATA isn't guaranteed
289 to be authenticated against the file digest th 289 to be authenticated against the file digest that would be returned by
290 `FS_IOC_MEASURE_VERITY`_, as the metadata is e 290 `FS_IOC_MEASURE_VERITY`_, as the metadata is expected to be used to
291 implement fs-verity compatible verification an 291 implement fs-verity compatible verification anyway (though absent a
292 malicious disk, the metadata will indeed match 292 malicious disk, the metadata will indeed match). E.g. to implement
293 this ioctl, the filesystem is allowed to just 293 this ioctl, the filesystem is allowed to just read the Merkle tree
294 blocks from disk without actually verifying th 294 blocks from disk without actually verifying the path to the root node.
295 295
296 FS_IOC_READ_VERITY_METADATA can fail with the 296 FS_IOC_READ_VERITY_METADATA can fail with the following errors:
297 297
298 - ``EFAULT``: the caller provided inaccessible 298 - ``EFAULT``: the caller provided inaccessible memory
299 - ``EINTR``: the ioctl was interrupted before 299 - ``EINTR``: the ioctl was interrupted before any data was read
300 - ``EINVAL``: reserved fields were set, or ``o 300 - ``EINVAL``: reserved fields were set, or ``offset + length``
301 overflowed 301 overflowed
302 - ``ENODATA``: the file is not a verity file, 302 - ``ENODATA``: the file is not a verity file, or
303 FS_VERITY_METADATA_TYPE_SIGNATURE was reques 303 FS_VERITY_METADATA_TYPE_SIGNATURE was requested but the file doesn't
304 have a builtin signature 304 have a builtin signature
305 - ``ENOTTY``: this type of filesystem does not 305 - ``ENOTTY``: this type of filesystem does not implement fs-verity, or
306 this ioctl is not yet implemented on it 306 this ioctl is not yet implemented on it
307 - ``EOPNOTSUPP``: the kernel was not configure 307 - ``EOPNOTSUPP``: the kernel was not configured with fs-verity
308 support, or the filesystem superblock has no 308 support, or the filesystem superblock has not had the 'verity'
309 feature enabled on it. (See `Filesystem sup 309 feature enabled on it. (See `Filesystem support`_.)
310 310
311 FS_IOC_GETFLAGS 311 FS_IOC_GETFLAGS
312 --------------- 312 ---------------
313 313
314 The existing ioctl FS_IOC_GETFLAGS (which isn' 314 The existing ioctl FS_IOC_GETFLAGS (which isn't specific to fs-verity)
315 can also be used to check whether a file has f 315 can also be used to check whether a file has fs-verity enabled or not.
316 To do so, check for FS_VERITY_FL (0x00100000) 316 To do so, check for FS_VERITY_FL (0x00100000) in the returned flags.
317 317
318 The verity flag is not settable via FS_IOC_SET 318 The verity flag is not settable via FS_IOC_SETFLAGS. You must use
319 FS_IOC_ENABLE_VERITY instead, since parameters 319 FS_IOC_ENABLE_VERITY instead, since parameters must be provided.
320 320
321 statx 321 statx
322 ----- 322 -----
323 323
324 Since Linux v5.5, the statx() system call sets 324 Since Linux v5.5, the statx() system call sets STATX_ATTR_VERITY if
325 the file has fs-verity enabled. This can perf 325 the file has fs-verity enabled. This can perform better than
326 FS_IOC_GETFLAGS and FS_IOC_MEASURE_VERITY beca 326 FS_IOC_GETFLAGS and FS_IOC_MEASURE_VERITY because it doesn't require
327 opening the file, and opening verity files can 327 opening the file, and opening verity files can be expensive.
328 328
329 .. _accessing_verity_files: 329 .. _accessing_verity_files:
330 330
331 Accessing verity files 331 Accessing verity files
332 ====================== 332 ======================
333 333
334 Applications can transparently access a verity 334 Applications can transparently access a verity file just like a
335 non-verity one, with the following exceptions: 335 non-verity one, with the following exceptions:
336 336
337 - Verity files are readonly. They cannot be o 337 - Verity files are readonly. They cannot be opened for writing or
338 truncate()d, even if the file mode bits allo 338 truncate()d, even if the file mode bits allow it. Attempts to do
339 one of these things will fail with EPERM. H 339 one of these things will fail with EPERM. However, changes to
340 metadata such as owner, mode, timestamps, an 340 metadata such as owner, mode, timestamps, and xattrs are still
341 allowed, since these are not measured by fs- 341 allowed, since these are not measured by fs-verity. Verity files
342 can also still be renamed, deleted, and link 342 can also still be renamed, deleted, and linked to.
343 343
344 - Direct I/O is not supported on verity files. 344 - Direct I/O is not supported on verity files. Attempts to use direct
345 I/O on such files will fall back to buffered 345 I/O on such files will fall back to buffered I/O.
346 346
347 - DAX (Direct Access) is not supported on veri 347 - DAX (Direct Access) is not supported on verity files, because this
348 would circumvent the data verification. 348 would circumvent the data verification.
349 349
350 - Reads of data that doesn't match the verity 350 - Reads of data that doesn't match the verity Merkle tree will fail
351 with EIO (for read()) or SIGBUS (for mmap() 351 with EIO (for read()) or SIGBUS (for mmap() reads).
352 352
353 - If the sysctl "fs.verity.require_signatures" 353 - If the sysctl "fs.verity.require_signatures" is set to 1 and the
354 file is not signed by a key in the ".fs-veri 354 file is not signed by a key in the ".fs-verity" keyring, then
355 opening the file will fail. See `Built-in s 355 opening the file will fail. See `Built-in signature verification`_.
356 356
357 Direct access to the Merkle tree is not suppor 357 Direct access to the Merkle tree is not supported. Therefore, if a
358 verity file is copied, or is backed up and res 358 verity file is copied, or is backed up and restored, then it will lose
359 its "verity"-ness. fs-verity is primarily mea 359 its "verity"-ness. fs-verity is primarily meant for files like
360 executables that are managed by a package mana 360 executables that are managed by a package manager.
361 361
362 File digest computation 362 File digest computation
363 ======================= 363 =======================
364 364
365 This section describes how fs-verity hashes th 365 This section describes how fs-verity hashes the file contents using a
366 Merkle tree to produce the digest which crypto 366 Merkle tree to produce the digest which cryptographically identifies
367 the file contents. This algorithm is the same 367 the file contents. This algorithm is the same for all filesystems
368 that support fs-verity. 368 that support fs-verity.
369 369
370 Userspace only needs to be aware of this algor 370 Userspace only needs to be aware of this algorithm if it needs to
371 compute fs-verity file digests itself, e.g. in 371 compute fs-verity file digests itself, e.g. in order to sign files.
372 372
373 .. _fsverity_merkle_tree: 373 .. _fsverity_merkle_tree:
374 374
375 Merkle tree 375 Merkle tree
376 ----------- 376 -----------
377 377
378 The file contents is divided into blocks, wher 378 The file contents is divided into blocks, where the block size is
379 configurable but is usually 4096 bytes. The e 379 configurable but is usually 4096 bytes. The end of the last block is
380 zero-padded if needed. Each block is then has 380 zero-padded if needed. Each block is then hashed, producing the first
381 level of hashes. Then, the hashes in this fir 381 level of hashes. Then, the hashes in this first level are grouped
382 into 'blocksize'-byte blocks (zero-padding the 382 into 'blocksize'-byte blocks (zero-padding the ends as needed) and
383 these blocks are hashed, producing the second 383 these blocks are hashed, producing the second level of hashes. This
384 proceeds up the tree until only a single block 384 proceeds up the tree until only a single block remains. The hash of
385 this block is the "Merkle tree root hash". 385 this block is the "Merkle tree root hash".
386 386
387 If the file fits in one block and is nonempty, 387 If the file fits in one block and is nonempty, then the "Merkle tree
388 root hash" is simply the hash of the single da 388 root hash" is simply the hash of the single data block. If the file
389 is empty, then the "Merkle tree root hash" is 389 is empty, then the "Merkle tree root hash" is all zeroes.
390 390
391 The "blocks" here are not necessarily the same 391 The "blocks" here are not necessarily the same as "filesystem blocks".
392 392
393 If a salt was specified, then it's zero-padded 393 If a salt was specified, then it's zero-padded to the closest multiple
394 of the input size of the hash algorithm's comp 394 of the input size of the hash algorithm's compression function, e.g.
395 64 bytes for SHA-256 or 128 bytes for SHA-512. 395 64 bytes for SHA-256 or 128 bytes for SHA-512. The padded salt is
396 prepended to every data or Merkle tree block t 396 prepended to every data or Merkle tree block that is hashed.
397 397
398 The purpose of the block padding is to cause e 398 The purpose of the block padding is to cause every hash to be taken
399 over the same amount of data, which simplifies 399 over the same amount of data, which simplifies the implementation and
400 keeps open more possibilities for hardware acc 400 keeps open more possibilities for hardware acceleration. The purpose
401 of the salt padding is to make the salting "fr 401 of the salt padding is to make the salting "free" when the salted hash
402 state is precomputed, then imported for each h 402 state is precomputed, then imported for each hash.
403 403
404 Example: in the recommended configuration of S 404 Example: in the recommended configuration of SHA-256 and 4K blocks,
405 128 hash values fit in each block. Thus, each 405 128 hash values fit in each block. Thus, each level of the Merkle
406 tree is approximately 128 times smaller than t 406 tree is approximately 128 times smaller than the previous, and for
407 large files the Merkle tree's size converges t 407 large files the Merkle tree's size converges to approximately 1/127 of
408 the original file size. However, for small fi 408 the original file size. However, for small files, the padding is
409 significant, making the space overhead proport 409 significant, making the space overhead proportionally more.
410 410
411 .. _fsverity_descriptor: 411 .. _fsverity_descriptor:
412 412
413 fs-verity descriptor 413 fs-verity descriptor
414 -------------------- 414 --------------------
415 415
416 By itself, the Merkle tree root hash is ambigu 416 By itself, the Merkle tree root hash is ambiguous. For example, it
417 can't a distinguish a large file from a small 417 can't a distinguish a large file from a small second file whose data
418 is exactly the top-level hash block of the fir 418 is exactly the top-level hash block of the first file. Ambiguities
419 also arise from the convention of padding to t 419 also arise from the convention of padding to the next block boundary.
420 420
421 To solve this problem, the fs-verity file dige 421 To solve this problem, the fs-verity file digest is actually computed
422 as a hash of the following structure, which co 422 as a hash of the following structure, which contains the Merkle tree
423 root hash as well as other fields such as the 423 root hash as well as other fields such as the file size::
424 424
425 struct fsverity_descriptor { 425 struct fsverity_descriptor {
426 __u8 version; /* must be 426 __u8 version; /* must be 1 */
427 __u8 hash_algorithm; /* Merkle 427 __u8 hash_algorithm; /* Merkle tree hash algorithm */
428 __u8 log_blocksize; /* log2 of 428 __u8 log_blocksize; /* log2 of size of data and tree blocks */
429 __u8 salt_size; /* size of 429 __u8 salt_size; /* size of salt in bytes; 0 if none */
430 __le32 __reserved_0x04; /* must be 430 __le32 __reserved_0x04; /* must be 0 */
431 __le64 data_size; /* size of 431 __le64 data_size; /* size of file the Merkle tree is built over */
432 __u8 root_hash[64]; /* Merkle 432 __u8 root_hash[64]; /* Merkle tree root hash */
433 __u8 salt[32]; /* salt pr 433 __u8 salt[32]; /* salt prepended to each hashed block */
434 __u8 __reserved[144]; /* must be 434 __u8 __reserved[144]; /* must be 0's */
435 }; 435 };
436 436
437 Built-in signature verification 437 Built-in signature verification
438 =============================== 438 ===============================
439 439
440 CONFIG_FS_VERITY_BUILTIN_SIGNATURES=y adds sup 440 CONFIG_FS_VERITY_BUILTIN_SIGNATURES=y adds supports for in-kernel
441 verification of fs-verity builtin signatures. 441 verification of fs-verity builtin signatures.
442 442
443 **IMPORTANT**! Please take great care before 443 **IMPORTANT**! Please take great care before using this feature.
444 It is not the only way to do signatures with f 444 It is not the only way to do signatures with fs-verity, and the
445 alternatives (such as userspace signature veri 445 alternatives (such as userspace signature verification, and IMA
446 appraisal) can be much better. It's also easy 446 appraisal) can be much better. It's also easy to fall into a trap
447 of thinking this feature solves more problems 447 of thinking this feature solves more problems than it actually does.
448 448
449 Enabling this option adds the following: 449 Enabling this option adds the following:
450 450
451 1. At boot time, the kernel creates a keyring 451 1. At boot time, the kernel creates a keyring named ".fs-verity". The
452 root user can add trusted X.509 certificate 452 root user can add trusted X.509 certificates to this keyring using
453 the add_key() system call. 453 the add_key() system call.
454 454
455 2. `FS_IOC_ENABLE_VERITY`_ accepts a pointer t 455 2. `FS_IOC_ENABLE_VERITY`_ accepts a pointer to a PKCS#7 formatted
456 detached signature in DER format of the fil 456 detached signature in DER format of the file's fs-verity digest.
457 On success, the ioctl persists the signatur 457 On success, the ioctl persists the signature alongside the Merkle
458 tree. Then, any time the file is opened, t 458 tree. Then, any time the file is opened, the kernel verifies the
459 file's actual digest against this signature 459 file's actual digest against this signature, using the certificates
460 in the ".fs-verity" keyring. 460 in the ".fs-verity" keyring.
461 461
462 3. A new sysctl "fs.verity.require_signatures" 462 3. A new sysctl "fs.verity.require_signatures" is made available.
463 When set to 1, the kernel requires that all 463 When set to 1, the kernel requires that all verity files have a
464 correctly signed digest as described in (2) 464 correctly signed digest as described in (2).
465 465
466 The data that the signature as described in (2 466 The data that the signature as described in (2) must be a signature of
467 is the fs-verity file digest in the following 467 is the fs-verity file digest in the following format::
468 468
469 struct fsverity_formatted_digest { 469 struct fsverity_formatted_digest {
470 char magic[8]; /* 470 char magic[8]; /* must be "FSVerity" */
471 __le16 digest_algorithm; 471 __le16 digest_algorithm;
472 __le16 digest_size; 472 __le16 digest_size;
473 __u8 digest[]; 473 __u8 digest[];
474 }; 474 };
475 475
476 That's it. It should be emphasized again that 476 That's it. It should be emphasized again that fs-verity builtin
477 signatures are not the only way to do signatur 477 signatures are not the only way to do signatures with fs-verity. See
478 `Use cases`_ for an overview of ways in which 478 `Use cases`_ for an overview of ways in which fs-verity can be used.
479 fs-verity builtin signatures have some major l 479 fs-verity builtin signatures have some major limitations that should
480 be carefully considered before using them: 480 be carefully considered before using them:
481 481
482 - Builtin signature verification does *not* ma 482 - Builtin signature verification does *not* make the kernel enforce
483 that any files actually have fs-verity enabl 483 that any files actually have fs-verity enabled. Thus, it is not a
484 complete authentication policy. Currently, 484 complete authentication policy. Currently, if it is used, the only
485 way to complete the authentication policy is 485 way to complete the authentication policy is for trusted userspace
486 code to explicitly check whether files have 486 code to explicitly check whether files have fs-verity enabled with a
487 signature before they are accessed. (With 487 signature before they are accessed. (With
488 fs.verity.require_signatures=1, just checkin 488 fs.verity.require_signatures=1, just checking whether fs-verity is
489 enabled suffices.) But, in this case the tr 489 enabled suffices.) But, in this case the trusted userspace code
490 could just store the signature alongside the 490 could just store the signature alongside the file and verify it
491 itself using a cryptographic library, instea 491 itself using a cryptographic library, instead of using this feature.
492 492
493 - A file's builtin signature can only be set a 493 - A file's builtin signature can only be set at the same time that
494 fs-verity is being enabled on the file. Cha 494 fs-verity is being enabled on the file. Changing or deleting the
495 builtin signature later requires re-creating 495 builtin signature later requires re-creating the file.
496 496
497 - Builtin signature verification uses the same 497 - Builtin signature verification uses the same set of public keys for
498 all fs-verity enabled files on the system. 498 all fs-verity enabled files on the system. Different keys cannot be
499 trusted for different files; each key is all 499 trusted for different files; each key is all or nothing.
500 500
501 - The sysctl fs.verity.require_signatures appl 501 - The sysctl fs.verity.require_signatures applies system-wide.
502 Setting it to 1 only works when all users of 502 Setting it to 1 only works when all users of fs-verity on the system
503 agree that it should be set to 1. This limi 503 agree that it should be set to 1. This limitation can prevent
504 fs-verity from being used in cases where it 504 fs-verity from being used in cases where it would be helpful.
505 505
506 - Builtin signature verification can only use 506 - Builtin signature verification can only use signature algorithms
507 that are supported by the kernel. For examp 507 that are supported by the kernel. For example, the kernel does not
508 yet support Ed25519, even though this is oft 508 yet support Ed25519, even though this is often the signature
509 algorithm that is recommended for new crypto 509 algorithm that is recommended for new cryptographic designs.
510 510
511 - fs-verity builtin signatures are in PKCS#7 f 511 - fs-verity builtin signatures are in PKCS#7 format, and the public
512 keys are in X.509 format. These formats are 512 keys are in X.509 format. These formats are commonly used,
513 including by some other kernel features (whi 513 including by some other kernel features (which is why the fs-verity
514 builtin signatures use them), and are very f 514 builtin signatures use them), and are very feature rich.
515 Unfortunately, history has shown that code t 515 Unfortunately, history has shown that code that parses and handles
516 these formats (which are from the 1990s and 516 these formats (which are from the 1990s and are based on ASN.1)
517 often has vulnerabilities as a result of the 517 often has vulnerabilities as a result of their complexity. This
518 complexity is not inherent to the cryptograp 518 complexity is not inherent to the cryptography itself.
519 519
520 fs-verity users who do not need advanced fea 520 fs-verity users who do not need advanced features of X.509 and
521 PKCS#7 should strongly consider using simple 521 PKCS#7 should strongly consider using simpler formats, such as plain
522 Ed25519 keys and signatures, and verifying s 522 Ed25519 keys and signatures, and verifying signatures in userspace.
523 523
524 fs-verity users who choose to use X.509 and 524 fs-verity users who choose to use X.509 and PKCS#7 anyway should
525 still consider that verifying those signatur 525 still consider that verifying those signatures in userspace is more
526 flexible (for other reasons mentioned earlie 526 flexible (for other reasons mentioned earlier in this document) and
527 eliminates the need to enable CONFIG_FS_VERI 527 eliminates the need to enable CONFIG_FS_VERITY_BUILTIN_SIGNATURES
528 and its associated increase in kernel attack 528 and its associated increase in kernel attack surface. In some cases
529 it can even be necessary, since advanced X.5 529 it can even be necessary, since advanced X.509 and PKCS#7 features
530 do not always work as intended with the kern 530 do not always work as intended with the kernel. For example, the
531 kernel does not check X.509 certificate vali 531 kernel does not check X.509 certificate validity times.
532 532
533 Note: IMA appraisal, which supports fs-verit 533 Note: IMA appraisal, which supports fs-verity, does not use PKCS#7
534 for its signatures, so it partially avoids t 534 for its signatures, so it partially avoids the issues discussed
535 here. IMA appraisal does use X.509. 535 here. IMA appraisal does use X.509.
536 536
537 Filesystem support 537 Filesystem support
538 ================== 538 ==================
539 539
540 fs-verity is supported by several filesystems, 540 fs-verity is supported by several filesystems, described below. The
541 CONFIG_FS_VERITY kconfig option must be enable 541 CONFIG_FS_VERITY kconfig option must be enabled to use fs-verity on
542 any of these filesystems. 542 any of these filesystems.
543 543
544 ``include/linux/fsverity.h`` declares the inte 544 ``include/linux/fsverity.h`` declares the interface between the
545 ``fs/verity/`` support layer and filesystems. 545 ``fs/verity/`` support layer and filesystems. Briefly, filesystems
546 must provide an ``fsverity_operations`` struct 546 must provide an ``fsverity_operations`` structure that provides
547 methods to read and write the verity metadata 547 methods to read and write the verity metadata to a filesystem-specific
548 location, including the Merkle tree blocks and 548 location, including the Merkle tree blocks and
549 ``fsverity_descriptor``. Filesystems must als 549 ``fsverity_descriptor``. Filesystems must also call functions in
550 ``fs/verity/`` at certain times, such as when 550 ``fs/verity/`` at certain times, such as when a file is opened or when
551 pages have been read into the pagecache. (See 551 pages have been read into the pagecache. (See `Verifying data`_.)
552 552
553 ext4 553 ext4
554 ---- 554 ----
555 555
556 ext4 supports fs-verity since Linux v5.4 and e 556 ext4 supports fs-verity since Linux v5.4 and e2fsprogs v1.45.2.
557 557
558 To create verity files on an ext4 filesystem, 558 To create verity files on an ext4 filesystem, the filesystem must have
559 been formatted with ``-O verity`` or had ``tun 559 been formatted with ``-O verity`` or had ``tune2fs -O verity`` run on
560 it. "verity" is an RO_COMPAT filesystem featu 560 it. "verity" is an RO_COMPAT filesystem feature, so once set, old
561 kernels will only be able to mount the filesys 561 kernels will only be able to mount the filesystem readonly, and old
562 versions of e2fsck will be unable to check the 562 versions of e2fsck will be unable to check the filesystem.
563 563
564 Originally, an ext4 filesystem with the "verit 564 Originally, an ext4 filesystem with the "verity" feature could only be
565 mounted when its block size was equal to the s 565 mounted when its block size was equal to the system page size
566 (typically 4096 bytes). In Linux v6.3, this l 566 (typically 4096 bytes). In Linux v6.3, this limitation was removed.
567 567
568 ext4 sets the EXT4_VERITY_FL on-disk inode fla 568 ext4 sets the EXT4_VERITY_FL on-disk inode flag on verity files. It
569 can only be set by `FS_IOC_ENABLE_VERITY`_, an 569 can only be set by `FS_IOC_ENABLE_VERITY`_, and it cannot be cleared.
570 570
571 ext4 also supports encryption, which can be us 571 ext4 also supports encryption, which can be used simultaneously with
572 fs-verity. In this case, the plaintext data i 572 fs-verity. In this case, the plaintext data is verified rather than
573 the ciphertext. This is necessary in order to 573 the ciphertext. This is necessary in order to make the fs-verity file
574 digest meaningful, since every file is encrypt 574 digest meaningful, since every file is encrypted differently.
575 575
576 ext4 stores the verity metadata (Merkle tree a 576 ext4 stores the verity metadata (Merkle tree and fsverity_descriptor)
577 past the end of the file, starting at the firs 577 past the end of the file, starting at the first 64K boundary beyond
578 i_size. This approach works because (a) verit 578 i_size. This approach works because (a) verity files are readonly,
579 and (b) pages fully beyond i_size aren't visib 579 and (b) pages fully beyond i_size aren't visible to userspace but can
580 be read/written internally by ext4 with only s 580 be read/written internally by ext4 with only some relatively small
581 changes to ext4. This approach avoids having 581 changes to ext4. This approach avoids having to depend on the
582 EA_INODE feature and on rearchitecturing ext4' 582 EA_INODE feature and on rearchitecturing ext4's xattr support to
583 support paging multi-gigabyte xattrs into memo 583 support paging multi-gigabyte xattrs into memory, and to support
584 encrypting xattrs. Note that the verity metad 584 encrypting xattrs. Note that the verity metadata *must* be encrypted
585 when the file is, since it contains hashes of 585 when the file is, since it contains hashes of the plaintext data.
586 586
587 ext4 only allows verity on extent-based files. 587 ext4 only allows verity on extent-based files.
588 588
589 f2fs 589 f2fs
590 ---- 590 ----
591 591
592 f2fs supports fs-verity since Linux v5.4 and f 592 f2fs supports fs-verity since Linux v5.4 and f2fs-tools v1.11.0.
593 593
594 To create verity files on an f2fs filesystem, 594 To create verity files on an f2fs filesystem, the filesystem must have
595 been formatted with ``-O verity``. 595 been formatted with ``-O verity``.
596 596
597 f2fs sets the FADVISE_VERITY_BIT on-disk inode 597 f2fs sets the FADVISE_VERITY_BIT on-disk inode flag on verity files.
598 It can only be set by `FS_IOC_ENABLE_VERITY`_, 598 It can only be set by `FS_IOC_ENABLE_VERITY`_, and it cannot be
599 cleared. 599 cleared.
600 600
601 Like ext4, f2fs stores the verity metadata (Me 601 Like ext4, f2fs stores the verity metadata (Merkle tree and
602 fsverity_descriptor) past the end of the file, 602 fsverity_descriptor) past the end of the file, starting at the first
603 64K boundary beyond i_size. See explanation f 603 64K boundary beyond i_size. See explanation for ext4 above.
604 Moreover, f2fs supports at most 4096 bytes of 604 Moreover, f2fs supports at most 4096 bytes of xattr entries per inode
605 which usually wouldn't be enough for even a si 605 which usually wouldn't be enough for even a single Merkle tree block.
606 606
607 f2fs doesn't support enabling verity on files 607 f2fs doesn't support enabling verity on files that currently have
608 atomic or volatile writes pending. 608 atomic or volatile writes pending.
609 609
610 btrfs 610 btrfs
611 ----- 611 -----
612 612
613 btrfs supports fs-verity since Linux v5.15. V 613 btrfs supports fs-verity since Linux v5.15. Verity-enabled inodes are
614 marked with a RO_COMPAT inode flag, and the ve 614 marked with a RO_COMPAT inode flag, and the verity metadata is stored
615 in separate btree items. 615 in separate btree items.
616 616
617 Implementation details 617 Implementation details
618 ====================== 618 ======================
619 619
620 Verifying data 620 Verifying data
621 -------------- 621 --------------
622 622
623 fs-verity ensures that all reads of a verity f 623 fs-verity ensures that all reads of a verity file's data are verified,
624 regardless of which syscall is used to do the 624 regardless of which syscall is used to do the read (e.g. mmap(),
625 read(), pread()) and regardless of whether it' 625 read(), pread()) and regardless of whether it's the first read or a
626 later read (unless the later read can return c 626 later read (unless the later read can return cached data that was
627 already verified). Below, we describe how fil 627 already verified). Below, we describe how filesystems implement this.
628 628
629 Pagecache 629 Pagecache
630 ~~~~~~~~~ 630 ~~~~~~~~~
631 631
632 For filesystems using Linux's pagecache, the ` 632 For filesystems using Linux's pagecache, the ``->read_folio()`` and
633 ``->readahead()`` methods must be modified to 633 ``->readahead()`` methods must be modified to verify folios before
634 they are marked Uptodate. Merely hooking ``-> 634 they are marked Uptodate. Merely hooking ``->read_iter()`` would be
635 insufficient, since ``->read_iter()`` is not u 635 insufficient, since ``->read_iter()`` is not used for memory maps.
636 636
637 Therefore, fs/verity/ provides the function fs 637 Therefore, fs/verity/ provides the function fsverity_verify_blocks()
638 which verifies data that has been read into th 638 which verifies data that has been read into the pagecache of a verity
639 inode. The containing folio must still be loc 639 inode. The containing folio must still be locked and not Uptodate, so
640 it's not yet readable by userspace. As needed 640 it's not yet readable by userspace. As needed to do the verification,
641 fsverity_verify_blocks() will call back into t 641 fsverity_verify_blocks() will call back into the filesystem to read
642 hash blocks via fsverity_operations::read_merk 642 hash blocks via fsverity_operations::read_merkle_tree_page().
643 643
644 fsverity_verify_blocks() returns false if veri 644 fsverity_verify_blocks() returns false if verification failed; in this
645 case, the filesystem must not set the folio Up 645 case, the filesystem must not set the folio Uptodate. Following this,
646 as per the usual Linux pagecache behavior, att 646 as per the usual Linux pagecache behavior, attempts by userspace to
647 read() from the part of the file containing th 647 read() from the part of the file containing the folio will fail with
648 EIO, and accesses to the folio within a memory 648 EIO, and accesses to the folio within a memory map will raise SIGBUS.
649 649
650 In principle, verifying a data block requires 650 In principle, verifying a data block requires verifying the entire
651 path in the Merkle tree from the data block to 651 path in the Merkle tree from the data block to the root hash.
652 However, for efficiency the filesystem may cac 652 However, for efficiency the filesystem may cache the hash blocks.
653 Therefore, fsverity_verify_blocks() only ascen 653 Therefore, fsverity_verify_blocks() only ascends the tree reading hash
654 blocks until an already-verified hash block is 654 blocks until an already-verified hash block is seen. It then verifies
655 the path to that block. 655 the path to that block.
656 656
657 This optimization, which is also used by dm-ve 657 This optimization, which is also used by dm-verity, results in
658 excellent sequential read performance. This i 658 excellent sequential read performance. This is because usually (e.g.
659 127 in 128 times for 4K blocks and SHA-256) th 659 127 in 128 times for 4K blocks and SHA-256) the hash block from the
660 bottom level of the tree will already be cache 660 bottom level of the tree will already be cached and checked from
661 reading a previous data block. However, rando 661 reading a previous data block. However, random reads perform worse.
662 662
663 Block device based filesystems 663 Block device based filesystems
664 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 664 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
665 665
666 Block device based filesystems (e.g. ext4 and 666 Block device based filesystems (e.g. ext4 and f2fs) in Linux also use
667 the pagecache, so the above subsection applies 667 the pagecache, so the above subsection applies too. However, they
668 also usually read many data blocks from a file 668 also usually read many data blocks from a file at once, grouped into a
669 structure called a "bio". To make it easier f 669 structure called a "bio". To make it easier for these types of
670 filesystems to support fs-verity, fs/verity/ a 670 filesystems to support fs-verity, fs/verity/ also provides a function
671 fsverity_verify_bio() which verifies all data 671 fsverity_verify_bio() which verifies all data blocks in a bio.
672 672
673 ext4 and f2fs also support encryption. If a v 673 ext4 and f2fs also support encryption. If a verity file is also
674 encrypted, the data must be decrypted before b 674 encrypted, the data must be decrypted before being verified. To
675 support this, these filesystems allocate a "po 675 support this, these filesystems allocate a "post-read context" for
676 each bio and store it in ``->bi_private``:: 676 each bio and store it in ``->bi_private``::
677 677
678 struct bio_post_read_ctx { 678 struct bio_post_read_ctx {
679 struct bio *bio; 679 struct bio *bio;
680 struct work_struct work; 680 struct work_struct work;
681 unsigned int cur_step; 681 unsigned int cur_step;
682 unsigned int enabled_steps; 682 unsigned int enabled_steps;
683 }; 683 };
684 684
685 ``enabled_steps`` is a bitmask that specifies 685 ``enabled_steps`` is a bitmask that specifies whether decryption,
686 verity, or both is enabled. After the bio com 686 verity, or both is enabled. After the bio completes, for each needed
687 postprocessing step the filesystem enqueues th 687 postprocessing step the filesystem enqueues the bio_post_read_ctx on a
688 workqueue, and then the workqueue work does th 688 workqueue, and then the workqueue work does the decryption or
689 verification. Finally, folios where no decryp 689 verification. Finally, folios where no decryption or verity error
690 occurred are marked Uptodate, and the folios a 690 occurred are marked Uptodate, and the folios are unlocked.
691 691
692 On many filesystems, files can contain holes. 692 On many filesystems, files can contain holes. Normally,
693 ``->readahead()`` simply zeroes hole blocks an 693 ``->readahead()`` simply zeroes hole blocks and considers the
694 corresponding data to be up-to-date; no bios a 694 corresponding data to be up-to-date; no bios are issued. To prevent
695 this case from bypassing fs-verity, filesystem 695 this case from bypassing fs-verity, filesystems use
696 fsverity_verify_blocks() to verify hole blocks 696 fsverity_verify_blocks() to verify hole blocks.
697 697
698 Filesystems also disable direct I/O on verity 698 Filesystems also disable direct I/O on verity files, since otherwise
699 direct I/O would bypass fs-verity. 699 direct I/O would bypass fs-verity.
700 700
701 Userspace utility 701 Userspace utility
702 ================= 702 =================
703 703
704 This document focuses on the kernel, but a use 704 This document focuses on the kernel, but a userspace utility for
705 fs-verity can be found at: 705 fs-verity can be found at:
706 706
707 https://git.kernel.org/pub/scm/fs/fsve 707 https://git.kernel.org/pub/scm/fs/fsverity/fsverity-utils.git
708 708
709 See the README.md file in the fsverity-utils s 709 See the README.md file in the fsverity-utils source tree for details,
710 including examples of setting up fs-verity pro 710 including examples of setting up fs-verity protected files.
711 711
712 Tests 712 Tests
713 ===== 713 =====
714 714
715 To test fs-verity, use xfstests. For example, 715 To test fs-verity, use xfstests. For example, using `kvm-xfstests
716 <https://github.com/tytso/xfstests-bld/blob/ma 716 <https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md>`_::
717 717
718 kvm-xfstests -c ext4,f2fs,btrfs -g verity 718 kvm-xfstests -c ext4,f2fs,btrfs -g verity
719 719
720 FAQ 720 FAQ
721 === 721 ===
722 722
723 This section answers frequently asked question 723 This section answers frequently asked questions about fs-verity that
724 weren't already directly answered in other par 724 weren't already directly answered in other parts of this document.
725 725
726 :Q: Why isn't fs-verity part of IMA? 726 :Q: Why isn't fs-verity part of IMA?
727 :A: fs-verity and IMA (Integrity Measurement A 727 :A: fs-verity and IMA (Integrity Measurement Architecture) have
728 different focuses. fs-verity is a filesys 728 different focuses. fs-verity is a filesystem-level mechanism for
729 hashing individual files using a Merkle tr 729 hashing individual files using a Merkle tree. In contrast, IMA
730 specifies a system-wide policy that specif 730 specifies a system-wide policy that specifies which files are
731 hashed and what to do with those hashes, s 731 hashed and what to do with those hashes, such as log them,
732 authenticate them, or add them to a measur 732 authenticate them, or add them to a measurement list.
733 733
734 IMA supports the fs-verity hashing mechani 734 IMA supports the fs-verity hashing mechanism as an alternative
735 to full file hashes, for those who want th 735 to full file hashes, for those who want the performance and
736 security benefits of the Merkle tree based 736 security benefits of the Merkle tree based hash. However, it
737 doesn't make sense to force all uses of fs 737 doesn't make sense to force all uses of fs-verity to be through
738 IMA. fs-verity already meets many users' 738 IMA. fs-verity already meets many users' needs even as a
739 standalone filesystem feature, and it's te 739 standalone filesystem feature, and it's testable like other
740 filesystem features e.g. with xfstests. 740 filesystem features e.g. with xfstests.
741 741
742 :Q: Isn't fs-verity useless because the attack 742 :Q: Isn't fs-verity useless because the attacker can just modify the
743 hashes in the Merkle tree, which is stored 743 hashes in the Merkle tree, which is stored on-disk?
744 :A: To verify the authenticity of an fs-verity 744 :A: To verify the authenticity of an fs-verity file you must verify
745 the authenticity of the "fs-verity file di 745 the authenticity of the "fs-verity file digest", which
746 incorporates the root hash of the Merkle t 746 incorporates the root hash of the Merkle tree. See `Use cases`_.
747 747
748 :Q: Isn't fs-verity useless because the attack 748 :Q: Isn't fs-verity useless because the attacker can just replace a
749 verity file with a non-verity one? 749 verity file with a non-verity one?
750 :A: See `Use cases`_. In the initial use case 750 :A: See `Use cases`_. In the initial use case, it's really trusted
751 userspace code that authenticates the file 751 userspace code that authenticates the files; fs-verity is just a
752 tool to do this job efficiently and secure 752 tool to do this job efficiently and securely. The trusted
753 userspace code will consider non-verity fi 753 userspace code will consider non-verity files to be inauthentic.
754 754
755 :Q: Why does the Merkle tree need to be stored 755 :Q: Why does the Merkle tree need to be stored on-disk? Couldn't you
756 store just the root hash? 756 store just the root hash?
757 :A: If the Merkle tree wasn't stored on-disk, 757 :A: If the Merkle tree wasn't stored on-disk, then you'd have to
758 compute the entire tree when the file is f 758 compute the entire tree when the file is first accessed, even if
759 just one byte is being read. This is a fu 759 just one byte is being read. This is a fundamental consequence of
760 how Merkle tree hashing works. To verify 760 how Merkle tree hashing works. To verify a leaf node, you need to
761 verify the whole path to the root hash, in 761 verify the whole path to the root hash, including the root node
762 (the thing which the root hash is a hash o 762 (the thing which the root hash is a hash of). But if the root
763 node isn't stored on-disk, you have to com 763 node isn't stored on-disk, you have to compute it by hashing its
764 children, and so on until you've actually 764 children, and so on until you've actually hashed the entire file.
765 765
766 That defeats most of the point of doing a 766 That defeats most of the point of doing a Merkle tree-based hash,
767 since if you have to hash the whole file a 767 since if you have to hash the whole file ahead of time anyway,
768 then you could simply do sha256(file) inst 768 then you could simply do sha256(file) instead. That would be much
769 simpler, and a bit faster too. 769 simpler, and a bit faster too.
770 770
771 It's true that an in-memory Merkle tree co 771 It's true that an in-memory Merkle tree could still provide the
772 advantage of verification on every read ra 772 advantage of verification on every read rather than just on the
773 first read. However, it would be ineffici 773 first read. However, it would be inefficient because every time a
774 hash page gets evicted (you can't pin the 774 hash page gets evicted (you can't pin the entire Merkle tree into
775 memory, since it may be very large), in or 775 memory, since it may be very large), in order to restore it you
776 again need to hash everything below it in 776 again need to hash everything below it in the tree. This again
777 defeats most of the point of doing a Merkl 777 defeats most of the point of doing a Merkle tree-based hash, since
778 a single block read could trigger re-hashi 778 a single block read could trigger re-hashing gigabytes of data.
779 779
780 :Q: But couldn't you store just the leaf nodes 780 :Q: But couldn't you store just the leaf nodes and compute the rest?
781 :A: See previous answer; this really just move 781 :A: See previous answer; this really just moves up one level, since
782 one could alternatively interpret the data 782 one could alternatively interpret the data blocks as being the
783 leaf nodes of the Merkle tree. It's true 783 leaf nodes of the Merkle tree. It's true that the tree can be
784 computed much faster if the leaf level is 784 computed much faster if the leaf level is stored rather than just
785 the data, but that's only because each lev 785 the data, but that's only because each level is less than 1% the
786 size of the level below (assuming the reco 786 size of the level below (assuming the recommended settings of
787 SHA-256 and 4K blocks). For the exact sam 787 SHA-256 and 4K blocks). For the exact same reason, by storing
788 "just the leaf nodes" you'd already be sto 788 "just the leaf nodes" you'd already be storing over 99% of the
789 tree, so you might as well simply store th 789 tree, so you might as well simply store the whole tree.
790 790
791 :Q: Can the Merkle tree be built ahead of time 791 :Q: Can the Merkle tree be built ahead of time, e.g. distributed as
792 part of a package that is installed to man 792 part of a package that is installed to many computers?
793 :A: This isn't currently supported. It was pa 793 :A: This isn't currently supported. It was part of the original
794 design, but was removed to simplify the ke 794 design, but was removed to simplify the kernel UAPI and because it
795 wasn't a critical use case. Files are usu 795 wasn't a critical use case. Files are usually installed once and
796 used many times, and cryptographic hashing 796 used many times, and cryptographic hashing is somewhat fast on
797 most modern processors. 797 most modern processors.
798 798
799 :Q: Why doesn't fs-verity support writes? 799 :Q: Why doesn't fs-verity support writes?
800 :A: Write support would be very difficult and 800 :A: Write support would be very difficult and would require a
801 completely different design, so it's well 801 completely different design, so it's well outside the scope of
802 fs-verity. Write support would require: 802 fs-verity. Write support would require:
803 803
804 - A way to maintain consistency between th 804 - A way to maintain consistency between the data and hashes,
805 including all levels of hashes, since co 805 including all levels of hashes, since corruption after a crash
806 (especially of potentially the entire fi 806 (especially of potentially the entire file!) is unacceptable.
807 The main options for solving this are da 807 The main options for solving this are data journalling,
808 copy-on-write, and log-structured volume 808 copy-on-write, and log-structured volume. But it's very hard to
809 retrofit existing filesystems with new c 809 retrofit existing filesystems with new consistency mechanisms.
810 Data journalling is available on ext4, b 810 Data journalling is available on ext4, but is very slow.
811 811
812 - Rebuilding the Merkle tree after every w 812 - Rebuilding the Merkle tree after every write, which would be
813 extremely inefficient. Alternatively, a 813 extremely inefficient. Alternatively, a different authenticated
814 dictionary structure such as an "authent 814 dictionary structure such as an "authenticated skiplist" could
815 be used. However, this would be far mor 815 be used. However, this would be far more complex.
816 816
817 Compare it to dm-verity vs. dm-integrity. 817 Compare it to dm-verity vs. dm-integrity. dm-verity is very
818 simple: the kernel just verifies read-only 818 simple: the kernel just verifies read-only data against a
819 read-only Merkle tree. In contrast, dm-in 819 read-only Merkle tree. In contrast, dm-integrity supports writes
820 but is slow, is much more complex, and doe 820 but is slow, is much more complex, and doesn't actually support
821 full-device authentication since it authen 821 full-device authentication since it authenticates each sector
822 independently, i.e. there is no "root hash 822 independently, i.e. there is no "root hash". It doesn't really
823 make sense for the same device-mapper targ 823 make sense for the same device-mapper target to support these two
824 very different cases; the same applies to 824 very different cases; the same applies to fs-verity.
825 825
826 :Q: Since verity files are immutable, why isn' 826 :Q: Since verity files are immutable, why isn't the immutable bit set?
827 :A: The existing "immutable" bit (FS_IMMUTABLE 827 :A: The existing "immutable" bit (FS_IMMUTABLE_FL) already has a
828 specific set of semantics which not only m 828 specific set of semantics which not only make the file contents
829 read-only, but also prevent the file from 829 read-only, but also prevent the file from being deleted, renamed,
830 linked to, or having its owner or mode cha 830 linked to, or having its owner or mode changed. These extra
831 properties are unwanted for fs-verity, so 831 properties are unwanted for fs-verity, so reusing the immutable
832 bit isn't appropriate. 832 bit isn't appropriate.
833 833
834 :Q: Why does the API use ioctls instead of set 834 :Q: Why does the API use ioctls instead of setxattr() and getxattr()?
835 :A: Abusing the xattr interface for basically 835 :A: Abusing the xattr interface for basically arbitrary syscalls is
836 heavily frowned upon by most of the Linux 836 heavily frowned upon by most of the Linux filesystem developers.
837 An xattr should really just be an xattr on 837 An xattr should really just be an xattr on-disk, not an API to
838 e.g. magically trigger construction of a M 838 e.g. magically trigger construction of a Merkle tree.
839 839
840 :Q: Does fs-verity support remote filesystems? 840 :Q: Does fs-verity support remote filesystems?
841 :A: So far all filesystems that have implement 841 :A: So far all filesystems that have implemented fs-verity support are
842 local filesystems, but in principle any fi 842 local filesystems, but in principle any filesystem that can store
843 per-file verity metadata can support fs-ve 843 per-file verity metadata can support fs-verity, regardless of
844 whether it's local or remote. Some filesy 844 whether it's local or remote. Some filesystems may have fewer
845 options of where to store the verity metad 845 options of where to store the verity metadata; one possibility is
846 to store it past the end of the file and " 846 to store it past the end of the file and "hide" it from userspace
847 by manipulating i_size. The data verifica 847 by manipulating i_size. The data verification functions provided
848 by ``fs/verity/`` also assume that the fil 848 by ``fs/verity/`` also assume that the filesystem uses the Linux
849 pagecache, but both local and remote files 849 pagecache, but both local and remote filesystems normally do so.
850 850
851 :Q: Why is anything filesystem-specific at all 851 :Q: Why is anything filesystem-specific at all? Shouldn't fs-verity
852 be implemented entirely at the VFS level? 852 be implemented entirely at the VFS level?
853 :A: There are many reasons why this is not pos 853 :A: There are many reasons why this is not possible or would be very
854 difficult, including the following: 854 difficult, including the following:
855 855
856 - To prevent bypassing verification, folio 856 - To prevent bypassing verification, folios must not be marked
857 Uptodate until they've been verified. C 857 Uptodate until they've been verified. Currently, each
858 filesystem is responsible for marking fo 858 filesystem is responsible for marking folios Uptodate via
859 ``->readahead()``. Therefore, currently 859 ``->readahead()``. Therefore, currently it's not possible for
860 the VFS to do the verification on its ow 860 the VFS to do the verification on its own. Changing this would
861 require significant changes to the VFS a 861 require significant changes to the VFS and all filesystems.
862 862
863 - It would require defining a filesystem-i 863 - It would require defining a filesystem-independent way to store
864 the verity metadata. Extended attribute 864 the verity metadata. Extended attributes don't work for this
865 because (a) the Merkle tree may be gigab 865 because (a) the Merkle tree may be gigabytes, but many
866 filesystems assume that all xattrs fit i 866 filesystems assume that all xattrs fit into a single 4K
867 filesystem block, and (b) ext4 and f2fs 867 filesystem block, and (b) ext4 and f2fs encryption doesn't
868 encrypt xattrs, yet the Merkle tree *mus 868 encrypt xattrs, yet the Merkle tree *must* be encrypted when the
869 file contents are, because it stores has 869 file contents are, because it stores hashes of the plaintext
870 file contents. 870 file contents.
871 871
872 So the verity metadata would have to be 872 So the verity metadata would have to be stored in an actual
873 file. Using a separate file would be ve 873 file. Using a separate file would be very ugly, since the
874 metadata is fundamentally part of the fi 874 metadata is fundamentally part of the file to be protected, and
875 it could cause problems where users coul 875 it could cause problems where users could delete the real file
876 but not the metadata file or vice versa. 876 but not the metadata file or vice versa. On the other hand,
877 having it be in the same file would brea 877 having it be in the same file would break applications unless
878 filesystems' notion of i_size were divor 878 filesystems' notion of i_size were divorced from the VFS's,
879 which would be complex and require chang 879 which would be complex and require changes to all filesystems.
880 880
881 - It's desirable that FS_IOC_ENABLE_VERITY 881 - It's desirable that FS_IOC_ENABLE_VERITY uses the filesystem's
882 transaction mechanism so that either the 882 transaction mechanism so that either the file ends up with
883 verity enabled, or no changes were made. 883 verity enabled, or no changes were made. Allowing intermediate
884 states to occur after a crash may cause 884 states to occur after a crash may cause problems.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.