1 .. SPDX-License-Identifier: GPL-2.0 1 .. SPDX-License-Identifier: GPL-2.0
2 .. _iomap_porting: 2 .. _iomap_porting:
3 3
4 .. 4 ..
5 Dumb style notes to maintain the autho 5 Dumb style notes to maintain the author's sanity:
6 Please try to start sentences on separ 6 Please try to start sentences on separate lines so that
7 sentence changes don't bleed colors in 7 sentence changes don't bleed colors in diff.
8 Heading decorations are documented in 8 Heading decorations are documented in sphinx.rst.
9 9
10 ======================= 10 =======================
11 Porting Your Filesystem 11 Porting Your Filesystem
12 ======================= 12 =======================
13 13
14 .. contents:: Table of Contents 14 .. contents:: Table of Contents
15 :local: 15 :local:
16 16
17 Why Convert? 17 Why Convert?
18 ============ 18 ============
19 19
20 There are several reasons to convert a filesys 20 There are several reasons to convert a filesystem to iomap:
21 21
22 1. The classic Linux I/O path is not terribly 22 1. The classic Linux I/O path is not terribly efficient.
23 Pagecache operations lock a single base pa 23 Pagecache operations lock a single base page at a time and then call
24 into the filesystem to return a mapping fo 24 into the filesystem to return a mapping for only that page.
25 Direct I/O operations build I/O requests a 25 Direct I/O operations build I/O requests a single file block at a
26 time. 26 time.
27 This worked well enough for direct/indirec 27 This worked well enough for direct/indirect-mapped filesystems such
28 as ext2, but is very inefficient for exten 28 as ext2, but is very inefficient for extent-based filesystems such
29 as XFS. 29 as XFS.
30 30
31 2. Large folios are only supported via iomap; 31 2. Large folios are only supported via iomap; there are no plans to
32 convert the old buffer_head path to use th 32 convert the old buffer_head path to use them.
33 33
34 3. Direct access to storage on memory-like de 34 3. Direct access to storage on memory-like devices (fsdax) is only
35 supported via iomap. 35 supported via iomap.
36 36
37 4. Lower maintenance overhead for individual 37 4. Lower maintenance overhead for individual filesystem maintainers.
38 iomap handles common pagecache related ope 38 iomap handles common pagecache related operations itself, such as
39 allocating, instantiating, locking, and un 39 allocating, instantiating, locking, and unlocking of folios.
40 No ->write_begin(), ->write_end() or direc 40 No ->write_begin(), ->write_end() or direct_IO
41 address_space_operations are required to b 41 address_space_operations are required to be implemented by
42 filesystem using iomap. 42 filesystem using iomap.
43 43
44 How Do I Convert a Filesystem? 44 How Do I Convert a Filesystem?
45 ============================== 45 ==============================
46 46
47 First, add ``#include <linux/iomap.h>`` from y 47 First, add ``#include <linux/iomap.h>`` from your source code and add
48 ``select FS_IOMAP`` to your filesystem's Kconf 48 ``select FS_IOMAP`` to your filesystem's Kconfig option.
49 Build the kernel, run fstests with the ``-g al 49 Build the kernel, run fstests with the ``-g all`` option across a wide
50 variety of your filesystem's supported configu 50 variety of your filesystem's supported configurations to build a
51 baseline of which tests pass and which ones fa 51 baseline of which tests pass and which ones fail.
52 52
53 The recommended approach is first to implement 53 The recommended approach is first to implement ``->iomap_begin`` (and
54 ``->iomap_end`` if necessary) to allow iomap t 54 ``->iomap_end`` if necessary) to allow iomap to obtain a read-only
55 mapping of a file range. 55 mapping of a file range.
56 In most cases, this is a relatively trivial co 56 In most cases, this is a relatively trivial conversion of the existing
57 ``get_block()`` function for read-only mapping 57 ``get_block()`` function for read-only mappings.
58 ``FS_IOC_FIEMAP`` is a good first target becau 58 ``FS_IOC_FIEMAP`` is a good first target because it is trivial to
59 implement support for it and then to determine 59 implement support for it and then to determine that the extent map
60 iteration is correct from userspace. 60 iteration is correct from userspace.
61 If FIEMAP is returning the correct information 61 If FIEMAP is returning the correct information, it's a good sign that
62 other read-only mapping operations will do the 62 other read-only mapping operations will do the right thing.
63 63
64 Next, modify the filesystem's ``get_block(crea 64 Next, modify the filesystem's ``get_block(create = false)``
65 implementation to use the new ``->iomap_begin` 65 implementation to use the new ``->iomap_begin`` implementation to map
66 file space for selected read operations. 66 file space for selected read operations.
67 Hide behind a debugging knob the ability to sw 67 Hide behind a debugging knob the ability to switch on the iomap mapping
68 functions for selected call paths. 68 functions for selected call paths.
69 It is necessary to write some code to fill out 69 It is necessary to write some code to fill out the bufferhead-based
70 mapping information from the ``iomap`` structu 70 mapping information from the ``iomap`` structure, but the new functions
71 can be tested without needing to implement any 71 can be tested without needing to implement any iomap APIs.
72 72
73 Once the read-only functions are working like 73 Once the read-only functions are working like this, convert each high
74 level file operation one by one to use iomap n 74 level file operation one by one to use iomap native APIs instead of
75 going through ``get_block()``. 75 going through ``get_block()``.
76 Done one at a time, regressions should be self 76 Done one at a time, regressions should be self evident.
77 You *do* have a regression test baseline for f 77 You *do* have a regression test baseline for fstests, right?
78 It is suggested to convert swap file activatio 78 It is suggested to convert swap file activation, ``SEEK_DATA``, and
79 ``SEEK_HOLE`` before tackling the I/O paths. 79 ``SEEK_HOLE`` before tackling the I/O paths.
80 A likely complexity at this point will be conv 80 A likely complexity at this point will be converting the buffered read
81 I/O path because of bufferheads. 81 I/O path because of bufferheads.
82 The buffered read I/O paths doesn't need to be 82 The buffered read I/O paths doesn't need to be converted yet, though the
83 direct I/O read path should be converted in th 83 direct I/O read path should be converted in this phase.
84 84
85 At this point, you should look over your ``->i 85 At this point, you should look over your ``->iomap_begin`` function.
86 If it switches between large blocks of code ba 86 If it switches between large blocks of code based on dispatching of the
87 ``flags`` argument, you should consider breaki 87 ``flags`` argument, you should consider breaking it up into
88 per-operation iomap ops with smaller, more coh 88 per-operation iomap ops with smaller, more cohesive functions.
89 XFS is a good example of this. 89 XFS is a good example of this.
90 90
91 The next thing to do is implement ``get_blocks 91 The next thing to do is implement ``get_blocks(create == true)``
92 functionality in the ``->iomap_begin``/``->iom 92 functionality in the ``->iomap_begin``/``->iomap_end`` methods.
93 It is strongly recommended to create separate 93 It is strongly recommended to create separate mapping functions and
94 iomap ops for write operations. 94 iomap ops for write operations.
95 Then convert the direct I/O write path to ioma 95 Then convert the direct I/O write path to iomap, and start running fsx
96 w/ DIO enabled in earnest on filesystem. 96 w/ DIO enabled in earnest on filesystem.
97 This will flush out lots of data integrity cor 97 This will flush out lots of data integrity corner case bugs that the new
98 write mapping implementation introduces. 98 write mapping implementation introduces.
99 99
100 Now, convert any remaining file operations to 100 Now, convert any remaining file operations to call the iomap functions.
101 This will get the entire filesystem using the 101 This will get the entire filesystem using the new mapping functions, and
102 they should largely be debugged and working co 102 they should largely be debugged and working correctly after this step.
103 103
104 Most likely at this point, the buffered read a 104 Most likely at this point, the buffered read and write paths will still
105 need to be converted. 105 need to be converted.
106 The mapping functions should all work correctl 106 The mapping functions should all work correctly, so all that needs to be
107 done is rewriting all the code that interfaces 107 done is rewriting all the code that interfaces with bufferheads to
108 interface with iomap and folios. 108 interface with iomap and folios.
109 It is much easier first to get regular file I/ 109 It is much easier first to get regular file I/O (without any fancy
110 features like fscrypt, fsverity, compression, 110 features like fscrypt, fsverity, compression, or data=journaling)
111 converted to use iomap. 111 converted to use iomap.
112 Some of those fancy features (fscrypt and comp 112 Some of those fancy features (fscrypt and compression) aren't
113 implemented yet in iomap. 113 implemented yet in iomap.
114 For unjournalled filesystems that use the page 114 For unjournalled filesystems that use the pagecache for symbolic links
115 and directories, you might also try converting 115 and directories, you might also try converting their handling to iomap.
116 116
117 The rest is left as an exercise for the reader 117 The rest is left as an exercise for the reader, as it will be different
118 for every filesystem. 118 for every filesystem.
119 If you encounter problems, email the people an 119 If you encounter problems, email the people and lists in
120 ``get_maintainers.pl`` for help. 120 ``get_maintainers.pl`` for help.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.