1 .. SPDX-License-Identifier: GPL-2.0
2
3 ==============================
4 Network Filesystem Caching API
5 ==============================
6
7 Fscache provides an API by which a network fil
8 caching facilities. The API is arranged aroun
9
10 (1) A cache is logically organised into volum
11 within those volumes.
12
13 (2) Volumes and data storage objects are repr
14 cookie.
15
16 (3) Cookies have keys that distinguish them f
17
18 (4) Cookies have coherency data that allows a
19 cached data is still valid.
20
21 (5) I/O is done asynchronously where possible
22
23 This API is used by::
24
25 #include <linux/fscache.h>.
26
27 .. This document contains the following sectio
28
29 (1) Overview
30 (2) Volume registration
31 (3) Data file registration
32 (4) Declaring a cookie to be in use
33 (5) Resizing a data file (truncation)
34 (6) Data I/O API
35 (7) Data file coherency
36 (8) Data file invalidation
37 (9) Write back resource management
38 (10) Caching of local modifications
39 (11) Page release and invalidation
40
41
42 Overview
43 ========
44
45 The fscache hierarchy is organised on two leve
46 point of view. The upper level represents "vo
47 represents "data storage objects". These are
48 cookie, hereafter referred to as "volume cooki
49
50 A network filesystem acquires a volume cookie
51 which represents all the information that defi
52 or server address, volume ID or share name).
53 printable string that can be used as a directo
54 and shouldn't begin with a '.'). The maximum
55 maximum size of a filename component (allowing
56 its own purposes).
57
58 A filesystem would typically have a volume coo
59
60 The filesystem then acquires a cookie for each
61 object key. Object keys are binary blobs and
62 their parent volume. The cache backend is res
63 blob into something it can use and may employ
64 improve its ability to find an object. This i
65 filesystem.
66
67 A filesystem would typically have a cookie for
68 in iget and relinquish it when evicting the co
69
70 Once it has a cookie, the filesystem needs to
71 This causes fscache to send the cache backend
72 for the cookie in the background, to check its
73 mark the object as being under modification.
74
75 A filesystem would typically "use" the cookie
76 unuse it in file release and it needs to use t
77 truncate the cookie locally. It *also* needs
78 pagecache becomes dirty and unuse it when writ
79 slightly tricky, and provision is made for it.
80
81 When performing a read, write or resize on a c
82 begin an operation. This copies the resources
83 extra pins into the cache to stop cache withdr
84 structures being used. The actual operation c
85 invalidations can be detected upon completion.
86
87 The filesystem is expected to use netfslib to
88 actually required and it can use the fscache I
89
90
91 Volume Registration
92 ===================
93
94 The first step for a network filesystem is to
95 volume it wants to access::
96
97 struct fscache_volume *
98 fscache_acquire_volume(const char *vol
99 const char *cac
100 const void *coh
101 size_t coherenc
102
103 This function creates a volume cookie with the
104 and notes the coherency data.
105
106 The volume key must be a printable string with
107 should begin with the name of the filesystem a
108 characters. It should uniquely represent the
109 what's stored in the cache.
110
111 The caller may also specify the name of the ca
112 fscache will look up or create a cache cookie
113 of that name if it is online or comes online.
114 it will use the first cache that comes to hand
115
116 The specified coherency data is stored in the
117 against coherency data stored on disk. The da
118 is provided. If the coherency data doesn't ma
119 be invalidated.
120
121 This function can return errors such as EBUSY
122 use by an acquired volume or ENOMEM if an allo
123 also return a NULL volume cookie if fscache is
124 pass a NULL cookie to any function that takes
125 cause that function to do nothing.
126
127
128 When the network filesystem has finished with
129 by calling::
130
131 void fscache_relinquish_volume(struct
132 const v
133 bool in
134
135 This will cause the volume to be committed or
136 coherency data will be set to the value suppli
137 must match the length specified when the volum
138 data cookies obtained in this volume must be r
139 relinquished.
140
141
142 Data File Registration
143 ======================
144
145 Once it has a volume cookie, a network filesys
146 cookie for data storage::
147
148 struct fscache_cookie *
149 fscache_acquire_cookie(struct fscache_
150 u8 advice,
151 const void *ind
152 size_t index_ke
153 const void *aux
154 size_t aux_data
155 loff_t object_s
156
157 This creates the cookie in the volume using th
158 key is a binary blob of the given length and m
159 This is saved into the cookie. There are no r
160 its length shouldn't exceed about three quarte
161 to allow for encoding.
162
163 The caller should also pass in a piece of cohe
164 of size aux_data_len will be allocated and the
165 assumed that the size is invariant over time.
166 check the validity of data in the cache. Func
167 coherency data can be updated.
168
169 The file size of the object being cached shoul
170 used to trim the data and will be stored with
171
172 This function never returns an error, though i
173 allocation failure or if fscache is not enable
174 volume cookie and pass the NULL cookie returne
175 This will cause that function to do nothing.
176
177
178 When the network filesystem has finished with
179 by calling::
180
181 void fscache_relinquish_cookie(struct
182 bool re
183
184 This will cause fscache to either commit the s
185 delete it.
186
187
188 Marking A Cookie In-Use
189 =======================
190
191 Once a cookie has been acquired by a network f
192 tell fscache when it intends to use the cookie
193 and should say when it has finished with it (t
194
195 void fscache_use_cookie(struct fscache
196 bool will_modi
197 void fscache_unuse_cookie(struct fscac
198 const void *
199 const loff_t
200
201 The *use* function tells fscache that it will
202 indicate if the user is intending to modify th
203 done, this will trigger the cache backend to g
204 needs to access/store data in the cache. This
205 so may not be complete by the time the functio
206
207 The *unuse* function indicates that a filesyst
208 It optionally updates the stored coherency dat
209 decreases the in-use counter. When the last u
210 scheduled for garbage collection. If not reus
211 resources will be released to reduce system re
212
213 A cookie must be marked in-use before it can b
214 resize - and an in-use mark must be kept whils
215 pagecache in order to avoid an oops due to try
216 exit.
217
218 Note that in-use marks are cumulative. For ea
219 in-use, it must be unused.
220
221
222 Resizing A Data File (Truncation)
223 =================================
224
225 If a network filesystem file is resized locall
226 should be called to notify the cache::
227
228 void fscache_resize_cookie(struct fsca
229 loff_t new_
230
231 The caller must have first marked the cookie i
232 size are passed in and the cache is synchronou
233 be called from ``->setattr()`` inode operation
234
235
236 Data I/O API
237 ============
238
239 To do data I/O operations directly through a c
240 are available::
241
242 int fscache_begin_read_operation(struc
243 struc
244 int fscache_read(struct netfs_cache_re
245 loff_t start_pos,
246 struct iov_iter *iter
247 enum netfs_read_from_
248 netfs_io_terminated_t
249 void *term_func_priv)
250 int fscache_write(struct netfs_cache_r
251 loff_t start_pos,
252 struct iov_iter *ite
253 netfs_io_terminated_
254 void *term_func_priv
255
256 The *begin* function sets up an operation, att
257 the cache resources block from the cookie. As
258 (for instance, it will return -ENOBUFS if give
259 nothing), then one of the other two functions
260
261 The *read* and *write* functions initiate a di
262 previously set up cache resources block, an in
263 position, and an I/O iterator that describes b
264 data.
265
266 The read function also takes a parameter to in
267 partially populated region (a hole) in the dis
268 it, skip over an initial hole and place zeros
269
270 The read and write functions can be given an o
271 will be run on completion::
272
273 typedef
274 void (*netfs_io_terminated_t)(void *pr
275 bool was
276
277 If a termination function is given, the operat
278 and the termination function will be called up
279 operation will be run synchronously. Note tha
280 possible for the operation to complete before
281
282 Both the read and write functions end the oper
283 detaching any pinned resources.
284
285 The read operation will fail with ESTALE if in
286 operation was ongoing.
287
288
289 Data File Coherency
290 ===================
291
292 To request an update of the coherency data and
293 following should be called::
294
295 void fscache_update_cookie(struct fsca
296 const void
297 const loff_
298
299 This will update the cookie's coherency data a
300
301
302 Data File Invalidation
303 ======================
304
305 Sometimes it will be necessary to invalidate a
306 Typically this will be necessary when the serv
307 of a remote third-party change - at which poin
308 away the state and cached data that it had for
309 server.
310
311 To indicate that a cache object should be inva
312 called::
313
314 void fscache_invalidate(struct fscache
315 const void *au
316 loff_t size,
317 unsigned int f
318
319 This increases the invalidation counter in the
320 reads to fail with -ESTALE, sets the coherency
321 information supplied, blocks new I/O on the co
322 go and get rid of the old data.
323
324 Invalidation runs asynchronously in a worker t
325 too much.
326
327
328 Write-Back Resource Management
329 ==============================
330
331 To write data to the cache from network filesy
332 resources required need to be pinned at the po
333 instance when the page is marked dirty) as it'
334 a thread that's exiting.
335
336 The following facilities are provided to manag
337
338 * An inode flag, ``I_PINNING_FSCACHE_WB``, is
339 in-use is held on the cookie for this inode
340 the inode lock is held.
341
342 * A flag, ``unpinned_fscache_wb`` is placed i
343 struct that gets set if ``__writeback_singl
344 ``I_PINNING_FSCACHE_WB`` because all the di
345
346 To support this, the following functions are p
347
348 bool fscache_dirty_folio(struct addres
349 struct folio
350 struct fscach
351 void fscache_unpin_writeback(struct wr
352 struct fs
353 void fscache_clear_inode_writeback(str
354 str
355 con
356
357 The *set* function is intended to be called fr
358 ``dirty_folio`` address space operation. If `
359 set, it sets that flag and increments the use
360 must already have called ``fscache_use_cookie(
361
362 The *unpin* function is intended to be called
363 ``write_inode`` superblock operation. It clea
364 the cookie if unpinned_fscache_wb is set in th
365
366 The *clear* function is intended to be called
367 superblock operation. It must be called *afte
368 ``truncate_inode_pages_final()``, but *before*
369 up any hanging ``I_PINNING_FSCACHE_WB``. It a
370 be updated.
371
372
373 Caching of Local Modifications
374 ==============================
375
376 If a network filesystem has locally modified d
377 cache, it needs to mark the pages to indicate
378 if the mark is already present, it needs to wa
379 (presumably due to an already in-progress oper
380 competing DIO writes to the same storage in th
381
382 Firstly, the netfs should determine if caching
383 like::
384
385 bool caching = fscache_cookie_enabled(
386
387 If caching is to be attempted, pages should be
388 the following functions provided by the netfs
389
390 void set_page_fscache(struct page *pag
391 void wait_on_page_fscache(struct page
392 int wait_on_page_fscache_killable(stru
393
394 Once all the pages in the span are marked, the
395 schedule a write of that region::
396
397 void fscache_write_to_cache(struct fsc
398 struct add
399 loff_t sta
400 netfs_io_t
401 void *term
402 bool cachi
403
404 And if an error occurs before that point is re
405 by calling::
406
407 void fscache_clear_page_bits(struct ad
408 loff_t st
409 bool cach
410
411 In these functions, a pointer to the mapping t
412 attached is passed in and start and len indica
413 going to be written (it doesn't have to align
414 but it does have to align to DIO boundaries on
415 caching parameter indicates if caching should
416 functions do nothing.
417
418 The write function takes some additional param
419 the cache object to be written to, i_size indi
420 and term_func indicates an optional completion
421 term_func_priv will be passed, along with the
422
423 Note that the write function will always run a
424 the pages upon completion before calling term_
425
426
427 Page Release and Invalidation
428 =============================
429
430 Fscache keeps track of whether we have any dat
431 object we've just created. It knows it doesn'
432 has done a write and then the page it wrote fr
433 after which it *has* to look in the cache.
434
435 To inform fscache that a page might now be in
436 should be called from the ``release_folio`` ad
437
438 void fscache_note_page_release(struct
439
440 if the page has been released (ie. release_fol
441
442 Page release and page invalidation should also
443 page to say that a DIO write is underway from
444
445 void wait_on_page_fscache(struct page
446 int wait_on_page_fscache_killable(stru
447
448
449 API Function Reference
450 ======================
451
452 .. kernel-doc:: include/linux/fscache.h
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.