1 .. SPDX-License-Identifier: GPL-2.0-only
2
3 dm-vdo
4 ======
5
6 The dm-vdo (virtual data optimizer) device map
7 block-level deduplication, compression, and th
8 mapper target, it can add these features to th
9 with any file system. The vdo target does not
10 corruption, relying instead on integrity prote
11 it. It is strongly recommended that lvm be use
12 lvmvdo(7).
13
14 Userspace component
15 ===================
16
17 Formatting a vdo volume requires the use of th
18 at:
19
20 https://github.com/dm-vdo/vdo/
21
22 In most cases, a vdo target will recover from
23 next time it is started. In cases where it enc
24 error (either during normal operation or crash
25 enter or come up in read-only mode. Because re
26 data-loss, a positive action must be taken to
27 mode. The 'vdoforcerebuild' tool, available fr
28 prepare a read-only vdo to exit read-only mode
29 the vdo target will rebuild its metadata the n
30 started. Although some data may be lost, the r
31 internally consistent and the target will be w
32
33 The repo also contains additional userspace to
34 inspect a vdo target's on-disk metadata. Fortu
35 rarely needed except by dm-vdo developers.
36
37 Metadata requirements
38 =====================
39
40 Each vdo volume reserves 3GB of space for meta
41 its configuration. It is helpful to check that
42 deduplication and compression is not cancelled
43 requirements. An estimation of the space saved
44 be computed with the vdo estimator tool, which
45
46 https://github.com/dm-vdo/vdoestimator/
47
48 Target interface
49 ================
50
51 Table line
52 ----------
53
54 ::
55
56 <offset> <logical device size> vdo V4
57 <storage device size> <minimum I/O siz
58 <block map era length> [optional argum
59
60
61 Required parameters:
62
63 offset:
64 The offset, in sectors, at whi
65 space begins.
66
67 logical device size:
68 The size of the device which t
69 in sectors. Must match the cur
70 volume.
71
72 storage device:
73 The device holding the vdo vol
74
75 storage device size:
76 The size of the device holding
77 of 4096-byte blocks. Must matc
78 volume.
79
80 minimum I/O size:
81 The minimum I/O size for this
82 bytes. Valid values are 512 or
83 is 4096.
84
85 block map cache size:
86 The size of the block map cach
87 blocks. The minimum and recomm
88 If the logical thread count is
89 must be at least 4096 blocks p
90
91 block map era length:
92 The speed with which the block
93 modified block map pages. A sm
94 reduce the amount of time spen
95 increased block map writes dur
96 maximum and recommended value
97 is 1.
98
99 Optional parameters:
100 --------------------
101 Some or all of these parameters may be specifi
102
103 Thread related parameters:
104
105 Different categories of work are assigned to s
106 the number of threads in each group can be con
107
108 If <hash>, <logical>, and <physical> are all s
109 all three thread types will be handled by a si
110 values are non-zero, all of them must be non-z
111
112 ack:
113 The number of threads used to
114 completing a bio calls an arbi
115 outside the vdo volume, thread
116 volume to continue processing
117 completion is slow. The defaul
118
119 bio:
120 The number of threads used to
121 storage. Threads of this type
122 continue processing requests e
123 slow. The default is 4.
124
125 bioRotationInterval:
126 The number of bios to enqueue
127 switching to the next thread.
128 than 0 and not more than 1024;
129
130 cpu:
131 The number of threads used to
132 as hashing and compression. Th
133
134 hash:
135 The number of threads used to
136 deduplication based on the has
137 default is 0.
138
139 logical:
140 The number of threads used to
141 based on the logical address o
142 is 0; the maximum is 60.
143
144 physical:
145 The number of threads used to
146 underlying storage device. At
147 the vdo is chosen; the vdo sto
148 enough to have at least 1 slab
149 default is 0; the maximum is 1
150
151 Miscellaneous parameters:
152
153 maxDiscard:
154 The maximum size of discard bi
155 blocks. I/O requests to a vdo
156 into 4096-byte blocks, and pro
157 However, discard requests to a
158 automatically split to a large
159 4096-byte blocks in a single b
160 at a time. Increasing this val
161 performance, at the cost of in
162 individual discard requests. T
163 the maximum is UINT_MAX / 4096
164
165 deduplication:
166 Whether deduplication is enabl
167 acceptable values are 'on' and
168
169 compression:
170 Whether compression is enabled
171 acceptable values are 'on' and
172
173 Device modification
174 -------------------
175
176 A modified table may be loaded into a running,
177 The modifications will take effect when the de
178 modifiable parameters are <logical device size
179 <maxDiscard>, <compression>, and <deduplicatio
180
181 If the logical device size or physical device
182 successful resume vdo will store the new value
183 startups. These two parameters may not be decr
184 size may not exceed 4 PB. The physical device
185 least 32832 4096-byte blocks if at all, and mu
186 underlying storage device. Additionally, when
187 slab size is chosen: the physical device size
188 size which provides 8192 slabs, and each incre
189 add at least one new slab.
190
191 Examples:
192
193 Start a previously-formatted vdo volume with 1
194 physical space, storing to /dev/dm-1 which has
195
196 ::
197
198 dmsetup create vdo0 --table \
199 "0 2097152 vdo V4 /dev/dm-1 262144 409
200
201 Grow the logical size to 4 GB.
202
203 ::
204
205 dmsetup reload vdo0 --table \
206 "0 8388608 vdo V4 /dev/dm-1 262144 409
207 dmsetup resume vdo0
208
209 Grow the physical size to 2 GB.
210
211 ::
212
213 dmsetup reload vdo0 --table \
214 "0 8388608 vdo V4 /dev/dm-1 524288 409
215 dmsetup resume vdo0
216
217 Grow the physical size by 1 GB more and increa
218
219 ::
220
221 dmsetup reload vdo0 --table \
222 "0 10485760 vdo V4 /dev/dm-1 786432 40
223 dmsetup resume vdo0
224
225 Stop the vdo volume.
226
227 ::
228
229 dmsetup remove vdo0
230
231 Start the vdo volume again. Note that the logi
232 must still match, but other parameters can cha
233
234 ::
235
236 dmsetup create vdo1 --table \
237 "0 10485760 vdo V4 /dev/dm-1 786432 51
238
239 Messages
240 --------
241 All vdo devices accept messages in the form:
242
243 ::
244
245 dmsetup message <target-name> 0 <messa
246
247 The messages are:
248
249 stats:
250 Outputs the current view of th
251 by the vdostats userspace prog
252 buffer.
253
254 config:
255 Outputs useful vdo configurati
256 by users who want to recreate
257 want to know the creation conf
258
259 dump:
260 Dumps many internal structures
261 not always safe to run, so it
262 a hung vdo. Optional parameter
263 dump are:
264
265 viopool: The pool of I
266 pools: A synonym of 'v
267 vdo: Most of the struc
268 queues: Basic informat
269 threads: A synonym of
270 default: Equivalent to
271 all: All of the above.
272
273 dump-on-shutdown:
274 Perform a default dump next ti
275
276
277 Status
278 ------
279
280 ::
281
282 <device> <operating mode> <in recovery> <i
283 <compression state> <physical blocks used>
284
285 device:
286 The name of the vdo volume.
287
288 operating mode:
289 The current operating mode of
290 'normal', 'recovering' (the vo
291 with its metadata and is attem
292 'read-only' (an error has occu
293 volume to only support read op
294
295 in recovery:
296 Whether the vdo volume is curr
297 values may be 'recovering' or
298 recovering.
299
300 index state:
301 The current state of the dedup
302 volume; values may be 'closed'
303 'offline', 'online', 'opening'
304
305 compression state:
306 The current state of compressi
307 may be 'offline' and 'online'.
308
309 used physical blocks:
310 The number of physical blocks
311
312 total physical blocks:
313 The total number of physical b
314 the difference between this va
315 <used physical blocks> is the
316 volume has left before being f
317
318 Memory Requirements
319 ===================
320
321 A vdo target requires a fixed 38 MB of RAM alo
322 that scale with the target:
323
324 - 1.15 MB of RAM for each 1 MB of configured b
325 block map cache requires a minimum of 150 MB
326 - 1.6 MB of RAM for each 1 TB of logical space
327 - 268 MB of RAM for each 1 TB of physical stor
328
329 The deduplication index requires additional me
330 size of the deduplication window. For dense in
331 GB of RAM per 1 TB of window. For sparse index
332 of RAM per 10 TB of window. The index configur
333 is formatted and may not be modified.
334
335 Module Parameters
336 =================
337
338 The vdo driver has a numeric parameter 'log_le
339 verbosity of logging from the driver. The defa
340 (LOGLEVEL_INFO and more severe messages).
341
342 Run-time Usage
343 ==============
344
345 When using dm-vdo, it is important to be aware
346 behavior differs from other storage targets.
347
348 - There is no guarantee that over-writes of ex
349 Because the underlying storage may be multip
350 an existing block generally requires a vdo t
351 available.
352
353 - When blocks are no longer in use, sending a
354 blocks lets the vdo release references for t
355 thinly provisioned, discarding unused blocks
356 target from running out of space. However, d
357 duplicate blocks, no discard request for any
358 guaranteed to reclaim space.
359
360 - Assuming the underlying storage properly imp
361 is resilient against crashes, however, unflu
362 persist after a crash.
363
364 - Each write to a vdo target entails a signifi
365 However, much of the work is paralellizable.
366 achieve better throughput at higher I/O dept
367 requests in parallel.
368
369 Tuning
370 ======
371
372 The vdo device has many options, and it can be
373 choices without perfect knowledge of the workl
374 configuration options must be set when a vdo t
375 be changed without shutting it down completely
376 changed while the target is active. Ideally, t
377 workloads should be performed before deploying
378 environments.
379
380 The most important value to adjust is the bloc
381 service a request for any logical address, a v
382 the block map which holds the relevant mapping
383 Performance will suffer when the working set d
384 default, a vdo allocates 128 MB of metadata ca
385 efficient access to 100 GB of logical space at
386 up proportionally for larger working sets.
387
388 The logical and physical thread counts should
389 thread controls a disjoint section of the bloc
390 threads increase parallelism and can increase
391 control a disjoint section of the data blocks,
392 threads can also increase throughput. However,
393 resources and increase contention.
394
395 Bio submission threads control the parallelism
396 the underlying storage; fewer threads mean the
397 reorder I/O requests for performance benefit,
398 request has to wait longer before being submit
399
400 Bio acknowledgment threads are used for finish
401 done on dedicated threads since the amount of
402 bio's callback can not be controlled by the vd
403 is sufficient but additional threads may be be
404 bios have CPU-heavy callbacks.
405
406 CPU threads are used for hashing and for compr
407 compression enabled, more threads may result i
408
409 Hash threads are used to sort active requests
410 they should deduplicate; the most CPU intensiv
411 threads are comparison of 4096-byte data block
412 hash thread is sufficient.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.