>> 1 .. _admin_guide_transhuge:
>> 2
1 ============================ 3 ============================
2 Transparent Hugepage Support 4 Transparent Hugepage Support
3 ============================ 5 ============================
4 6
5 Objective 7 Objective
6 ========= 8 =========
7 9
8 Performance critical computing applications de 10 Performance critical computing applications dealing with large memory
9 working sets are already running on top of lib 11 working sets are already running on top of libhugetlbfs and in turn
10 hugetlbfs. Transparent HugePage Support (THP) 12 hugetlbfs. Transparent HugePage Support (THP) is an alternative mean of
11 using huge pages for the backing of virtual me 13 using huge pages for the backing of virtual memory with huge pages
12 that supports the automatic promotion and demo 14 that supports the automatic promotion and demotion of page sizes and
13 without the shortcomings of hugetlbfs. 15 without the shortcomings of hugetlbfs.
14 16
15 Currently THP only works for anonymous memory 17 Currently THP only works for anonymous memory mappings and tmpfs/shmem.
16 But in the future it can expand to other files 18 But in the future it can expand to other filesystems.
17 19
18 .. note:: 20 .. note::
19 in the examples below we presume that the b 21 in the examples below we presume that the basic page size is 4K and
20 the huge page size is 2M, although the actu 22 the huge page size is 2M, although the actual numbers may vary
21 depending on the CPU architecture. 23 depending on the CPU architecture.
22 24
23 The reason applications are running faster is 25 The reason applications are running faster is because of two
24 factors. The first factor is almost completely 26 factors. The first factor is almost completely irrelevant and it's not
25 of significant interest because it'll also hav 27 of significant interest because it'll also have the downside of
26 requiring larger clear-page copy-page in page 28 requiring larger clear-page copy-page in page faults which is a
27 potentially negative effect. The first factor 29 potentially negative effect. The first factor consists in taking a
28 single page fault for each 2M virtual region t 30 single page fault for each 2M virtual region touched by userland (so
29 reducing the enter/exit kernel frequency by a 31 reducing the enter/exit kernel frequency by a 512 times factor). This
30 only matters the first time the memory is acce 32 only matters the first time the memory is accessed for the lifetime of
31 a memory mapping. The second long lasting and 33 a memory mapping. The second long lasting and much more important
32 factor will affect all subsequent accesses to 34 factor will affect all subsequent accesses to the memory for the whole
33 runtime of the application. The second factor 35 runtime of the application. The second factor consist of two
34 components: 36 components:
35 37
36 1) the TLB miss will run faster (especially wi 38 1) the TLB miss will run faster (especially with virtualization using
37 nested pagetables but almost always also on 39 nested pagetables but almost always also on bare metal without
38 virtualization) 40 virtualization)
39 41
40 2) a single TLB entry will be mapping a much l 42 2) a single TLB entry will be mapping a much larger amount of virtual
41 memory in turn reducing the number of TLB m 43 memory in turn reducing the number of TLB misses. With
42 virtualization and nested pagetables the TL 44 virtualization and nested pagetables the TLB can be mapped of
43 larger size only if both KVM and the Linux 45 larger size only if both KVM and the Linux guest are using
44 hugepages but a significant speedup already 46 hugepages but a significant speedup already happens if only one of
45 the two is using hugepages just because of 47 the two is using hugepages just because of the fact the TLB miss is
46 going to run faster. 48 going to run faster.
47 49
48 Modern kernels support "multi-size THP" (mTHP) <<
49 ability to allocate memory in blocks that are <<
50 but smaller than traditional PMD-size (as desc <<
51 increments of a power-of-2 number of pages. mT <<
52 memory (for example 16K, 32K, 64K, etc). These <<
53 PTE-mapped, but in many cases can still provid <<
54 those outlined above: Page faults are signific <<
55 factor of e.g. 4, 8, 16, etc), but latency spi <<
56 prominent because the size of each page isn't <<
57 variant and there is less memory to clear in e <<
58 architectures also employ TLB compression mech <<
59 entries in when a set of PTEs are virtually an <<
60 and approporiately aligned. In this case, TLB <<
61 often. <<
62 <<
63 THP can be enabled system wide or restricted t 50 THP can be enabled system wide or restricted to certain tasks or even
64 memory ranges inside task's address space. Unl 51 memory ranges inside task's address space. Unless THP is completely
65 disabled, there is ``khugepaged`` daemon that 52 disabled, there is ``khugepaged`` daemon that scans memory and
66 collapses sequences of basic pages into PMD-si !! 53 collapses sequences of basic pages into huge pages.
67 54
68 The THP behaviour is controlled via :ref:`sysf 55 The THP behaviour is controlled via :ref:`sysfs <thp_sysfs>`
69 interface and using madvise(2) and prctl(2) sy 56 interface and using madvise(2) and prctl(2) system calls.
70 57
71 Transparent Hugepage Support maximizes the use 58 Transparent Hugepage Support maximizes the usefulness of free memory
72 if compared to the reservation approach of hug 59 if compared to the reservation approach of hugetlbfs by allowing all
73 unused memory to be used as cache or other mov 60 unused memory to be used as cache or other movable (or even unmovable
74 entities). It doesn't require reservation to p 61 entities). It doesn't require reservation to prevent hugepage
75 allocation failures to be noticeable from user 62 allocation failures to be noticeable from userland. It allows paging
76 and all other advanced VM features to be avail 63 and all other advanced VM features to be available on the
77 hugepages. It requires no modifications for ap 64 hugepages. It requires no modifications for applications to take
78 advantage of it. 65 advantage of it.
79 66
80 Applications however can be further optimized 67 Applications however can be further optimized to take advantage of
81 this feature, like for example they've been op 68 this feature, like for example they've been optimized before to avoid
82 a flood of mmap system calls for every malloc( 69 a flood of mmap system calls for every malloc(4k). Optimizing userland
83 is by far not mandatory and khugepaged already 70 is by far not mandatory and khugepaged already can take care of long
84 lived page allocations even for hugepage unawa 71 lived page allocations even for hugepage unaware applications that
85 deals with large amounts of memory. 72 deals with large amounts of memory.
86 73
87 In certain cases when hugepages are enabled sy 74 In certain cases when hugepages are enabled system wide, application
88 may end up allocating more memory resources. A 75 may end up allocating more memory resources. An application may mmap a
89 large region but only touch 1 byte of it, in t 76 large region but only touch 1 byte of it, in that case a 2M page might
90 be allocated instead of a 4k page for no good. 77 be allocated instead of a 4k page for no good. This is why it's
91 possible to disable hugepages system-wide and 78 possible to disable hugepages system-wide and to only have them inside
92 MADV_HUGEPAGE madvise regions. 79 MADV_HUGEPAGE madvise regions.
93 80
94 Embedded systems should enable hugepages only 81 Embedded systems should enable hugepages only inside madvise regions
95 to eliminate any risk of wasting any precious 82 to eliminate any risk of wasting any precious byte of memory and to
96 only run faster. 83 only run faster.
97 84
98 Applications that gets a lot of benefit from h 85 Applications that gets a lot of benefit from hugepages and that don't
99 risk to lose memory by using hugepages, should 86 risk to lose memory by using hugepages, should use
100 madvise(MADV_HUGEPAGE) on their critical mmapp 87 madvise(MADV_HUGEPAGE) on their critical mmapped regions.
101 88
102 .. _thp_sysfs: 89 .. _thp_sysfs:
103 90
104 sysfs 91 sysfs
105 ===== 92 =====
106 93
107 Global THP controls 94 Global THP controls
108 ------------------- 95 -------------------
109 96
110 Transparent Hugepage Support for anonymous mem 97 Transparent Hugepage Support for anonymous memory can be entirely disabled
111 (mostly for debugging purposes) or only enable 98 (mostly for debugging purposes) or only enabled inside MADV_HUGEPAGE
112 regions (to avoid the risk of consuming more m 99 regions (to avoid the risk of consuming more memory resources) or enabled
113 system wide. This can be achieved per-supporte !! 100 system wide. This can be achieved with one of::
114 <<
115 echo always >/sys/kernel/mm/transparen <<
116 echo madvise >/sys/kernel/mm/transpare <<
117 echo never >/sys/kernel/mm/transparent <<
118 <<
119 where <size> is the hugepage size being addres <<
120 for which vary by system. <<
121 <<
122 For example:: <<
123 <<
124 echo always >/sys/kernel/mm/transparen <<
125 <<
126 Alternatively it is possible to specify that a <<
127 will inherit the top-level "enabled" value:: <<
128 <<
129 echo inherit >/sys/kernel/mm/transpare <<
130 <<
131 For example:: <<
132 <<
133 echo inherit >/sys/kernel/mm/transpare <<
134 <<
135 The top-level setting (for use with "inherit") <<
136 one of the following commands:: <<
137 101
138 echo always >/sys/kernel/mm/transparen 102 echo always >/sys/kernel/mm/transparent_hugepage/enabled
139 echo madvise >/sys/kernel/mm/transpare 103 echo madvise >/sys/kernel/mm/transparent_hugepage/enabled
140 echo never >/sys/kernel/mm/transparent 104 echo never >/sys/kernel/mm/transparent_hugepage/enabled
141 105
142 By default, PMD-sized hugepages have enabled=" <<
143 hugepage sizes have enabled="never". If enabli <<
144 sizes, the kernel will select the most appropr <<
145 given allocation. <<
146 <<
147 It's also possible to limit defrag efforts in 106 It's also possible to limit defrag efforts in the VM to generate
148 anonymous hugepages in case they're not immedi 107 anonymous hugepages in case they're not immediately free to madvise
149 regions or to never try to defrag memory and s 108 regions or to never try to defrag memory and simply fallback to regular
150 pages unless hugepages are immediately availab 109 pages unless hugepages are immediately available. Clearly if we spend CPU
151 time to defrag memory, we would expect to gain 110 time to defrag memory, we would expect to gain even more by the fact we
152 use hugepages later instead of regular pages. 111 use hugepages later instead of regular pages. This isn't always
153 guaranteed, but it may be more likely in case 112 guaranteed, but it may be more likely in case the allocation is for a
154 MADV_HUGEPAGE region. 113 MADV_HUGEPAGE region.
155 114
156 :: 115 ::
157 116
158 echo always >/sys/kernel/mm/transparen 117 echo always >/sys/kernel/mm/transparent_hugepage/defrag
159 echo defer >/sys/kernel/mm/transparent 118 echo defer >/sys/kernel/mm/transparent_hugepage/defrag
160 echo defer+madvise >/sys/kernel/mm/tra 119 echo defer+madvise >/sys/kernel/mm/transparent_hugepage/defrag
161 echo madvise >/sys/kernel/mm/transpare 120 echo madvise >/sys/kernel/mm/transparent_hugepage/defrag
162 echo never >/sys/kernel/mm/transparent 121 echo never >/sys/kernel/mm/transparent_hugepage/defrag
163 122
164 always 123 always
165 means that an application requesting T 124 means that an application requesting THP will stall on
166 allocation failure and directly reclai 125 allocation failure and directly reclaim pages and compact
167 memory in an effort to allocate a THP 126 memory in an effort to allocate a THP immediately. This may be
168 desirable for virtual machines that be 127 desirable for virtual machines that benefit heavily from THP
169 use and are willing to delay the VM st 128 use and are willing to delay the VM start to utilise them.
170 129
171 defer 130 defer
172 means that an application will wake ks 131 means that an application will wake kswapd in the background
173 to reclaim pages and wake kcompactd to 132 to reclaim pages and wake kcompactd to compact memory so that
174 THP is available in the near future. I 133 THP is available in the near future. It's the responsibility
175 of khugepaged to then install the THP 134 of khugepaged to then install the THP pages later.
176 135
177 defer+madvise 136 defer+madvise
178 will enter direct reclaim and compacti 137 will enter direct reclaim and compaction like ``always``, but
179 only for regions that have used madvis 138 only for regions that have used madvise(MADV_HUGEPAGE); all
180 other regions will wake kswapd in the 139 other regions will wake kswapd in the background to reclaim
181 pages and wake kcompactd to compact me 140 pages and wake kcompactd to compact memory so that THP is
182 available in the near future. 141 available in the near future.
183 142
184 madvise 143 madvise
185 will enter direct reclaim like ``alway 144 will enter direct reclaim like ``always`` but only for regions
186 that are have used madvise(MADV_HUGEPA 145 that are have used madvise(MADV_HUGEPAGE). This is the default
187 behaviour. 146 behaviour.
188 147
189 never 148 never
190 should be self-explanatory. 149 should be self-explanatory.
191 150
192 By default kernel tries to use huge, PMD-mappa !! 151 By default kernel tries to use huge zero page on read page fault to
193 page fault to anonymous mapping. It's possible !! 152 anonymous mapping. It's possible to disable huge zero page by writing 0
194 page by writing 0 or enable it back by writing !! 153 or enable it back by writing 1::
195 154
196 echo 0 >/sys/kernel/mm/transparent_hug 155 echo 0 >/sys/kernel/mm/transparent_hugepage/use_zero_page
197 echo 1 >/sys/kernel/mm/transparent_hug 156 echo 1 >/sys/kernel/mm/transparent_hugepage/use_zero_page
198 157
199 Some userspace (such as a test program, or an !! 158 Some userspace (such as a test program, or an optimized memory allocation
200 allocation library) may want to know the size !! 159 library) may want to know the size (in bytes) of a transparent hugepage::
201 PMD-mappable transparent hugepage:: <<
202 160
203 cat /sys/kernel/mm/transparent_hugepag 161 cat /sys/kernel/mm/transparent_hugepage/hpage_pmd_size
204 162
205 All THPs at fault and collapse time will be ad !! 163 khugepaged will be automatically started when
206 and will therefore be split under memory presu !! 164 transparent_hugepage/enabled is set to "always" or "madvise, and it'll
207 "underused". A THP is underused if the number !! 165 be automatically shutdown if it's set to "never".
208 the THP is above max_ptes_none (see below). It <<
209 this behaviour by writing 0 to shrink_underuse <<
210 1 to it:: <<
211 <<
212 echo 0 > /sys/kernel/mm/transparent_hu <<
213 echo 1 > /sys/kernel/mm/transparent_hu <<
214 <<
215 khugepaged will be automatically started when <<
216 (either of the per-size anon control or the to <<
217 to "always" or "madvise"), and it'll be automa <<
218 PMD-sized THP is disabled (when both the per-s <<
219 top-level control are "never") <<
220 166
221 Khugepaged controls 167 Khugepaged controls
222 ------------------- 168 -------------------
223 169
224 .. note:: <<
225 khugepaged currently only searches for oppo <<
226 PMD-sized THP and no attempt is made to col <<
227 sizes. <<
228 <<
229 khugepaged runs usually at low frequency so wh 170 khugepaged runs usually at low frequency so while one may not want to
230 invoke defrag algorithms synchronously during 171 invoke defrag algorithms synchronously during the page faults, it
231 should be worth invoking defrag at least in kh 172 should be worth invoking defrag at least in khugepaged. However it's
232 also possible to disable defrag in khugepaged 173 also possible to disable defrag in khugepaged by writing 0 or enable
233 defrag in khugepaged by writing 1:: 174 defrag in khugepaged by writing 1::
234 175
235 echo 0 >/sys/kernel/mm/transparent_hug 176 echo 0 >/sys/kernel/mm/transparent_hugepage/khugepaged/defrag
236 echo 1 >/sys/kernel/mm/transparent_hug 177 echo 1 >/sys/kernel/mm/transparent_hugepage/khugepaged/defrag
237 178
238 You can also control how many pages khugepaged 179 You can also control how many pages khugepaged should scan at each
239 pass:: 180 pass::
240 181
241 /sys/kernel/mm/transparent_hugepage/kh 182 /sys/kernel/mm/transparent_hugepage/khugepaged/pages_to_scan
242 183
243 and how many milliseconds to wait in khugepage 184 and how many milliseconds to wait in khugepaged between each pass (you
244 can set this to 0 to run khugepaged at 100% ut 185 can set this to 0 to run khugepaged at 100% utilization of one core)::
245 186
246 /sys/kernel/mm/transparent_hugepage/kh 187 /sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecs
247 188
248 and how many milliseconds to wait in khugepage 189 and how many milliseconds to wait in khugepaged if there's an hugepage
249 allocation failure to throttle the next alloca 190 allocation failure to throttle the next allocation attempt::
250 191
251 /sys/kernel/mm/transparent_hugepage/kh 192 /sys/kernel/mm/transparent_hugepage/khugepaged/alloc_sleep_millisecs
252 193
253 The khugepaged progress can be seen in the num !! 194 The khugepaged progress can be seen in the number of pages collapsed::
254 that this counter may not be an exact count of <<
255 collapsed, since "collapsed" could mean multip <<
256 being replaced by a PMD mapping, or (2) All 4K <<
257 one 2M hugepage. Each may happen independently <<
258 the type of memory and the failures that occur <<
259 be interpreted roughly as a sign of progress, <<
260 consulted for more accurate accounting):: <<
261 195
262 /sys/kernel/mm/transparent_hugepage/kh 196 /sys/kernel/mm/transparent_hugepage/khugepaged/pages_collapsed
263 197
264 for each pass:: 198 for each pass::
265 199
266 /sys/kernel/mm/transparent_hugepage/kh 200 /sys/kernel/mm/transparent_hugepage/khugepaged/full_scans
267 201
268 ``max_ptes_none`` specifies how many extra sma 202 ``max_ptes_none`` specifies how many extra small pages (that are
269 not already mapped) can be allocated when coll 203 not already mapped) can be allocated when collapsing a group
270 of small pages into one large page:: 204 of small pages into one large page::
271 205
272 /sys/kernel/mm/transparent_hugepage/kh 206 /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none
273 207
274 A higher value leads to use additional memory 208 A higher value leads to use additional memory for programs.
275 A lower value leads to gain less thp performan 209 A lower value leads to gain less thp performance. Value of
276 max_ptes_none can waste cpu time very little, 210 max_ptes_none can waste cpu time very little, you can
277 ignore it. 211 ignore it.
278 212
279 ``max_ptes_swap`` specifies how many pages can 213 ``max_ptes_swap`` specifies how many pages can be brought in from
280 swap when collapsing a group of pages into a t 214 swap when collapsing a group of pages into a transparent huge page::
281 215
282 /sys/kernel/mm/transparent_hugepage/kh 216 /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_swap
283 217
284 A higher value can cause excessive swap IO and 218 A higher value can cause excessive swap IO and waste
285 memory. A lower value can prevent THPs from be 219 memory. A lower value can prevent THPs from being
286 collapsed, resulting fewer pages being collaps 220 collapsed, resulting fewer pages being collapsed into
287 THPs, and lower memory access performance. 221 THPs, and lower memory access performance.
288 222
289 ``max_ptes_shared`` specifies how many pages c 223 ``max_ptes_shared`` specifies how many pages can be shared across multiple
290 processes. khugepaged might treat pages of THP !! 224 processes. Exceeding the number would block the collapse::
291 that THP is shared. Exceeding the number would <<
292 225
293 /sys/kernel/mm/transparent_hugepage/kh 226 /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_shared
294 227
295 A higher value may increase memory footprint f 228 A higher value may increase memory footprint for some workloads.
296 229
297 Boot parameters !! 230 Boot parameter
298 =============== !! 231 ==============
299 232
300 You can change the sysfs boot time default for !! 233 You can change the sysfs boot time defaults of Transparent Hugepage
301 control by passing the parameter ``transparent !! 234 Support by passing the parameter ``transparent_hugepage=always`` or
302 ``transparent_hugepage=madvise`` or ``transpar !! 235 ``transparent_hugepage=madvise`` or ``transparent_hugepage=never``
303 kernel command line. !! 236 to the kernel command line.
304 <<
305 Alternatively, each supported anonymous THP si <<
306 passing ``thp_anon=<size>[KMG],<size>[KMG]:<st <<
307 where ``<size>`` is the THP size (must be a po <<
308 supported anonymous THP) and ``<state>`` is o <<
309 ``never`` or ``inherit``. <<
310 <<
311 For example, the following will set 16K, 32K, <<
312 set 128K, 512K to ``inherit``, set 256K to ``m <<
313 to ``never``:: <<
314 <<
315 thp_anon=16K-64K:always;128K,512K:inhe <<
316 <<
317 ``thp_anon=`` may be specified multiple times <<
318 required. If ``thp_anon=`` is specified at lea <<
319 not explicitly configured on the command line <<
320 ``never``. <<
321 <<
322 ``transparent_hugepage`` setting only affects <<
323 ``thp_anon`` is not specified, PMD_ORDER THP w <<
324 However, if a valid ``thp_anon`` setting is pr <<
325 PMD_ORDER THP policy will be overridden. If th <<
326 is not defined within a valid ``thp_anon``, it <<
327 ``never``. <<
328 237
329 Hugepages in tmpfs/shmem 238 Hugepages in tmpfs/shmem
330 ======================== 239 ========================
331 240
332 You can control hugepage allocation policy in 241 You can control hugepage allocation policy in tmpfs with mount option
333 ``huge=``. It can have following values: 242 ``huge=``. It can have following values:
334 243
335 always 244 always
336 Attempt to allocate huge pages every time 245 Attempt to allocate huge pages every time we need a new page;
337 246
338 never 247 never
339 Do not allocate huge pages; 248 Do not allocate huge pages;
340 249
341 within_size 250 within_size
342 Only allocate huge page if it will be full 251 Only allocate huge page if it will be fully within i_size.
343 Also respect fadvise()/madvise() hints; 252 Also respect fadvise()/madvise() hints;
344 253
345 advise 254 advise
346 Only allocate huge pages if requested with 255 Only allocate huge pages if requested with fadvise()/madvise();
347 256
348 The default policy is ``never``. 257 The default policy is ``never``.
349 258
350 ``mount -o remount,huge= /mountpoint`` works f 259 ``mount -o remount,huge= /mountpoint`` works fine after mount: remounting
351 ``huge=never`` will not attempt to break up hu 260 ``huge=never`` will not attempt to break up huge pages at all, just stop more
352 from being allocated. 261 from being allocated.
353 262
354 There's also sysfs knob to control hugepage al 263 There's also sysfs knob to control hugepage allocation policy for internal
355 shmem mount: /sys/kernel/mm/transparent_hugepa 264 shmem mount: /sys/kernel/mm/transparent_hugepage/shmem_enabled. The mount
356 is used for SysV SHM, memfds, shared anonymous 265 is used for SysV SHM, memfds, shared anonymous mmaps (of /dev/zero or
357 MAP_ANONYMOUS), GPU drivers' DRM objects, Ashm 266 MAP_ANONYMOUS), GPU drivers' DRM objects, Ashmem.
358 267
359 In addition to policies listed above, shmem_en 268 In addition to policies listed above, shmem_enabled allows two further
360 values: 269 values:
361 270
362 deny 271 deny
363 For use in emergencies, to force the huge 272 For use in emergencies, to force the huge option off from
364 all mounts; 273 all mounts;
365 force 274 force
366 Force the huge option on for all - very us 275 Force the huge option on for all - very useful for testing;
367 276
368 Shmem can also use "multi-size THP" (mTHP) by <<
369 control mTHP allocation: <<
370 '/sys/kernel/mm/transparent_hugepage/hugepages <<
371 and its value for each mTHP is essentially con <<
372 setting. An 'inherit' option is added to ensu <<
373 global settings. Conversely, the options 'for <<
374 which are rather testing artifacts from the ol <<
375 <<
376 always <<
377 Attempt to allocate <size> huge pages ever <<
378 <<
379 inherit <<
380 Inherit the top-level "shmem_enabled" valu <<
381 have enabled="inherit" and all other hugep <<
382 <<
383 never <<
384 Do not allocate <size> huge pages; <<
385 <<
386 within_size <<
387 Only allocate <size> huge page if it will <<
388 Also respect fadvise()/madvise() hints; <<
389 <<
390 advise <<
391 Only allocate <size> huge pages if request <<
392 <<
393 Need of application restart 277 Need of application restart
394 =========================== 278 ===========================
395 279
396 The transparent_hugepage/enabled and !! 280 The transparent_hugepage/enabled values and tmpfs mount option only affect
397 transparent_hugepage/hugepages-<size>kB/enable !! 281 future behavior. So to make them effective you need to restart any
398 option only affect future behavior. So to make !! 282 application that could have been using hugepages. This also applies to the
399 to restart any application that could have bee !! 283 regions registered in khugepaged.
400 also applies to the regions registered in khug <<
401 284
402 Monitoring usage 285 Monitoring usage
403 ================ 286 ================
404 287
405 The number of PMD-sized anonymous transparent !! 288 The number of anonymous transparent huge pages currently used by the
406 system is available by reading the AnonHugePag 289 system is available by reading the AnonHugePages field in ``/proc/meminfo``.
407 To identify what applications are using PMD-si !! 290 To identify what applications are using anonymous transparent huge pages,
408 pages, it is necessary to read ``/proc/PID/sma !! 291 it is necessary to read ``/proc/PID/smaps`` and count the AnonHugePages fields
409 fields for each mapping. (Note that AnonHugePa !! 292 for each mapping.
410 PMD-sized THP for historical reasons and shoul <<
411 AnonHugePmdMapped). <<
412 293
413 The number of file transparent huge pages mapp 294 The number of file transparent huge pages mapped to userspace is available
414 by reading ShmemPmdMapped and ShmemHugePages f 295 by reading ShmemPmdMapped and ShmemHugePages fields in ``/proc/meminfo``.
415 To identify what applications are mapping file 296 To identify what applications are mapping file transparent huge pages, it
416 is necessary to read ``/proc/PID/smaps`` and c 297 is necessary to read ``/proc/PID/smaps`` and count the FileHugeMapped fields
417 for each mapping. 298 for each mapping.
418 299
419 Note that reading the smaps file is expensive 300 Note that reading the smaps file is expensive and reading it
420 frequently will incur overhead. 301 frequently will incur overhead.
421 302
422 There are a number of counters in ``/proc/vmst 303 There are a number of counters in ``/proc/vmstat`` that may be used to
423 monitor how successfully the system is providi 304 monitor how successfully the system is providing huge pages for use.
424 305
425 thp_fault_alloc 306 thp_fault_alloc
426 is incremented every time a huge page 307 is incremented every time a huge page is successfully
427 allocated and charged to handle a page !! 308 allocated to handle a page fault.
428 309
429 thp_collapse_alloc 310 thp_collapse_alloc
430 is incremented by khugepaged when it h 311 is incremented by khugepaged when it has found
431 a range of pages to collapse into one 312 a range of pages to collapse into one huge page and has
432 successfully allocated a new huge page 313 successfully allocated a new huge page to store the data.
433 314
434 thp_fault_fallback 315 thp_fault_fallback
435 is incremented if a page fault fails t !! 316 is incremented if a page fault fails to allocate
436 a huge page and instead falls back to 317 a huge page and instead falls back to using small pages.
437 318
438 thp_fault_fallback_charge 319 thp_fault_fallback_charge
439 is incremented if a page fault fails t 320 is incremented if a page fault fails to charge a huge page and
440 instead falls back to using small page 321 instead falls back to using small pages even though the
441 allocation was successful. 322 allocation was successful.
442 323
443 thp_collapse_alloc_failed 324 thp_collapse_alloc_failed
444 is incremented if khugepaged found a r 325 is incremented if khugepaged found a range
445 of pages that should be collapsed into 326 of pages that should be collapsed into one huge page but failed
446 the allocation. 327 the allocation.
447 328
448 thp_file_alloc 329 thp_file_alloc
449 is incremented every time a shmem huge !! 330 is incremented every time a file huge page is successfully
450 allocated (Note that despite being nam !! 331 allocated.
451 measures only shmem). <<
452 332
453 thp_file_fallback 333 thp_file_fallback
454 is incremented if a shmem huge page is !! 334 is incremented if a file huge page is attempted to be allocated
455 but fails and instead falls back to us !! 335 but fails and instead falls back to using small pages.
456 despite being named after "file", the <<
457 336
458 thp_file_fallback_charge 337 thp_file_fallback_charge
459 is incremented if a shmem huge page ca !! 338 is incremented if a file huge page cannot be charged and instead
460 falls back to using small pages even t 339 falls back to using small pages even though the allocation was
461 successful. (Note that despite being n !! 340 successful.
462 counter measures only shmem). <<
463 341
464 thp_file_mapped 342 thp_file_mapped
465 is incremented every time a file or sh !! 343 is incremented every time a file huge page is mapped into
466 user address space. 344 user address space.
467 345
468 thp_split_page 346 thp_split_page
469 is incremented every time a huge page 347 is incremented every time a huge page is split into base
470 pages. This can happen for a variety o 348 pages. This can happen for a variety of reasons but a common
471 reason is that a huge page is old and 349 reason is that a huge page is old and is being reclaimed.
472 This action implies splitting all PMD 350 This action implies splitting all PMD the page mapped with.
473 351
474 thp_split_page_failed 352 thp_split_page_failed
475 is incremented if kernel fails to spli 353 is incremented if kernel fails to split huge
476 page. This can happen if the page was 354 page. This can happen if the page was pinned by somebody.
477 355
478 thp_deferred_split_page 356 thp_deferred_split_page
479 is incremented when a huge page is put 357 is incremented when a huge page is put onto split
480 queue. This happens when a huge page i 358 queue. This happens when a huge page is partially unmapped and
481 splitting it would free up some memory 359 splitting it would free up some memory. Pages on split queue are
482 going to be split under memory pressur 360 going to be split under memory pressure.
483 361
484 thp_underused_split_page <<
485 is incremented when a huge page on the <<
486 because it was underused. A THP is und <<
487 zero pages in the THP is above a certa <<
488 (/sys/kernel/mm/transparent_hugepage/k <<
489 <<
490 thp_split_pmd 362 thp_split_pmd
491 is incremented every time a PMD split 363 is incremented every time a PMD split into table of PTEs.
492 This can happen, for instance, when ap 364 This can happen, for instance, when application calls mprotect() or
493 munmap() on part of huge page. It does 365 munmap() on part of huge page. It doesn't split huge page, only
494 page table entry. 366 page table entry.
495 367
496 thp_zero_page_alloc 368 thp_zero_page_alloc
497 is incremented every time a huge zero !! 369 is incremented every time a huge zero page is
498 successfully allocated. Note, it doesn !! 370 successfully allocated. It includes allocations which where
499 the huge zero page, only its allocatio !! 371 dropped due race with other allocation. Note, it doesn't count
>> 372 every map of the huge zero page, only its allocation.
500 373
501 thp_zero_page_alloc_failed 374 thp_zero_page_alloc_failed
502 is incremented if kernel fails to allo 375 is incremented if kernel fails to allocate
503 huge zero page and falls back to using 376 huge zero page and falls back to using small pages.
504 377
505 thp_swpout 378 thp_swpout
506 is incremented every time a huge page 379 is incremented every time a huge page is swapout in one
507 piece without splitting. 380 piece without splitting.
508 381
509 thp_swpout_fallback 382 thp_swpout_fallback
510 is incremented if a huge page has to b 383 is incremented if a huge page has to be split before swapout.
511 Usually because failed to allocate som 384 Usually because failed to allocate some continuous swap space
512 for the huge page. 385 for the huge page.
513 386
514 In /sys/kernel/mm/transparent_hugepage/hugepag <<
515 also individual counters for each huge page si <<
516 monitor the system's effectiveness in providin <<
517 counter has its own corresponding file. <<
518 <<
519 anon_fault_alloc <<
520 is incremented every time a huge page <<
521 allocated and charged to handle a page <<
522 <<
523 anon_fault_fallback <<
524 is incremented if a page fault fails t <<
525 a huge page and instead falls back to <<
526 lower orders or small pages. <<
527 <<
528 anon_fault_fallback_charge <<
529 is incremented if a page fault fails t <<
530 instead falls back to using huge pages <<
531 small pages even though the allocation <<
532 <<
533 swpout <<
534 is incremented every time a huge page <<
535 piece without splitting. <<
536 <<
537 swpout_fallback <<
538 is incremented if a huge page has to b <<
539 Usually because failed to allocate som <<
540 for the huge page. <<
541 <<
542 shmem_alloc <<
543 is incremented every time a shmem huge <<
544 allocated. <<
545 <<
546 shmem_fallback <<
547 is incremented if a shmem huge page is <<
548 but fails and instead falls back to us <<
549 <<
550 shmem_fallback_charge <<
551 is incremented if a shmem huge page ca <<
552 falls back to using small pages even t <<
553 successful. <<
554 <<
555 split <<
556 is incremented every time a huge page <<
557 smaller orders. This can happen for a <<
558 common reason is that a huge page is o <<
559 <<
560 split_failed <<
561 is incremented if kernel fails to spli <<
562 page. This can happen if the page was <<
563 <<
564 split_deferred <<
565 is incremented when a huge page is put <<
566 This happens when a huge page is parti <<
567 it would free up some memory. Pages on <<
568 be split under memory pressure, if spl <<
569 <<
570 nr_anon <<
571 the number of anonymous THP we have in <<
572 might be currently entirely mapped or h <<
573 subpages. <<
574 <<
575 nr_anon_partially_mapped <<
576 the number of anonymous THP which are l <<
577 wasting memory, and have been queued fo <<
578 Note that in corner some cases (e.g., f <<
579 an anonymous THP as "partially mapped" <<
580 is not actually partially mapped anymor <<
581 <<
582 As the system ages, allocating huge pages may 387 As the system ages, allocating huge pages may be expensive as the
583 system uses memory compaction to copy data aro 388 system uses memory compaction to copy data around memory to free a
584 huge page for use. There are some counters in 389 huge page for use. There are some counters in ``/proc/vmstat`` to help
585 monitor this overhead. 390 monitor this overhead.
586 391
587 compact_stall 392 compact_stall
588 is incremented every time a process st 393 is incremented every time a process stalls to run
589 memory compaction so that a huge page 394 memory compaction so that a huge page is free for use.
590 395
591 compact_success 396 compact_success
592 is incremented if the system compacted 397 is incremented if the system compacted memory and
593 freed a huge page for use. 398 freed a huge page for use.
594 399
595 compact_fail 400 compact_fail
596 is incremented if the system tries to 401 is incremented if the system tries to compact memory
597 but failed. 402 but failed.
598 403
>> 404 compact_pages_moved
>> 405 is incremented each time a page is moved. If
>> 406 this value is increasing rapidly, it implies that the system
>> 407 is copying a lot of data to satisfy the huge page allocation.
>> 408 It is possible that the cost of copying exceeds any savings
>> 409 from reduced TLB misses.
>> 410
>> 411 compact_pagemigrate_failed
>> 412 is incremented when the underlying mechanism
>> 413 for moving a page failed.
>> 414
>> 415 compact_blocks_moved
>> 416 is incremented each time memory compaction examines
>> 417 a huge page aligned range of pages.
>> 418
599 It is possible to establish how long the stall 419 It is possible to establish how long the stalls were using the function
600 tracer to record how long was spent in __alloc !! 420 tracer to record how long was spent in __alloc_pages_nodemask and
601 using the mm_page_alloc tracepoint to identify 421 using the mm_page_alloc tracepoint to identify which allocations were
602 for huge pages. 422 for huge pages.
603 423
604 Optimizing the applications 424 Optimizing the applications
605 =========================== 425 ===========================
606 426
607 To be guaranteed that the kernel will map a TH !! 427 To be guaranteed that the kernel will map a 2M page immediately in any
608 memory region, the mmap region has to be hugep 428 memory region, the mmap region has to be hugepage naturally
609 aligned. posix_memalign() can provide that gua 429 aligned. posix_memalign() can provide that guarantee.
610 430
611 Hugetlbfs 431 Hugetlbfs
612 ========= 432 =========
613 433
614 You can use hugetlbfs on a kernel that has tra 434 You can use hugetlbfs on a kernel that has transparent hugepage
615 support enabled just fine as always. No differ 435 support enabled just fine as always. No difference can be noted in
616 hugetlbfs other than there will be less overal 436 hugetlbfs other than there will be less overall fragmentation. All
617 usual features belonging to hugetlbfs are pres 437 usual features belonging to hugetlbfs are preserved and
618 unaffected. libhugetlbfs will also work fine a 438 unaffected. libhugetlbfs will also work fine as usual.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.