1
2 .. SPDX-License-Identifier: GPL-2.0
3
4 =========================================
5 A vmemmap diet for HugeTLB and Device DAX
6 =========================================
7
8 HugeTLB
9 =======
10
11 This section is to explain how HugeTLB Vmemmap
12
13 The ``struct page`` structures are used to des
14 default, there is a one-to-one mapping from a
15 ``struct page``.
16
17 HugeTLB pages consist of multiple base page si
18 architectures. See Documentation/admin-guide/m
19 details. On the x86-64 architecture, HugeTLB p
20 currently supported. Since the base page size
21 consists of 512 base pages and a 1GB HugeTLB p
22 For each base page, there is a corresponding `
23
24 Within the HugeTLB subsystem, only the first 4
25 contain unique information about a HugeTLB pag
26 this upper limit. The only 'useful' informatio
27 is the compound_head field, and this field is
28
29 By removing redundant ``struct page`` for Huge
30 to the buddy allocator for other uses.
31
32 Different architectures support different Huge
33 following table is the HugeTLB page size suppo
34 architectures. Because arm64 supports 4k, 16k,
35 supports contiguous entries, so it supports ma
36 page.
37
38 +--------------+-----------+------------------
39 | Architecture | Page Size | Hu
40 +--------------+-----------+-----------+------
41 | x86-64 | 4KB | 2MB | 1G
42 +--------------+-----------+-----------+------
43 | | 4KB | 64KB | 2M
44 | +-----------+-----------+------
45 | arm64 | 16KB | 2MB | 32M
46 | +-----------+-----------+------
47 | | 64KB | 2MB | 512M
48 +--------------+-----------+-----------+------
49
50 When the system boot up, every HugeTLB page ha
51 structs which size is (unit: pages)::
52
53 struct_size = HugeTLB_Size / PAGE_SIZE * si
54
55 Where HugeTLB_Size is the size of the HugeTLB
56 of the HugeTLB page is always n times PAGE_SIZ
57 relationship::
58
59 HugeTLB_Size = n * PAGE_SIZE
60
61 Then::
62
63 struct_size = n * PAGE_SIZE / PAGE_SIZE * s
64 = n * sizeof(struct page) / PAG
65
66 We can use huge mapping at the pud/pmd level f
67
68 For the HugeTLB page of the pmd level mapping,
69
70 struct_size = n * sizeof(struct page) / PAG
71 = PAGE_SIZE / sizeof(pte_t) * s
72 = sizeof(struct page) / sizeof(
73 = 64 / 8
74 = 8 (pages)
75
76 Where n is how many pte entries which one page
77 n is (PAGE_SIZE / sizeof(pte_t)).
78
79 This optimization only supports 64-bit system,
80 is 8. And this optimization also applicable on
81 is a power of two. In most cases, the size of
82 x86-64 and arm64). So if we use pmd level mapp
83 size of ``struct page`` structs of it is 8 pag
84 size of the base page.
85
86 For the HugeTLB page of the pud level mapping,
87
88 struct_size = PAGE_SIZE / sizeof(pmd_t) * s
89 = PAGE_SIZE / 8 * 8 (pages)
90 = PAGE_SIZE (pages)
91
92 Where the struct_size(pmd) is the size of the
93 HugeTLB page of the pmd level mapping.
94
95 E.g.: A 2MB HugeTLB page on x86_64 consists in
96 HugeTLB page consists in 4096.
97
98 Next, we take the pmd level mapping of the Hug
99 show the internal implementation of this optim
100 ``struct page`` structs associated with a Huge
101
102 Here is how things look before optimization::
103
104 HugeTLB struct pages(8 pa
105 +-----------+ ---virt_to_page---> +----------
106 | | | 0
107 | | +----------
108 | | | 1
109 | | +----------
110 | | | 2
111 | | +----------
112 | | | 3
113 | | +----------
114 | | | 4
115 | PMD | +----------
116 | level | | 5
117 | mapping | +----------
118 | | | 6
119 | | +----------
120 | | | 7
121 | | +----------
122 | |
123 | |
124 | |
125 +-----------+
126
127 The value of page->compound_head is the same f
128 page of ``struct page`` (page 0) associated wi
129 ``struct page`` necessary to describe the Huge
130 pages of ``struct page`` (page 1 to page 7) is
131 Therefore, we can remap pages 1 to 7 to page 0
132 will be used for each HugeTLB page. This will
133 7 pages to the buddy allocator.
134
135 Here is how things look after remapping::
136
137 HugeTLB struct pages(8 pa
138 +-----------+ ---virt_to_page---> +----------
139 | | | 0
140 | | +----------
141 | | | 1
142 | | +----------
143 | | | 2
144 | | +----------
145 | | | 3
146 | | +----------
147 | | | 4
148 | PMD | +----------
149 | level | | 5
150 | mapping | +----------
151 | | | 6
152 | | +----------
153 | | | 7
154 | | +----------
155 | |
156 | |
157 | |
158 +-----------+
159
160 When a HugeTLB is freed to the buddy system, w
161 vmemmap pages and restore the previous mapping
162
163 For the HugeTLB page of the pud level mapping.
164 We also can use this approach to free (PAGE_SI
165
166 Apart from the HugeTLB page of the pmd/pud lev
167 (e.g. aarch64) provides a contiguous bit in th
168 that hints to the MMU to indicate that it is o
169 entries that can be cached in a single TLB ent
170
171 The contiguous bit is used to increase the map
172 (last) level. So this type of HugeTLB page can
173 size of the ``struct page`` structs is greater
174
175 Notice: The head vmemmap page is not freed to
176 tail vmemmap pages are mapped to the head vmem
177 more than one ``struct page`` struct with ``PG
178 page) associated with each HugeTLB page. The `
179 this correctly. There is only **one** head ``s
180 ``struct page`` with ``PG_head`` are fake head
181 approach to distinguish between those two diff
182 that ``compound_head()`` can return the real h
183 parameter is the tail ``struct page`` but with
184
185 Device DAX
186 ==========
187
188 The device-dax interface uses the same tail de
189 in the previous chapter, except when used with
190 the device (altmap).
191
192 The following page sizes are supported in DAX:
193 PMD_SIZE (2M on x86_64) and PUD_SIZE (1G on x8
194 For powerpc equivalent details see Documentati
195
196 The differences with HugeTLB are relatively mi
197
198 It only use 3 ``struct page`` for storing all
199 to 4 on HugeTLB pages.
200
201 There's no remapping of vmemmap given that dev
202 System RAM ranges initialized at boot. Thus th
203 happens at a later stage when we populate the
204 the head vmemmap page representing, whereas de
205 vmemmap page. This results in only half of the
206
207 Deduplicated tail pages are not mapped read-on
208
209 Here's how things look like on device-dax afte
210
211 +-----------+ ---virt_to_page---> +----------
212 | | | 0
213 | | +----------
214 | | | 1
215 | | +----------
216 | | | 2
217 | | +----------
218 | | | 3
219 | | +----------
220 | | | 4
221 | PMD | +----------
222 | level | | 5
223 | mapping | +----------
224 | | | 6
225 | | +----------
226 | | | 7
227 | | +----------
228 | |
229 | |
230 | |
231 +-----------+
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.