1 ==================
2 HugeTLB Controller
3 ==================
4
5 HugeTLB controller can be created by first mou
6
7 # mount -t cgroup -o hugetlb none /sys/fs/cgro
8
9 With the above step, the initial or the parent
10 visible at /sys/fs/cgroup. At bootup, this gro
11 the system. /sys/fs/cgroup/tasks lists the tas
12
13 New groups can be created under the parent gro
14
15 # cd /sys/fs/cgroup
16 # mkdir g1
17 # echo $$ > g1/tasks
18
19 The above steps create a new group g1 and move
20 process (bash) into it.
21
22 Brief summary of control files::
23
24 hugetlb.<hugepagesize>.rsvd.limit_in_bytes
25 hugetlb.<hugepagesize>.rsvd.max_usage_in_byte
26 hugetlb.<hugepagesize>.rsvd.usage_in_bytes
27 hugetlb.<hugepagesize>.rsvd.failcnt
28 hugetlb.<hugepagesize>.limit_in_bytes
29 hugetlb.<hugepagesize>.max_usage_in_bytes
30 hugetlb.<hugepagesize>.usage_in_bytes
31 hugetlb.<hugepagesize>.failcnt
32 hugetlb.<hugepagesize>.numa_stat
33
34 For a system supporting three hugepage sizes (
35 files include::
36
37 hugetlb.1GB.limit_in_bytes
38 hugetlb.1GB.max_usage_in_bytes
39 hugetlb.1GB.numa_stat
40 hugetlb.1GB.usage_in_bytes
41 hugetlb.1GB.failcnt
42 hugetlb.1GB.rsvd.limit_in_bytes
43 hugetlb.1GB.rsvd.max_usage_in_bytes
44 hugetlb.1GB.rsvd.usage_in_bytes
45 hugetlb.1GB.rsvd.failcnt
46 hugetlb.64KB.limit_in_bytes
47 hugetlb.64KB.max_usage_in_bytes
48 hugetlb.64KB.numa_stat
49 hugetlb.64KB.usage_in_bytes
50 hugetlb.64KB.failcnt
51 hugetlb.64KB.rsvd.limit_in_bytes
52 hugetlb.64KB.rsvd.max_usage_in_bytes
53 hugetlb.64KB.rsvd.usage_in_bytes
54 hugetlb.64KB.rsvd.failcnt
55 hugetlb.32MB.limit_in_bytes
56 hugetlb.32MB.max_usage_in_bytes
57 hugetlb.32MB.numa_stat
58 hugetlb.32MB.usage_in_bytes
59 hugetlb.32MB.failcnt
60 hugetlb.32MB.rsvd.limit_in_bytes
61 hugetlb.32MB.rsvd.max_usage_in_bytes
62 hugetlb.32MB.rsvd.usage_in_bytes
63 hugetlb.32MB.rsvd.failcnt
64
65
66 1. Page fault accounting
67
68 ::
69
70 hugetlb.<hugepagesize>.limit_in_bytes
71 hugetlb.<hugepagesize>.max_usage_in_bytes
72 hugetlb.<hugepagesize>.usage_in_bytes
73 hugetlb.<hugepagesize>.failcnt
74
75 The HugeTLB controller allows users to limit t
76 control group and enforces the limit during pa
77 doesn't support page reclaim, enforcing the li
78 that, the application will get SIGBUS signal i
79 pages beyond its limit. Therefore the applicat
80 HugeTLB pages it uses before hand, and the sys
81 there are enough available on the machine for
82 getting SIGBUS.
83
84
85 2. Reservation accounting
86
87 ::
88
89 hugetlb.<hugepagesize>.rsvd.limit_in_bytes
90 hugetlb.<hugepagesize>.rsvd.max_usage_in_byt
91 hugetlb.<hugepagesize>.rsvd.usage_in_bytes
92 hugetlb.<hugepagesize>.rsvd.failcnt
93
94 The HugeTLB controller allows to limit the Hug
95 group and enforces the controller limit at res
96 HugeTLB memory for which no reservation exists
97 enforced at reservation time (on mmap or shget
98 the application to get SIGBUS signal if the me
99 MAP_NORESERVE allocations, the reservation lim
100 limit, enforcing memory usage at fault time an
101 receive a SIGBUS if it's crossing its limit.
102
103 Reservation limits are superior to page fault
104 reservation limits are enforced at reservation
105 never causes the application to get SIGBUS sig
106 before hand. This allows for easier fallback t
107 non-HugeTLB memory for example. In the case of
108 hard to avoid processes getting SIGBUS since t
109 the HugeTLB usage of all the tasks in the syst
110 pages to satisfy all requests. Avoiding tasks
111 systems is practically impossible with page fa
112
113
114 3. Caveats with shared memory
115
116 For shared HugeTLB memory, both HugeTLB reserv
117 to the first task that causes the memory to be
118 subsequent uses of this reserved or faulted me
119
120 Shared HugeTLB memory is only uncharged when i
121 This is usually when the HugeTLB file is delet
122 caused the reservation or fault has exited.
123
124
125 4. Caveats with HugeTLB cgroup offline.
126
127 When a HugeTLB cgroup goes offline with some r
128 charged to it, the behavior is as follows:
129
130 - The fault charges are charged to the parent
131 - the reservation charges remain on the offlin
132
133 This means that if a HugeTLB cgroup gets offli
134 reservations charged to it, that cgroup persis
135 reservations are uncharged. HugeTLB reservatio
136 the memory controller whose cgroups also persi
137 memory is uncharged. Also, the tracking of Hug
138 complex compared to the tracking of HugeTLB fa
139 harder to reparent reservations at offline tim
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.