1 .. SPDX-License-Identifier: GPL-2.0
2
3 ===========================
4 Ramfs, rootfs and initramfs
5 ===========================
6
7 October 17, 2005
8
9 :Author: Rob Landley <rob@landley.net>
10
11 What is ramfs?
12 --------------
13
14 Ramfs is a very simple filesystem that exports Linux's disk caching
15 mechanisms (the page cache and dentry cache) as a dynamically resizable
16 RAM-based filesystem.
17
18 Normally all files are cached in memory by Linux. Pages of data read from
19 backing store (usually the block device the filesystem is mounted on) are kept
20 around in case it's needed again, but marked as clean (freeable) in case the
21 Virtual Memory system needs the memory for something else. Similarly, data
22 written to files is marked clean as soon as it has been written to backing
23 store, but kept around for caching purposes until the VM reallocates the
24 memory. A similar mechanism (the dentry cache) greatly speeds up access to
25 directories.
26
27 With ramfs, there is no backing store. Files written into ramfs allocate
28 dentries and page cache as usual, but there's nowhere to write them to.
29 This means the pages are never marked clean, so they can't be freed by the
30 VM when it's looking to recycle memory.
31
32 The amount of code required to implement ramfs is tiny, because all the
33 work is done by the existing Linux caching infrastructure. Basically,
34 you're mounting the disk cache as a filesystem. Because of this, ramfs is not
35 an optional component removable via menuconfig, since there would be negligible
36 space savings.
37
38 ramfs and ramdisk:
39 ------------------
40
41 The older "ram disk" mechanism created a synthetic block device out of
42 an area of RAM and used it as backing store for a filesystem. This block
43 device was of fixed size, so the filesystem mounted on it was of fixed
44 size. Using a ram disk also required unnecessarily copying memory from the
45 fake block device into the page cache (and copying changes back out), as well
46 as creating and destroying dentries. Plus it needed a filesystem driver
47 (such as ext2) to format and interpret this data.
48
49 Compared to ramfs, this wastes memory (and memory bus bandwidth), creates
50 unnecessary work for the CPU, and pollutes the CPU caches. (There are tricks
51 to avoid this copying by playing with the page tables, but they're unpleasantly
52 complicated and turn out to be about as expensive as the copying anyway.)
53 More to the point, all the work ramfs is doing has to happen _anyway_,
54 since all file access goes through the page and dentry caches. The RAM
55 disk is simply unnecessary; ramfs is internally much simpler.
56
57 Another reason ramdisks are semi-obsolete is that the introduction of
58 loopback devices offered a more flexible and convenient way to create
59 synthetic block devices, now from files instead of from chunks of memory.
60 See losetup (8) for details.
61
62 ramfs and tmpfs:
63 ----------------
64
65 One downside of ramfs is you can keep writing data into it until you fill
66 up all memory, and the VM can't free it because the VM thinks that files
67 should get written to backing store (rather than swap space), but ramfs hasn't
68 got any backing store. Because of this, only root (or a trusted user) should
69 be allowed write access to a ramfs mount.
70
71 A ramfs derivative called tmpfs was created to add size limits, and the ability
72 to write the data to swap space. Normal users can be allowed write access to
73 tmpfs mounts. See Documentation/filesystems/tmpfs.rst for more information.
74
75 What is rootfs?
76 ---------------
77
78 Rootfs is a special instance of ramfs (or tmpfs, if that's enabled), which is
79 always present in 2.6 systems. You can't unmount rootfs for approximately the
80 same reason you can't kill the init process; rather than having special code
81 to check for and handle an empty list, it's smaller and simpler for the kernel
82 to just make sure certain lists can't become empty.
83
84 Most systems just mount another filesystem over rootfs and ignore it. The
85 amount of space an empty instance of ramfs takes up is tiny.
86
87 If CONFIG_TMPFS is enabled, rootfs will use tmpfs instead of ramfs by
88 default. To force ramfs, add "rootfstype=ramfs" to the kernel command
89 line.
90
91 What is initramfs?
92 ------------------
93
94 All 2.6 Linux kernels contain a gzipped "cpio" format archive, which is
95 extracted into rootfs when the kernel boots up. After extracting, the kernel
96 checks to see if rootfs contains a file "init", and if so it executes it as PID
97 1. If found, this init process is responsible for bringing the system the
98 rest of the way up, including locating and mounting the real root device (if
99 any). If rootfs does not contain an init program after the embedded cpio
100 archive is extracted into it, the kernel will fall through to the older code
101 to locate and mount a root partition, then exec some variant of /sbin/init
102 out of that.
103
104 All this differs from the old initrd in several ways:
105
106 - The old initrd was always a separate file, while the initramfs archive is
107 linked into the linux kernel image. (The directory ``linux-*/usr`` is
108 devoted to generating this archive during the build.)
109
110 - The old initrd file was a gzipped filesystem image (in some file format,
111 such as ext2, that needed a driver built into the kernel), while the new
112 initramfs archive is a gzipped cpio archive (like tar only simpler,
113 see cpio(1) and Documentation/driver-api/early-userspace/buffer-format.rst).
114 The kernel's cpio extraction code is not only extremely small, it's also
115 __init text and data that can be discarded during the boot process.
116
117 - The program run by the old initrd (which was called /initrd, not /init) did
118 some setup and then returned to the kernel, while the init program from
119 initramfs is not expected to return to the kernel. (If /init needs to hand
120 off control it can overmount / with a new root device and exec another init
121 program. See the switch_root utility, below.)
122
123 - When switching another root device, initrd would pivot_root and then
124 umount the ramdisk. But initramfs is rootfs: you can neither pivot_root
125 rootfs, nor unmount it. Instead delete everything out of rootfs to
126 free up the space (find -xdev / -exec rm '{}' ';'), overmount rootfs
127 with the new root (cd /newmount; mount --move . /; chroot .), attach
128 stdin/stdout/stderr to the new /dev/console, and exec the new init.
129
130 Since this is a remarkably persnickety process (and involves deleting
131 commands before you can run them), the klibc package introduced a helper
132 program (utils/run_init.c) to do all this for you. Most other packages
133 (such as busybox) have named this command "switch_root".
134
135 Populating initramfs:
136 ---------------------
137
138 The 2.6 kernel build process always creates a gzipped cpio format initramfs
139 archive and links it into the resulting kernel binary. By default, this
140 archive is empty (consuming 134 bytes on x86).
141
142 The config option CONFIG_INITRAMFS_SOURCE (in General Setup in menuconfig,
143 and living in usr/Kconfig) can be used to specify a source for the
144 initramfs archive, which will automatically be incorporated into the
145 resulting binary. This option can point to an existing gzipped cpio
146 archive, a directory containing files to be archived, or a text file
147 specification such as the following example::
148
149 dir /dev 755 0 0
150 nod /dev/console 644 0 0 c 5 1
151 nod /dev/loop0 644 0 0 b 7 0
152 dir /bin 755 1000 1000
153 slink /bin/sh busybox 777 0 0
154 file /bin/busybox initramfs/busybox 755 0 0
155 dir /proc 755 0 0
156 dir /sys 755 0 0
157 dir /mnt 755 0 0
158 file /init initramfs/init.sh 755 0 0
159
160 Run "usr/gen_init_cpio" (after the kernel build) to get a usage message
161 documenting the above file format.
162
163 One advantage of the configuration file is that root access is not required to
164 set permissions or create device nodes in the new archive. (Note that those
165 two example "file" entries expect to find files named "init.sh" and "busybox" in
166 a directory called "initramfs", under the linux-2.6.* directory. See
167 Documentation/driver-api/early-userspace/early_userspace_support.rst for more details.)
168
169 The kernel does not depend on external cpio tools. If you specify a
170 directory instead of a configuration file, the kernel's build infrastructure
171 creates a configuration file from that directory (usr/Makefile calls
172 usr/gen_initramfs.sh), and proceeds to package up that directory
173 using the config file (by feeding it to usr/gen_init_cpio, which is created
174 from usr/gen_init_cpio.c). The kernel's build-time cpio creation code is
175 entirely self-contained, and the kernel's boot-time extractor is also
176 (obviously) self-contained.
177
178 The one thing you might need external cpio utilities installed for is creating
179 or extracting your own preprepared cpio files to feed to the kernel build
180 (instead of a config file or directory).
181
182 The following command line can extract a cpio image (either by the above script
183 or by the kernel build) back into its component files::
184
185 cpio -i -d -H newc -F initramfs_data.cpio --no-absolute-filenames
186
187 The following shell script can create a prebuilt cpio archive you can
188 use in place of the above config file::
189
190 #!/bin/sh
191
192 # Copyright 2006 Rob Landley <rob@landley.net> and TimeSys Corporation.
193 # Licensed under GPL version 2
194
195 if [ $# -ne 2 ]
196 then
197 echo "usage: mkinitramfs directory imagename.cpio.gz"
198 exit 1
199 fi
200
201 if [ -d "$1" ]
202 then
203 echo "creating $2 from $1"
204 (cd "$1"; find . | cpio -o -H newc | gzip) > "$2"
205 else
206 echo "First argument must be a directory"
207 exit 1
208 fi
209
210 .. Note::
211
212 The cpio man page contains some bad advice that will break your initramfs
213 archive if you follow it. It says "A typical way to generate the list
214 of filenames is with the find command; you should give find the -depth
215 option to minimize problems with permissions on directories that are
216 unwritable or not searchable." Don't do this when creating
217 initramfs.cpio.gz images, it won't work. The Linux kernel cpio extractor
218 won't create files in a directory that doesn't exist, so the directory
219 entries must go before the files that go in those directories.
220 The above script gets them in the right order.
221
222 External initramfs images:
223 --------------------------
224
225 If the kernel has initrd support enabled, an external cpio.gz archive can also
226 be passed into a 2.6 kernel in place of an initrd. In this case, the kernel
227 will autodetect the type (initramfs, not initrd) and extract the external cpio
228 archive into rootfs before trying to run /init.
229
230 This has the memory efficiency advantages of initramfs (no ramdisk block
231 device) but the separate packaging of initrd (which is nice if you have
232 non-GPL code you'd like to run from initramfs, without conflating it with
233 the GPL licensed Linux kernel binary).
234
235 It can also be used to supplement the kernel's built-in initramfs image. The
236 files in the external archive will overwrite any conflicting files in
237 the built-in initramfs archive. Some distributors also prefer to customize
238 a single kernel image with task-specific initramfs images, without recompiling.
239
240 Contents of initramfs:
241 ----------------------
242
243 An initramfs archive is a complete self-contained root filesystem for Linux.
244 If you don't already understand what shared libraries, devices, and paths
245 you need to get a minimal root filesystem up and running, here are some
246 references:
247
248 - https://www.tldp.org/HOWTO/Bootdisk-HOWTO/
249 - https://www.tldp.org/HOWTO/From-PowerUp-To-Bash-Prompt-HOWTO.html
250 - http://www.linuxfromscratch.org/lfs/view/stable/
251
252 The "klibc" package (https://www.kernel.org/pub/linux/libs/klibc) is
253 designed to be a tiny C library to statically link early userspace
254 code against, along with some related utilities. It is BSD licensed.
255
256 I use uClibc (https://www.uclibc.org) and busybox (https://www.busybox.net)
257 myself. These are LGPL and GPL, respectively. (A self-contained initramfs
258 package is planned for the busybox 1.3 release.)
259
260 In theory you could use glibc, but that's not well suited for small embedded
261 uses like this. (A "hello world" program statically linked against glibc is
262 over 400k. With uClibc it's 7k. Also note that glibc dlopens libnss to do
263 name lookups, even when otherwise statically linked.)
264
265 A good first step is to get initramfs to run a statically linked "hello world"
266 program as init, and test it under an emulator like qemu (www.qemu.org) or
267 User Mode Linux, like so::
268
269 cat > hello.c << EOF
270 #include <stdio.h>
271 #include <unistd.h>
272
273 int main(int argc, char *argv[])
274 {
275 printf("Hello world!\n");
276 sleep(999999999);
277 }
278 EOF
279 gcc -static hello.c -o init
280 echo init | cpio -o -H newc | gzip > test.cpio.gz
281 # Testing external initramfs using the initrd loading mechanism.
282 qemu -kernel /boot/vmlinuz -initrd test.cpio.gz /dev/zero
283
284 When debugging a normal root filesystem, it's nice to be able to boot with
285 "init=/bin/sh". The initramfs equivalent is "rdinit=/bin/sh", and it's
286 just as useful.
287
288 Why cpio rather than tar?
289 -------------------------
290
291 This decision was made back in December, 2001. The discussion started here:
292
293 http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1538.html
294
295 And spawned a second thread (specifically on tar vs cpio), starting here:
296
297 http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1587.html
298
299 The quick and dirty summary version (which is no substitute for reading
300 the above threads) is:
301
302 1) cpio is a standard. It's decades old (from the AT&T days), and already
303 widely used on Linux (inside RPM, Red Hat's device driver disks). Here's
304 a Linux Journal article about it from 1996:
305
306 http://www.linuxjournal.com/article/1213
307
308 It's not as popular as tar because the traditional cpio command line tools
309 require _truly_hideous_ command line arguments. But that says nothing
310 either way about the archive format, and there are alternative tools,
311 such as:
312
313 http://freecode.com/projects/afio
314
315 2) The cpio archive format chosen by the kernel is simpler and cleaner (and
316 thus easier to create and parse) than any of the (literally dozens of)
317 various tar archive formats. The complete initramfs archive format is
318 explained in buffer-format.txt, created in usr/gen_init_cpio.c, and
319 extracted in init/initramfs.c. All three together come to less than 26k
320 total of human-readable text.
321
322 3) The GNU project standardizing on tar is approximately as relevant as
323 Windows standardizing on zip. Linux is not part of either, and is free
324 to make its own technical decisions.
325
326 4) Since this is a kernel internal format, it could easily have been
327 something brand new. The kernel provides its own tools to create and
328 extract this format anyway. Using an existing standard was preferable,
329 but not essential.
330
331 5) Al Viro made the decision (quote: "tar is ugly as hell and not going to be
332 supported on the kernel side"):
333
334 http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1540.html
335
336 explained his reasoning:
337
338 - http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1550.html
339 - http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1638.html
340
341 and, most importantly, designed and implemented the initramfs code.
342
343 Future directions:
344 ------------------
345
346 Today (2.6.16), initramfs is always compiled in, but not always used. The
347 kernel falls back to legacy boot code that is reached only if initramfs does
348 not contain an /init program. The fallback is legacy code, there to ensure a
349 smooth transition and allowing early boot functionality to gradually move to
350 "early userspace" (I.E. initramfs).
351
352 The move to early userspace is necessary because finding and mounting the real
353 root device is complex. Root partitions can span multiple devices (raid or
354 separate journal). They can be out on the network (requiring dhcp, setting a
355 specific MAC address, logging into a server, etc). They can live on removable
356 media, with dynamically allocated major/minor numbers and persistent naming
357 issues requiring a full udev implementation to sort out. They can be
358 compressed, encrypted, copy-on-write, loopback mounted, strangely partitioned,
359 and so on.
360
361 This kind of complexity (which inevitably includes policy) is rightly handled
362 in userspace. Both klibc and busybox/uClibc are working on simple initramfs
363 packages to drop into a kernel build.
364
365 The klibc package has now been accepted into Andrew Morton's 2.6.17-mm tree.
366 The kernel's current early boot code (partition detection, etc) will probably
367 be migrated into a default initramfs, automatically created and used by the
368 kernel build.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.