1 ====================== 1 ======================
2 Linux Kernel Selftests 2 Linux Kernel Selftests
3 ====================== 3 ======================
4 4
5 The kernel contains a set of "self tests" unde 5 The kernel contains a set of "self tests" under the tools/testing/selftests/
6 directory. These are intended to be small test 6 directory. These are intended to be small tests to exercise individual code
7 paths in the kernel. Tests are intended to be 7 paths in the kernel. Tests are intended to be run after building, installing
8 and booting a kernel. 8 and booting a kernel.
9 9
10 Kselftest from mainline can be run on older st <<
11 from mainline offers the best coverage. Severa <<
12 kselftest suite on stable releases. The reason <<
13 gets added to test existing code to regression <<
14 able to run that test on an older kernel. Henc <<
15 code that can still test an older kernel and m <<
16 gracefully on newer releases. <<
17 <<
18 You can find additional information on Kselfte 10 You can find additional information on Kselftest framework, how to
19 write new tests using the framework on Kselfte 11 write new tests using the framework on Kselftest wiki:
20 12
21 https://kselftest.wiki.kernel.org/ 13 https://kselftest.wiki.kernel.org/
22 14
23 On some systems, hot-plug tests could hang for 15 On some systems, hot-plug tests could hang forever waiting for cpu and
24 memory to be ready to be offlined. A special h 16 memory to be ready to be offlined. A special hot-plug target is created
25 to run the full range of hot-plug tests. In de 17 to run the full range of hot-plug tests. In default mode, hot-plug tests run
26 in safe mode with a limited scope. In limited 18 in safe mode with a limited scope. In limited mode, cpu-hotplug test is
27 run on a single cpu as opposed to all hotplug 19 run on a single cpu as opposed to all hotplug capable cpus, and memory
28 hotplug test is run on 2% of hotplug capable m 20 hotplug test is run on 2% of hotplug capable memory instead of 10%.
29 21
30 kselftest runs as a userspace process. Tests 22 kselftest runs as a userspace process. Tests that can be written/run in
31 userspace may wish to use the `Test Harness`_. 23 userspace may wish to use the `Test Harness`_. Tests that need to be
32 run in kernel space may wish to use a `Test Mo 24 run in kernel space may wish to use a `Test Module`_.
33 25
34 Running the selftests (hotplug tests are run i 26 Running the selftests (hotplug tests are run in limited mode)
35 ============================================== 27 =============================================================
36 28
37 To build the tests:: 29 To build the tests::
38 30
39 $ make headers <<
40 $ make -C tools/testing/selftests 31 $ make -C tools/testing/selftests
41 32
42 To run the tests:: 33 To run the tests::
43 34
44 $ make -C tools/testing/selftests run_tests 35 $ make -C tools/testing/selftests run_tests
45 36
46 To build and run the tests with a single comma 37 To build and run the tests with a single command, use::
47 38
48 $ make kselftest 39 $ make kselftest
49 40
50 Note that some tests will require root privile 41 Note that some tests will require root privileges.
51 42
52 Kselftest supports saving output files in a se 43 Kselftest supports saving output files in a separate directory and then
53 running tests. To locate output files in a sep 44 running tests. To locate output files in a separate directory two syntaxes
54 are supported. In both cases the working direc 45 are supported. In both cases the working directory must be the root of the
55 kernel src. This is applicable to "Running a s 46 kernel src. This is applicable to "Running a subset of selftests" section
56 below. 47 below.
57 48
58 To build, save output files in a separate dire 49 To build, save output files in a separate directory with O= ::
59 50
60 $ make O=/tmp/kselftest kselftest 51 $ make O=/tmp/kselftest kselftest
61 52
62 To build, save output files in a separate dire 53 To build, save output files in a separate directory with KBUILD_OUTPUT ::
63 54
64 $ export KBUILD_OUTPUT=/tmp/kselftest; make 55 $ export KBUILD_OUTPUT=/tmp/kselftest; make kselftest
65 56
66 The O= assignment takes precedence over the KB 57 The O= assignment takes precedence over the KBUILD_OUTPUT environment
67 variable. 58 variable.
68 59
69 The above commands by default run the tests an 60 The above commands by default run the tests and print full pass/fail report.
70 Kselftest supports "summary" option to make it 61 Kselftest supports "summary" option to make it easier to understand the test
71 results. Please find the detailed individual t 62 results. Please find the detailed individual test results for each test in
72 /tmp/testname file(s) when summary option is s 63 /tmp/testname file(s) when summary option is specified. This is applicable
73 to "Running a subset of selftests" section bel 64 to "Running a subset of selftests" section below.
74 65
75 To run kselftest with summary option enabled : 66 To run kselftest with summary option enabled ::
76 67
77 $ make summary=1 kselftest 68 $ make summary=1 kselftest
78 69
79 Running a subset of selftests 70 Running a subset of selftests
80 ============================= 71 =============================
81 72
82 You can use the "TARGETS" variable on the make 73 You can use the "TARGETS" variable on the make command line to specify
83 single test to run, or a list of tests to run. 74 single test to run, or a list of tests to run.
84 75
85 To run only tests targeted for a single subsys 76 To run only tests targeted for a single subsystem::
86 77
87 $ make -C tools/testing/selftests TARGETS=pt 78 $ make -C tools/testing/selftests TARGETS=ptrace run_tests
88 79
89 You can specify multiple tests to build and ru 80 You can specify multiple tests to build and run::
90 81
91 $ make TARGETS="size timers" kselftest 82 $ make TARGETS="size timers" kselftest
92 83
93 To build, save output files in a separate dire 84 To build, save output files in a separate directory with O= ::
94 85
95 $ make O=/tmp/kselftest TARGETS="size timers 86 $ make O=/tmp/kselftest TARGETS="size timers" kselftest
96 87
97 To build, save output files in a separate dire 88 To build, save output files in a separate directory with KBUILD_OUTPUT ::
98 89
99 $ export KBUILD_OUTPUT=/tmp/kselftest; make 90 $ export KBUILD_OUTPUT=/tmp/kselftest; make TARGETS="size timers" kselftest
100 91
101 Additionally you can use the "SKIP_TARGETS" va 92 Additionally you can use the "SKIP_TARGETS" variable on the make command
102 line to specify one or more targets to exclude 93 line to specify one or more targets to exclude from the TARGETS list.
103 94
104 To run all tests but a single subsystem:: 95 To run all tests but a single subsystem::
105 96
106 $ make -C tools/testing/selftests SKIP_TARGE 97 $ make -C tools/testing/selftests SKIP_TARGETS=ptrace run_tests
107 98
108 You can specify multiple tests to skip:: 99 You can specify multiple tests to skip::
109 100
110 $ make SKIP_TARGETS="size timers" kselftest 101 $ make SKIP_TARGETS="size timers" kselftest
111 102
112 You can also specify a restricted list of test 103 You can also specify a restricted list of tests to run together with a
113 dedicated skiplist:: 104 dedicated skiplist::
114 105
115 $ make TARGETS="breakpoints size timers" SK !! 106 $ make TARGETS="bpf breakpoints size timers" SKIP_TARGETS=bpf kselftest
116 107
117 See the top-level tools/testing/selftests/Make 108 See the top-level tools/testing/selftests/Makefile for the list of all
118 possible targets. 109 possible targets.
119 110
120 Running the full range hotplug selftests 111 Running the full range hotplug selftests
121 ======================================== 112 ========================================
122 113
123 To build the hotplug tests:: 114 To build the hotplug tests::
124 115
125 $ make -C tools/testing/selftests hotplug 116 $ make -C tools/testing/selftests hotplug
126 117
127 To run the hotplug tests:: 118 To run the hotplug tests::
128 119
129 $ make -C tools/testing/selftests run_hotplu 120 $ make -C tools/testing/selftests run_hotplug
130 121
131 Note that some tests will require root privile 122 Note that some tests will require root privileges.
132 123
133 124
134 Install selftests 125 Install selftests
135 ================= 126 =================
136 127
137 You can use the "install" target of "make" (wh 128 You can use the "install" target of "make" (which calls the `kselftest_install.sh`
138 tool) to install selftests in the default loca 129 tool) to install selftests in the default location (`tools/testing/selftests/kselftest_install`),
139 or in a user specified location via the `INSTA 130 or in a user specified location via the `INSTALL_PATH` "make" variable.
140 131
141 To install selftests in default location:: 132 To install selftests in default location::
142 133
143 $ make -C tools/testing/selftests install 134 $ make -C tools/testing/selftests install
144 135
145 To install selftests in a user specified locat 136 To install selftests in a user specified location::
146 137
147 $ make -C tools/testing/selftests install I 138 $ make -C tools/testing/selftests install INSTALL_PATH=/some/other/path
148 139
149 Running installed selftests 140 Running installed selftests
150 =========================== 141 ===========================
151 142
152 Found in the install directory, as well as in 143 Found in the install directory, as well as in the Kselftest tarball,
153 is a script named `run_kselftest.sh` to run th 144 is a script named `run_kselftest.sh` to run the tests.
154 145
155 You can simply do the following to run the ins 146 You can simply do the following to run the installed Kselftests. Please
156 note some tests will require root privileges:: 147 note some tests will require root privileges::
157 148
158 $ cd kselftest_install 149 $ cd kselftest_install
159 $ ./run_kselftest.sh 150 $ ./run_kselftest.sh
160 151
161 To see the list of available tests, the `-l` o 152 To see the list of available tests, the `-l` option can be used::
162 153
163 $ ./run_kselftest.sh -l 154 $ ./run_kselftest.sh -l
164 155
165 The `-c` option can be used to run all the tes 156 The `-c` option can be used to run all the tests from a test collection, or
166 the `-t` option for specific single tests. Eit 157 the `-t` option for specific single tests. Either can be used multiple times::
167 158
168 $ ./run_kselftest.sh -c size -c seccomp -t !! 159 $ ./run_kselftest.sh -c bpf -c seccomp -t timers:posix_timers -t timer:nanosleep
169 160
170 For other features see the script usage output 161 For other features see the script usage output, seen with the `-h` option.
171 162
172 Timeout for selftests <<
173 ===================== <<
174 <<
175 Selftests are designed to be quick and so a de <<
176 seconds for each test. Tests can override the <<
177 a settings file in their directory and set a t <<
178 configured a desired upper timeout for the tes <<
179 the timeout with a value higher than 45 second <<
180 it that way. Timeouts in selftests are not con <<
181 system under which a test runs may change and <<
182 expected time it takes to run a test. If you h <<
183 which will run the tests you can configure a t <<
184 use a greater or lower timeout on the command <<
185 the `--override-timeout` argument. For example <<
186 one would use:: <<
187 <<
188 $ ./run_kselftest.sh --override-timeout 165 <<
189 <<
190 You can look at the TAP output to see if you r <<
191 runners which know a test must run under a spe <<
192 treat these timeouts then as fatal. <<
193 <<
194 Packaging selftests 163 Packaging selftests
195 =================== 164 ===================
196 165
197 In some cases packaging is desired, such as wh 166 In some cases packaging is desired, such as when tests need to run on a
198 different system. To package selftests, run:: 167 different system. To package selftests, run::
199 168
200 $ make -C tools/testing/selftests gen_tar 169 $ make -C tools/testing/selftests gen_tar
201 170
202 This generates a tarball in the `INSTALL_PATH/ 171 This generates a tarball in the `INSTALL_PATH/kselftest-packages` directory. By
203 default, `.gz` format is used. The tar compres 172 default, `.gz` format is used. The tar compression format can be overridden by
204 specifying a `FORMAT` make variable. Any value 173 specifying a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_
205 option is supported, such as:: 174 option is supported, such as::
206 175
207 $ make -C tools/testing/selftests gen_tar 176 $ make -C tools/testing/selftests gen_tar FORMAT=.xz
208 177
209 `make gen_tar` invokes `make install` so you c 178 `make gen_tar` invokes `make install` so you can use it to package a subset of
210 tests by using variables specified in `Running 179 tests by using variables specified in `Running a subset of selftests`_
211 section:: 180 section::
212 181
213 $ make -C tools/testing/selftests gen_tar !! 182 $ make -C tools/testing/selftests gen_tar TARGETS="bpf" FORMAT=.xz
214 183
215 .. _tar's auto-compress: https://www.gnu.org/s 184 .. _tar's auto-compress: https://www.gnu.org/software/tar/manual/html_node/gzip.html#auto_002dcompress
216 185
217 Contributing new tests 186 Contributing new tests
218 ====================== 187 ======================
219 188
220 In general, the rules for selftests are 189 In general, the rules for selftests are
221 190
222 * Do as much as you can if you're not root; 191 * Do as much as you can if you're not root;
223 192
224 * Don't take too long; 193 * Don't take too long;
225 194
226 * Don't break the build on any architecture, 195 * Don't break the build on any architecture, and
227 196
228 * Don't cause the top-level "make run_tests" 197 * Don't cause the top-level "make run_tests" to fail if your feature is
229 unconfigured. 198 unconfigured.
230 199
231 * The output of tests must conform to the TAP <<
232 testing quality and to capture failures/err <<
233 The kselftest.h and kselftest_harness.h hea <<
234 outputting test results. These wrappers sho <<
235 fail, exit, and skip messages. CI systems c <<
236 messages to detect test results. <<
237 <<
238 Contributing new tests (details) 200 Contributing new tests (details)
239 ================================ 201 ================================
240 202
241 * In your Makefile, use facilities from lib.m <<
242 reinventing the wheel. Specify flags and bi <<
243 need basis before including lib.mk. :: <<
244 <<
245 CFLAGS = $(KHDR_INCLUDES) <<
246 TEST_GEN_PROGS := close_range_test <<
247 include ../lib.mk <<
248 <<
249 * Use TEST_GEN_XXX if such binaries or files 203 * Use TEST_GEN_XXX if such binaries or files are generated during
250 compiling. 204 compiling.
251 205
252 TEST_PROGS, TEST_GEN_PROGS mean it is the e 206 TEST_PROGS, TEST_GEN_PROGS mean it is the executable tested by
253 default. 207 default.
254 208
255 TEST_GEN_MODS_DIR should be used by tests t <<
256 before the test starts. The variable will c <<
257 containing the modules. <<
258 <<
259 TEST_CUSTOM_PROGS should be used by tests t 209 TEST_CUSTOM_PROGS should be used by tests that require custom build
260 rules and prevent common build rule use. 210 rules and prevent common build rule use.
261 211
262 TEST_PROGS are for test shell scripts. Plea 212 TEST_PROGS are for test shell scripts. Please ensure shell script has
263 its exec bit set. Otherwise, lib.mk run_tes 213 its exec bit set. Otherwise, lib.mk run_tests will generate a warning.
264 214
265 TEST_CUSTOM_PROGS and TEST_PROGS will be ru 215 TEST_CUSTOM_PROGS and TEST_PROGS will be run by common run_tests.
266 216
267 TEST_PROGS_EXTENDED, TEST_GEN_PROGS_EXTENDE 217 TEST_PROGS_EXTENDED, TEST_GEN_PROGS_EXTENDED mean it is the
268 executable which is not tested by default. 218 executable which is not tested by default.
269 <<
270 TEST_FILES, TEST_GEN_FILES mean it is the f 219 TEST_FILES, TEST_GEN_FILES mean it is the file which is used by
271 test. 220 test.
272 221
273 TEST_INCLUDES is similar to TEST_FILES, it <<
274 included when exporting or installing the t <<
275 differences: <<
276 <<
277 * symlinks to files in other directories a <<
278 * the part of paths below tools/testing/se <<
279 copying the files to the output director <<
280 <<
281 TEST_INCLUDES is meant to list dependencies <<
282 the selftests hierarchy. <<
283 <<
284 * First use the headers inside the kernel sou 222 * First use the headers inside the kernel source and/or git repo, and then the
285 system headers. Headers for the kernel rel 223 system headers. Headers for the kernel release as opposed to headers
286 installed by the distro on the system shoul 224 installed by the distro on the system should be the primary focus to be able
287 to find regressions. Use KHDR_INCLUDES in M !! 225 to find regressions.
288 the kernel source. <<
289 226
290 * If a test needs specific kernel config opti 227 * If a test needs specific kernel config options enabled, add a config file in
291 the test directory to enable them. 228 the test directory to enable them.
292 229
293 e.g: tools/testing/selftests/android/config 230 e.g: tools/testing/selftests/android/config
294 231
295 * Create a .gitignore file inside test direct <<
296 in it. <<
297 <<
298 * Add new test name in TARGETS in selftests/M <<
299 <<
300 TARGETS += android <<
301 <<
302 * All changes should pass:: <<
303 <<
304 kselftest-{all,install,clean,gen_tar} <<
305 kselftest-{all,install,clean,gen_tar} O=ab <<
306 kselftest-{all,install,clean,gen_tar} O=re <<
307 make -C tools/testing/selftests {all,insta <<
308 make -C tools/testing/selftests {all,insta <<
309 make -C tools/testing/selftests {all,insta <<
310 <<
311 Test Module 232 Test Module
312 =========== 233 ===========
313 234
314 Kselftest tests the kernel from userspace. So 235 Kselftest tests the kernel from userspace. Sometimes things need
315 testing from within the kernel, one method of 236 testing from within the kernel, one method of doing this is to create a
316 test module. We can tie the module into the k 237 test module. We can tie the module into the kselftest framework by
317 using a shell script test runner. ``kselftest 238 using a shell script test runner. ``kselftest/module.sh`` is designed
318 to facilitate this process. There is also a h 239 to facilitate this process. There is also a header file provided to
319 assist writing kernel modules that are for use 240 assist writing kernel modules that are for use with kselftest:
320 241
321 - ``tools/testing/selftests/kselftest_module.h !! 242 - ``tools/testing/kselftest/kselftest_module.h``
322 - ``tools/testing/selftests/kselftest/module.s !! 243 - ``tools/testing/kselftest/kselftest/module.sh``
323 <<
324 Note that test modules should taint the kernel <<
325 happen automatically for modules which are in <<
326 directory, or for modules which use the ``ksel <<
327 Otherwise, you'll need to add ``MODULE_INFO(te <<
328 source. selftests which do not load modules ty <<
329 kernel, but in cases where a non-test module i <<
330 applied from userspace by writing to ``/proc/s <<
331 244
332 How to use 245 How to use
333 ---------- 246 ----------
334 247
335 Here we show the typical steps to create a tes 248 Here we show the typical steps to create a test module and tie it into
336 kselftest. We use kselftests for lib/ as an e 249 kselftest. We use kselftests for lib/ as an example.
337 250
338 1. Create the test module 251 1. Create the test module
339 252
340 2. Create the test script that will run (load/ 253 2. Create the test script that will run (load/unload) the module
341 e.g. ``tools/testing/selftests/lib/printf.s 254 e.g. ``tools/testing/selftests/lib/printf.sh``
342 255
343 3. Add line to config file e.g. ``tools/testin 256 3. Add line to config file e.g. ``tools/testing/selftests/lib/config``
344 257
345 4. Add test script to makefile e.g. ``tools/t 258 4. Add test script to makefile e.g. ``tools/testing/selftests/lib/Makefile``
346 259
347 5. Verify it works: 260 5. Verify it works:
348 261
349 .. code-block:: sh 262 .. code-block:: sh
350 263
351 # Assumes you have booted a fresh build of 264 # Assumes you have booted a fresh build of this kernel tree
352 cd /path/to/linux/tree 265 cd /path/to/linux/tree
353 make kselftest-merge 266 make kselftest-merge
354 make modules 267 make modules
355 sudo make modules_install 268 sudo make modules_install
356 make TARGETS=lib kselftest 269 make TARGETS=lib kselftest
357 270
358 Example Module 271 Example Module
359 -------------- 272 --------------
360 273
361 A bare bones test module might look like this: 274 A bare bones test module might look like this:
362 275
363 .. code-block:: c 276 .. code-block:: c
364 277
365 // SPDX-License-Identifier: GPL-2.0+ 278 // SPDX-License-Identifier: GPL-2.0+
366 279
367 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt 280 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
368 281
369 #include "../tools/testing/selftests/kselft !! 282 #include "../tools/testing/selftests/kselftest/module.h"
370 283
371 KSTM_MODULE_GLOBALS(); 284 KSTM_MODULE_GLOBALS();
372 285
373 /* 286 /*
374 * Kernel module for testing the foobinator 287 * Kernel module for testing the foobinator
375 */ 288 */
376 289
377 static int __init test_function() 290 static int __init test_function()
378 { 291 {
379 ... 292 ...
380 } 293 }
381 294
382 static void __init selftest(void) 295 static void __init selftest(void)
383 { 296 {
384 KSTM_CHECK_ZERO(do_test_case("", 0) 297 KSTM_CHECK_ZERO(do_test_case("", 0));
385 } 298 }
386 299
387 KSTM_MODULE_LOADERS(test_foo); 300 KSTM_MODULE_LOADERS(test_foo);
388 MODULE_AUTHOR("John Developer <jd@fooman.org 301 MODULE_AUTHOR("John Developer <jd@fooman.org>");
389 MODULE_LICENSE("GPL"); 302 MODULE_LICENSE("GPL");
390 MODULE_INFO(test, "Y"); <<
391 303
392 Example test script 304 Example test script
393 ------------------- 305 -------------------
394 306
395 .. code-block:: sh 307 .. code-block:: sh
396 308
397 #!/bin/bash 309 #!/bin/bash
398 # SPDX-License-Identifier: GPL-2.0+ 310 # SPDX-License-Identifier: GPL-2.0+
399 $(dirname $0)/../kselftest/module.sh "foo" 311 $(dirname $0)/../kselftest/module.sh "foo" test_foo
400 312
401 313
402 Test Harness 314 Test Harness
403 ============ 315 ============
404 316
405 The kselftest_harness.h file contains useful h 317 The kselftest_harness.h file contains useful helpers to build tests. The
406 test harness is for userspace testing, for ker 318 test harness is for userspace testing, for kernel space testing see `Test
407 Module`_ above. 319 Module`_ above.
408 320
409 The tests from tools/testing/selftests/seccomp 321 The tests from tools/testing/selftests/seccomp/seccomp_bpf.c can be used as
410 example. 322 example.
411 323
412 Example 324 Example
413 ------- 325 -------
414 326
415 .. kernel-doc:: tools/testing/selftests/kselft 327 .. kernel-doc:: tools/testing/selftests/kselftest_harness.h
416 :doc: example 328 :doc: example
417 329
418 330
419 Helpers 331 Helpers
420 ------- 332 -------
421 333
422 .. kernel-doc:: tools/testing/selftests/kselft 334 .. kernel-doc:: tools/testing/selftests/kselftest_harness.h
423 :functions: TH_LOG TEST TEST_SIGNAL FIXTUR 335 :functions: TH_LOG TEST TEST_SIGNAL FIXTURE FIXTURE_DATA FIXTURE_SETUP
424 FIXTURE_TEARDOWN TEST_F TEST_H 336 FIXTURE_TEARDOWN TEST_F TEST_HARNESS_MAIN FIXTURE_VARIANT
425 FIXTURE_VARIANT_ADD 337 FIXTURE_VARIANT_ADD
426 338
427 Operators 339 Operators
428 --------- 340 ---------
429 341
430 .. kernel-doc:: tools/testing/selftests/kselft 342 .. kernel-doc:: tools/testing/selftests/kselftest_harness.h
431 :doc: operators 343 :doc: operators
432 344
433 .. kernel-doc:: tools/testing/selftests/kselft 345 .. kernel-doc:: tools/testing/selftests/kselftest_harness.h
434 :functions: ASSERT_EQ ASSERT_NE ASSERT_LT 346 :functions: ASSERT_EQ ASSERT_NE ASSERT_LT ASSERT_LE ASSERT_GT ASSERT_GE
435 ASSERT_NULL ASSERT_TRUE ASSERT 347 ASSERT_NULL ASSERT_TRUE ASSERT_NULL ASSERT_TRUE ASSERT_FALSE
436 ASSERT_STREQ ASSERT_STRNE EXPE 348 ASSERT_STREQ ASSERT_STRNE EXPECT_EQ EXPECT_NE EXPECT_LT
437 EXPECT_LE EXPECT_GT EXPECT_GE 349 EXPECT_LE EXPECT_GT EXPECT_GE EXPECT_NULL EXPECT_TRUE
438 EXPECT_FALSE EXPECT_STREQ EXPE 350 EXPECT_FALSE EXPECT_STREQ EXPECT_STRNE
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.