1 .. SPDX-License-Identifier: GPL-2.0
2
3 ============================
4 Tips For Running KUnit Tests
5 ============================
6
7 Using ``kunit.py run`` ("kunit tool")
8 =====================================
9
10 Running from any directory
11 --------------------------
12
13 It can be handy to create a bash function like
14
15 .. code-block:: bash
16
17 function run_kunit() {
18 ( cd "$(git rev-parse --show-topleve
19 }
20
21 .. note::
22 Early versions of ``kunit.py`` (before
23 the kernel root, hence the use of a su
24
25 Running a subset of tests
26 -------------------------
27
28 ``kunit.py run`` accepts an optional glob argu
29 is ``"<suite_glob>[.test_glob]"``.
30
31 Say that we wanted to run the sysctl tests, we
32
33 .. code-block:: bash
34
35 $ echo -e 'CONFIG_KUNIT=y\nCONFIG_KUNI
36 $ ./tools/testing/kunit/kunit.py run '
37
38 We can filter down to just the "write" tests v
39
40 .. code-block:: bash
41
42 $ echo -e 'CONFIG_KUNIT=y\nCONFIG_KUNI
43 $ ./tools/testing/kunit/kunit.py run '
44
45 We're paying the cost of building more tests t
46 easier than fiddling with ``.kunitconfig`` fil
47 ``kunit_suite``'s.
48
49 However, if we wanted to define a set of tests
50 tip is useful.
51
52 Defining a set of tests
53 -----------------------
54
55 ``kunit.py run`` (along with ``build``, and ``
56 ``--kunitconfig`` flag. So if you have a set o
57 regular basis (especially if they have other d
58 specific ``.kunitconfig`` for them.
59
60 E.g. kunit has one for its tests:
61
62 .. code-block:: bash
63
64 $ ./tools/testing/kunit/kunit.py run -
65
66 Alternatively, if you're following the convent
67 file ``.kunitconfig``, you can just pass in th
68
69 .. code-block:: bash
70
71 $ ./tools/testing/kunit/kunit.py run -
72
73 .. note::
74 This is a relatively new feature (5.12
75 conventions yet about on what files sh
76 kept around locally. It's up to you an
77 config is useful enough to submit (and
78
79 .. note::
80 Having ``.kunitconfig`` fragments in a
81 iffy. There's discussion about adding
82 files to make it possible to have a to
83 child directories. But that would mean
84 longer just simple .config fragments.
85
86 One alternative would be to have kunit
87 automagically, but tests could theoret
88 options, so handling that would be tri
89
90 Setting kernel commandline parameters
91 -------------------------------------
92
93 You can use ``--kernel_args`` to pass arbitrar
94
95 .. code-block:: bash
96
97 $ ./tools/testing/kunit/kunit.py run -
98
99
100 Generating code coverage reports under UML
101 ------------------------------------------
102
103 .. note::
104 TODO(brendanhiggins@google.com): There
105 versions of gcc 7 and up. You're likel
106 files or compile errors.
107
108 This is different from the "normal" way of get
109 documented in Documentation/dev-tools/gcov.rst
110
111 Instead of enabling ``CONFIG_GCOV_KERNEL=y``,
112
113 .. code-block:: none
114
115 CONFIG_DEBUG_KERNEL=y
116 CONFIG_DEBUG_INFO=y
117 CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFA
118 CONFIG_GCOV=y
119
120
121 Putting it together into a copy-pastable seque
122
123 .. code-block:: bash
124
125 # Append coverage options to the curre
126 $ ./tools/testing/kunit/kunit.py run -
127 # Extract the coverage information fro
128 $ lcov -t "my_kunit_tests" -o coverage
129
130 # From here on, it's the same process
131 # E.g. can generate an HTML report in
132 $ genhtml -o /tmp/coverage_html covera
133
134
135 If your installed version of gcc doesn't work,
136
137 .. code-block:: bash
138
139 $ ./tools/testing/kunit/kunit.py run -
140 $ lcov -t "my_kunit_tests" -o coverage
141
142 Alternatively, LLVM-based toolchains can also
143
144 .. code-block:: bash
145
146 # Build with LLVM and append coverage
147 $ ./tools/testing/kunit/kunit.py run -
148 $ llvm-profdata merge -sparse default.
149 $ llvm-cov export --format=lcov .kunit
150 # The coverage.info file is in lcov-co
151 $ genhtml -o /tmp/coverage_html covera
152
153
154 Running tests manually
155 ======================
156
157 Running tests without using ``kunit.py run`` i
158 Currently it's your only option if you want to
159 UML.
160
161 As running the tests under UML is fairly strai
162 the kernel, run the ``./linux`` binary), this
163 non-UML architectures.
164
165
166 Running built-in tests
167 ----------------------
168
169 When setting tests to ``=y``, the tests will r
170 results to dmesg in TAP format. So you just ne
171 ``.config``, build and boot your kernel as nor
172
173 So if we compiled our kernel with:
174
175 .. code-block:: none
176
177 CONFIG_KUNIT=y
178 CONFIG_KUNIT_EXAMPLE_TEST=y
179
180 Then we'd see output like this in dmesg signal
181
182 .. code-block:: none
183
184 TAP version 14
185 1..1
186 # Subtest: example
187 1..1
188 # example_simple_test: initializin
189 ok 1 - example_simple_test
190 ok 1 - example
191
192 Running tests as modules
193 ------------------------
194
195 Depending on the tests, you can build them as
196
197 For example, we'd change the config options fr
198
199 .. code-block:: none
200
201 CONFIG_KUNIT=y
202 CONFIG_KUNIT_EXAMPLE_TEST=m
203
204 Then after booting into our kernel, we can run
205
206 .. code-block:: none
207
208 $ modprobe kunit-example-test
209
210 This will then cause it to print TAP output to
211
212 .. note::
213 The ``modprobe`` will *not* have a non
214 failed (as of 5.13). But ``kunit.py pa
215
216 .. note::
217 You can set ``CONFIG_KUNIT=m`` as well
218 work and thus some tests might break.
219 depend on ``KUNIT=y`` in their ``Kconf
220 most test authors won't think about.
221 As of 5.13, the only difference is tha
222 not exist.
223
224 Pretty-printing results
225 -----------------------
226
227 You can use ``kunit.py parse`` to parse dmesg
228 results in the same familiar format that ``kun
229
230 .. code-block:: bash
231
232 $ ./tools/testing/kunit/kunit.py parse
233
234
235 Retrieving per suite results
236 ----------------------------
237
238 Regardless of how you're running your tests, y
239 ``CONFIG_KUNIT_DEBUGFS`` to expose per-suite T
240
241 .. code-block:: none
242
243 CONFIG_KUNIT=y
244 CONFIG_KUNIT_EXAMPLE_TEST=m
245 CONFIG_KUNIT_DEBUGFS=y
246
247 The results for each suite will be exposed und
248 ``/sys/kernel/debug/kunit/<suite>/results``.
249 So using our example config:
250
251 .. code-block:: bash
252
253 $ modprobe kunit-example-test > /dev/n
254 $ cat /sys/kernel/debug/kunit/example/
255 ... <TAP output> ...
256
257 # After removing the module, the corre
258 $ modprobe -r kunit-example-test
259 $ cat /sys/kernel/debug/kunit/example/
260 /sys/kernel/debug/kunit/example/result
261
262 Generating code coverage reports
263 --------------------------------
264
265 See Documentation/dev-tools/gcov.rst for detai
266
267 The only vaguely KUnit-specific advice here is
268 your tests as modules. That way you can isolat
269 other code executed during boot, e.g.
270
271 .. code-block:: bash
272
273 # Reset coverage counters before runni
274 $ echo 0 > /sys/kernel/debug/gcov/rese
275 $ modprobe kunit-example-test
276
277
278 Test Attributes and Filtering
279 =============================
280
281 Test suites and cases can be marked with test
282 test. These attributes will later be printed i
283 filter test execution.
284
285 Marking Test Attributes
286 -----------------------
287
288 Tests are marked with an attribute by includin
289 in the test definition.
290
291 Test cases can be marked using the ``KUNIT_CAS
292 macro to define the test case instead of ``KUN
293
294 .. code-block:: c
295
296 static const struct kunit_attributes e
297 .speed = KUNIT_VERY_SLOW,
298 };
299
300 static struct kunit_case example_test_
301 KUNIT_CASE_ATTR(example_test,
302 };
303
304 .. note::
305 To mark a test case as slow, you can a
306 This is a helpful macro as the slow at
307
308 Test suites can be marked with an attribute by
309 suite definition.
310
311 .. code-block:: c
312
313 static const struct kunit_attributes e
314 .speed = KUNIT_VERY_SLOW,
315 };
316
317 static struct kunit_suite example_test
318 ...,
319 .attr = example_attr,
320 };
321
322 .. note::
323 Not all attributes need to be set in a
324 attributes will remain uninitialized a
325 to 0 or NULL. Thus, if an attribute is
326 These unset attributes will not be rep
327 for filtering purposes.
328
329 Reporting Attributes
330 --------------------
331
332 When a user runs tests, attributes will be pre
333 KTAP format). Note that attributes will be hid
334 for all passing tests but the raw kernel outpu
335 ``--raw_output`` flag. This is an example of h
336 will be formatted in kernel output:
337
338 .. code-block:: none
339
340 # example_test.speed: slow
341 ok 1 example_test
342
343 This is an example of how test attributes for
344 kernel output:
345
346 .. code-block:: none
347
348 KTAP version 2
349 # Subtest: example_suite
350 # module: kunit_example_test
351 1..3
352 ...
353 ok 1 example_suite
354
355 Additionally, users can output a full attribut
356 attributes, using the command line flag ``--li
357
358 .. code-block:: bash
359
360 kunit.py run "example" --list_tests_at
361
362 .. note::
363 This report can be accessed when runni
364 module_param ``kunit.action=list_attr`
365
366 Filtering
367 ---------
368
369 Users can filter tests using the ``--filter``
370 tests. As an example:
371
372 .. code-block:: bash
373
374 kunit.py run --filter speed=slow
375
376
377 You can also use the following operations on f
378 "!=", and "=". Example:
379
380 .. code-block:: bash
381
382 kunit.py run --filter "speed>slow"
383
384 This example will run all tests with speeds fa
385 characters < and > are often interpreted by th
386 quoted or escaped, as above.
387
388 Additionally, you can use multiple filters at
389 using commas. Example:
390
391 .. code-block:: bash
392
393 kunit.py run --filter "speed>slow, mod
394
395 .. note::
396 You can use this filtering feature whe
397 the filter as a module param: ``kunit.
398
399 Filtered tests will not run or show up in the
400 ``--filter_action=skip`` flag to skip filtered
401 shown in the test output in the test but will
402 running KUnit manually, use the module param `
403
404 Rules of Filtering Procedure
405 ----------------------------
406
407 Since both suites and test cases can have attr
408 between attributes during filtering. The proce
409 rules:
410
411 - Filtering always operates at a per-test leve
412
413 - If a test has an attribute set, then the tes
414
415 - Otherwise, the value falls back to the suite
416
417 - If neither are set, the attribute has a glob
418
419 List of Current Attributes
420 --------------------------
421
422 ``speed``
423
424 This attribute indicates the speed of a test's
425 test is).
426
427 This attribute is saved as an enum with the fo
428 "slow", or "very_slow". The assumed default sp
429 indicates that the test takes a relatively tri
430 1 second), regardless of the machine it is run
431 this could be marked as "slow" or "very_slow".
432
433 The macro ``KUNIT_CASE_SLOW(test_name)`` can b
434 of a test case to "slow".
435
436 ``module``
437
438 This attribute indicates the name of the modul
439
440 This attribute is automatically saved as a str
441 Tests can also be filtered using this attribut
442
443 ``is_init``
444
445 This attribute indicates whether the test uses
446
447 This attribute is automatically saved as a boo
448 filtered using this attribute.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.