1 .. SPDX-License-Identifier: GPL-2.0 1 .. SPDX-License-Identifier: GPL-2.0
2 2
3 ================== 3 ==================
4 KUnit Architecture 4 KUnit Architecture
5 ================== 5 ==================
6 6
7 The KUnit architecture is divided into two par !! 7 The KUnit architecture can be divided into two parts:
8 8
9 - `In-Kernel Testing Framework`_ !! 9 - Kernel testing library
10 - `kunit_tool (Command-line Test Harness)`_ !! 10 - kunit_tool (Command line test harness)
11 11
12 In-Kernel Testing Framework 12 In-Kernel Testing Framework
13 =========================== 13 ===========================
14 14
15 The kernel testing library supports KUnit test 15 The kernel testing library supports KUnit tests written in C using
16 KUnit. These KUnit tests are kernel code. KUni !! 16 KUnit. KUnit tests are kernel code. KUnit does several things:
17 tasks: <<
18 17
19 - Organizes tests 18 - Organizes tests
20 - Reports test results 19 - Reports test results
21 - Provides test utilities 20 - Provides test utilities
22 21
23 Test Cases 22 Test Cases
24 ---------- 23 ----------
25 24
26 The test case is the fundamental unit in KUnit !! 25 The fundamental unit in KUnit is the test case. The KUnit test cases are
27 into suites. A KUnit test case is a function w !! 26 grouped into KUnit suites. A KUnit test case is a function with type
28 ``void (*)(struct kunit *test)``. These test c !! 27 signature ``void (*)(struct kunit *test)``.
29 struct called struct kunit_case. !! 28 These test case functions are wrapped in a struct called
>> 29 struct kunit_case.
30 30
31 .. note: 31 .. note:
32 ``generate_params`` is optional for no 32 ``generate_params`` is optional for non-parameterized tests.
33 33
34 Each KUnit test case receives a ``struct kunit !! 34 Each KUnit test case gets a ``struct kunit`` context
35 running test. The KUnit assertion macros and o !! 35 object passed to it that tracks a running test. The KUnit assertion
36 ``struct kunit`` context object. As an excepti !! 36 macros and other KUnit utilities use the ``struct kunit`` context
>> 37 object. As an exception, there are two fields:
37 38
38 - ``->priv``: The setup functions can use it t 39 - ``->priv``: The setup functions can use it to store arbitrary test
39 user data. 40 user data.
40 41
41 - ``->param_value``: It contains the parameter 42 - ``->param_value``: It contains the parameter value which can be
42 retrieved in the parameterized tests. 43 retrieved in the parameterized tests.
43 44
44 Test Suites 45 Test Suites
45 ----------- 46 -----------
46 47
47 A KUnit suite includes a collection of test ca 48 A KUnit suite includes a collection of test cases. The KUnit suites
48 are represented by the ``struct kunit_suite``. 49 are represented by the ``struct kunit_suite``. For example:
49 50
50 .. code-block:: c 51 .. code-block:: c
51 52
52 static struct kunit_case example_test_ 53 static struct kunit_case example_test_cases[] = {
53 KUNIT_CASE(example_test_foo), 54 KUNIT_CASE(example_test_foo),
54 KUNIT_CASE(example_test_bar), 55 KUNIT_CASE(example_test_bar),
55 KUNIT_CASE(example_test_baz), 56 KUNIT_CASE(example_test_baz),
56 {} 57 {}
57 }; 58 };
58 59
59 static struct kunit_suite example_test 60 static struct kunit_suite example_test_suite = {
60 .name = "example", 61 .name = "example",
61 .init = example_test_init, 62 .init = example_test_init,
62 .exit = example_test_exit, 63 .exit = example_test_exit,
63 .test_cases = example_test_cas 64 .test_cases = example_test_cases,
64 }; 65 };
65 kunit_test_suite(example_test_suite); 66 kunit_test_suite(example_test_suite);
66 67
67 In the above example, the test suite ``example 68 In the above example, the test suite ``example_test_suite``, runs the
68 test cases ``example_test_foo``, ``example_tes 69 test cases ``example_test_foo``, ``example_test_bar``, and
69 ``example_test_baz``. Before running the test, 70 ``example_test_baz``. Before running the test, the ``example_test_init``
70 is called and after running the test, ``exampl 71 is called and after running the test, ``example_test_exit`` is called.
71 The ``kunit_test_suite(example_test_suite)`` r 72 The ``kunit_test_suite(example_test_suite)`` registers the test suite
72 with the KUnit test framework. 73 with the KUnit test framework.
73 74
74 Executor 75 Executor
75 -------- 76 --------
76 77
77 The KUnit executor can list and run built-in K 78 The KUnit executor can list and run built-in KUnit tests on boot.
78 The Test suites are stored in a linker section 79 The Test suites are stored in a linker section
79 called ``.kunit_test_suites``. For the code, s !! 80 called ``.kunit_test_suites``. For code, see:
80 definition in !! 81 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/asm-generic/vmlinux.lds.h?h=v5.15#n945.
81 `include/asm-generic/vmlinux.lds.h <https://gi <<
82 The linker section consists of an array of poi 82 The linker section consists of an array of pointers to
83 ``struct kunit_suite``, and is populated by th 83 ``struct kunit_suite``, and is populated by the ``kunit_test_suites()``
84 macro. The KUnit executor iterates over the li !! 84 macro. To run all tests compiled into the kernel, the KUnit executor
85 run all the tests that are compiled into the k !! 85 iterates over the linker section array.
86 86
87 .. kernel-figure:: kunit_suitememorydiagram.sv 87 .. kernel-figure:: kunit_suitememorydiagram.svg
88 :alt: KUnit Suite Memory 88 :alt: KUnit Suite Memory
89 89
90 KUnit Suite Memory Diagram 90 KUnit Suite Memory Diagram
91 91
92 On the kernel boot, the KUnit executor uses th 92 On the kernel boot, the KUnit executor uses the start and end addresses
93 of this section to iterate over and run all te !! 93 of this section to iterate over and run all tests. For code, see:
94 executor, see !! 94 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/executor.c
95 `lib/kunit/executor.c <https://git.kernel.org/ !! 95
96 When built as a module, the ``kunit_test_suite 96 When built as a module, the ``kunit_test_suites()`` macro defines a
97 ``module_init()`` function, which runs all the 97 ``module_init()`` function, which runs all the tests in the compilation
98 unit instead of utilizing the executor. 98 unit instead of utilizing the executor.
99 99
100 In KUnit tests, some error classes do not affe 100 In KUnit tests, some error classes do not affect other tests
101 or parts of the kernel, each KUnit case execut 101 or parts of the kernel, each KUnit case executes in a separate thread
102 context. See the ``kunit_try_catch_run()`` fun !! 102 context. For code, see:
103 `lib/kunit/try-catch.c <https://git.kernel.org !! 103 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/try-catch.c?h=v5.15#n58
104 104
105 Assertion Macros 105 Assertion Macros
106 ---------------- 106 ----------------
107 107
108 KUnit tests verify state using expectations/as 108 KUnit tests verify state using expectations/assertions.
109 All expectations/assertions are formatted as: 109 All expectations/assertions are formatted as:
110 ``KUNIT_{EXPECT|ASSERT}_<op>[_MSG](kunit, prop 110 ``KUNIT_{EXPECT|ASSERT}_<op>[_MSG](kunit, property[, message])``
111 111
112 - ``{EXPECT|ASSERT}`` determines whether the c 112 - ``{EXPECT|ASSERT}`` determines whether the check is an assertion or an
113 expectation. 113 expectation.
114 In the event of a failure, the testing flow <<
115 114
116 - For expectations, the test is marked !! 115 - For an expectation, if the check fails, marks the test as failed
>> 116 and logs the failure.
117 117
118 - Failing assertions, on the other han !! 118 - An assertion, on failure, causes the test case to terminate
119 terminated immediately. !! 119 immediately.
120 120
121 - Assertions call the function !! 121 - Assertions call function:
122 ``void __noreturn __kunit_ab !! 122 ``void __noreturn kunit_abort(struct kunit *)``.
123 123
124 - ``__kunit_abort`` calls the !! 124 - ``kunit_abort`` calls function:
125 ``void __noreturn kunit_try_ 125 ``void __noreturn kunit_try_catch_throw(struct kunit_try_catch *try_catch)``.
126 126
127 - ``kunit_try_catch_throw`` ca !! 127 - ``kunit_try_catch_throw`` calls function:
128 ``void kthread_complete_and_ !! 128 ``void complete_and_exit(struct completion *, long) __noreturn;``
129 and terminates the special t 129 and terminates the special thread context.
130 130
131 - ``<op>`` denotes a check with options: ``TRU 131 - ``<op>`` denotes a check with options: ``TRUE`` (supplied property
132 has the boolean value "true"), ``EQ`` (two s !! 132 has the boolean value “true”), ``EQ`` (two supplied properties are
133 equal), ``NOT_ERR_OR_NULL`` (supplied pointe 133 equal), ``NOT_ERR_OR_NULL`` (supplied pointer is not null and does not
134 contain an "err" value). !! 134 contain an “err” value).
135 135
136 - ``[_MSG]`` prints a custom message on failur 136 - ``[_MSG]`` prints a custom message on failure.
137 137
138 Test Result Reporting 138 Test Result Reporting
139 --------------------- 139 ---------------------
140 KUnit prints the test results in KTAP format. !! 140 KUnit prints test results in KTAP format. KTAP is based on TAP14, see:
141 Documentation/dev-tools/ktap.rst. !! 141 https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md.
142 KTAP works with KUnit and Kselftest. The KUnit !! 142 KTAP (yet to be standardized format) works with KUnit and Kselftest.
143 dmesg, and debugfs (if configured). !! 143 The KUnit executor prints KTAP results to dmesg, and debugfs
>> 144 (if configured).
144 145
145 Parameterized Tests 146 Parameterized Tests
146 ------------------- 147 -------------------
147 148
148 Each KUnit parameterized test is associated wi 149 Each KUnit parameterized test is associated with a collection of
149 parameters. The test is invoked multiple times 150 parameters. The test is invoked multiple times, once for each parameter
150 value and the parameter is stored in the ``par 151 value and the parameter is stored in the ``param_value`` field.
151 The test case includes a KUNIT_CASE_PARAM() ma 152 The test case includes a KUNIT_CASE_PARAM() macro that accepts a
152 generator function. The generator function is !! 153 generator function.
153 and returns the next parameter. It also includ !! 154 The generator function is passed the previous parameter and returns the next
154 array-based common-case generators. !! 155 parameter. It also provides a macro to generate common-case generators based on
>> 156 arrays.
155 157
156 kunit_tool (Command-line Test Harness) !! 158 kunit_tool (Command Line Test Harness)
157 ====================================== 159 ======================================
158 160
159 ``kunit_tool`` is a Python script, found in `` !! 161 kunit_tool is a Python script ``(tools/testing/kunit/kunit.py)``
160 is used to configure, build, execute, parse te !! 162 that can be used to configure, build, exec, parse and run (runs other
161 previous commands in correct order (i.e., conf !! 163 commands in order) test results. You can either run KUnit tests using
162 You have two options for running KUnit tests: !! 164 kunit_tool or can include KUnit in kernel and parse manually.
163 enabled and manually parse the results (see <<
164 Documentation/dev-tools/kunit/run_manual.rst) <<
165 (see Documentation/dev-tools/kunit/run_wrapper <<
166 165
167 - ``configure`` command generates the kernel ` 166 - ``configure`` command generates the kernel ``.config`` from a
168 ``.kunitconfig`` file (and any architecture- 167 ``.kunitconfig`` file (and any architecture-specific options).
169 The Python scripts available in ``qemu_confi !! 168 For some architectures, additional config options are specified in the
170 (for example, ``tools/testing/kunit/qemu con !! 169 ``qemu_config`` Python script
171 additional configuration options for specifi !! 170 (For example: ``tools/testing/kunit/qemu_configs/powerpc.py``).
172 It parses both the existing ``.config`` and 171 It parses both the existing ``.config`` and the ``.kunitconfig`` files
173 to ensure that ``.config`` is a superset of !! 172 and ensures that ``.config`` is a superset of ``.kunitconfig``.
174 If not, it will combine the two and run ``ma !! 173 If this is not the case, it will combine the two and run
175 the ``.config`` file. It then checks to see !! 174 ``make olddefconfig`` to regenerate the ``.config`` file. It then
176 This verifies that all the Kconfig dependenc !! 175 verifies that ``.config`` is now a superset. This checks if all
177 file ``.kunitconfig``. The ``kunit_config.py !! 176 Kconfig dependencies are correctly specified in ``.kunitconfig``.
178 Kconfigs. The code which runs ``make olddefc !! 177 ``kunit_config.py`` includes the parsing Kconfigs code. The code which
179 ``kunit_kernel.py`` script. You can invoke t !! 178 runs ``make olddefconfig`` is a part of ``kunit_kernel.py``. You can
180 ``./tools/testing/kunit/kunit.py config`` an !! 179 invoke this command via: ``./tools/testing/kunit/kunit.py config`` and
181 generate a ``.config`` file. 180 generate a ``.config`` file.
182 - ``build`` runs ``make`` on the kernel tree w 181 - ``build`` runs ``make`` on the kernel tree with required options
183 (depends on the architecture and some option 182 (depends on the architecture and some options, for example: build_dir)
184 and reports any errors. 183 and reports any errors.
185 To build a KUnit kernel from the current ``. 184 To build a KUnit kernel from the current ``.config``, you can use the
186 ``build`` argument: ``./tools/testing/kunit/ 185 ``build`` argument: ``./tools/testing/kunit/kunit.py build``.
187 - ``exec`` command executes kernel results eit 186 - ``exec`` command executes kernel results either directly (using
188 User-mode Linux configuration), or through a !! 187 User-mode Linux configuration), or via an emulator such
189 as QEMU. It reads results from the log using !! 188 as QEMU. It reads results from the log via standard
190 output (stdout), and passes them to ``parse` 189 output (stdout), and passes them to ``parse`` to be parsed.
191 If you already have built a kernel with buil 190 If you already have built a kernel with built-in KUnit tests,
192 you can run the kernel and display the test 191 you can run the kernel and display the test results with the ``exec``
193 argument: ``./tools/testing/kunit/kunit.py e 192 argument: ``./tools/testing/kunit/kunit.py exec``.
194 - ``parse`` extracts the KTAP output from a ke 193 - ``parse`` extracts the KTAP output from a kernel log, parses
195 the test results, and prints a summary. For 194 the test results, and prints a summary. For failed tests, any
196 diagnostic output will be included. 195 diagnostic output will be included.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.