~ [ source navigation ] ~ [ diff markup ] ~ [ identifier search ] ~

TOMOYO Linux Cross Reference
Linux/Documentation/dev-tools/kunit/running_tips.rst

Version: ~ [ linux-6.12-rc7 ] ~ [ linux-6.11.7 ] ~ [ linux-6.10.14 ] ~ [ linux-6.9.12 ] ~ [ linux-6.8.12 ] ~ [ linux-6.7.12 ] ~ [ linux-6.6.60 ] ~ [ linux-6.5.13 ] ~ [ linux-6.4.16 ] ~ [ linux-6.3.13 ] ~ [ linux-6.2.16 ] ~ [ linux-6.1.116 ] ~ [ linux-6.0.19 ] ~ [ linux-5.19.17 ] ~ [ linux-5.18.19 ] ~ [ linux-5.17.15 ] ~ [ linux-5.16.20 ] ~ [ linux-5.15.171 ] ~ [ linux-5.14.21 ] ~ [ linux-5.13.19 ] ~ [ linux-5.12.19 ] ~ [ linux-5.11.22 ] ~ [ linux-5.10.229 ] ~ [ linux-5.9.16 ] ~ [ linux-5.8.18 ] ~ [ linux-5.7.19 ] ~ [ linux-5.6.19 ] ~ [ linux-5.5.19 ] ~ [ linux-5.4.285 ] ~ [ linux-5.3.18 ] ~ [ linux-5.2.21 ] ~ [ linux-5.1.21 ] ~ [ linux-5.0.21 ] ~ [ linux-4.20.17 ] ~ [ linux-4.19.323 ] ~ [ linux-4.18.20 ] ~ [ linux-4.17.19 ] ~ [ linux-4.16.18 ] ~ [ linux-4.15.18 ] ~ [ linux-4.14.336 ] ~ [ linux-4.13.16 ] ~ [ linux-4.12.14 ] ~ [ linux-4.11.12 ] ~ [ linux-4.10.17 ] ~ [ linux-4.9.337 ] ~ [ linux-4.4.302 ] ~ [ linux-3.10.108 ] ~ [ linux-2.6.32.71 ] ~ [ linux-2.6.0 ] ~ [ linux-2.4.37.11 ] ~ [ unix-v6-master ] ~ [ ccs-tools-1.8.12 ] ~ [ policy-sample ] ~
Architecture: ~ [ i386 ] ~ [ alpha ] ~ [ m68k ] ~ [ mips ] ~ [ ppc ] ~ [ sparc ] ~ [ sparc64 ] ~

Diff markup

Differences between /Documentation/dev-tools/kunit/running_tips.rst (Architecture sparc64) and /Documentation/dev-tools/kunit/running_tips.rst (Architecture m68k)


  1 .. SPDX-License-Identifier: GPL-2.0                 1 .. SPDX-License-Identifier: GPL-2.0
  2                                                     2 
  3 ============================                        3 ============================
  4 Tips For Running KUnit Tests                        4 Tips For Running KUnit Tests
  5 ============================                        5 ============================
  6                                                     6 
  7 Using ``kunit.py run`` ("kunit tool")               7 Using ``kunit.py run`` ("kunit tool")
  8 =====================================               8 =====================================
  9                                                     9 
 10 Running from any directory                         10 Running from any directory
 11 --------------------------                         11 --------------------------
 12                                                    12 
 13 It can be handy to create a bash function like     13 It can be handy to create a bash function like:
 14                                                    14 
 15 .. code-block:: bash                               15 .. code-block:: bash
 16                                                    16 
 17         function run_kunit() {                     17         function run_kunit() {
 18           ( cd "$(git rev-parse --show-topleve     18           ( cd "$(git rev-parse --show-toplevel)" && ./tools/testing/kunit/kunit.py run "$@" )
 19         }                                          19         }
 20                                                    20 
 21 .. note::                                          21 .. note::
 22         Early versions of ``kunit.py`` (before     22         Early versions of ``kunit.py`` (before 5.6) didn't work unless run from
 23         the kernel root, hence the use of a su     23         the kernel root, hence the use of a subshell and ``cd``.
 24                                                    24 
 25 Running a subset of tests                          25 Running a subset of tests
 26 -------------------------                          26 -------------------------
 27                                                    27 
 28 ``kunit.py run`` accepts an optional glob argu     28 ``kunit.py run`` accepts an optional glob argument to filter tests. The format
 29 is ``"<suite_glob>[.test_glob]"``.                 29 is ``"<suite_glob>[.test_glob]"``.
 30                                                    30 
 31 Say that we wanted to run the sysctl tests, we     31 Say that we wanted to run the sysctl tests, we could do so via:
 32                                                    32 
 33 .. code-block:: bash                               33 .. code-block:: bash
 34                                                    34 
 35         $ echo -e 'CONFIG_KUNIT=y\nCONFIG_KUNI     35         $ echo -e 'CONFIG_KUNIT=y\nCONFIG_KUNIT_ALL_TESTS=y' > .kunit/.kunitconfig
 36         $ ./tools/testing/kunit/kunit.py run '     36         $ ./tools/testing/kunit/kunit.py run 'sysctl*'
 37                                                    37 
 38 We can filter down to just the "write" tests v     38 We can filter down to just the "write" tests via:
 39                                                    39 
 40 .. code-block:: bash                               40 .. code-block:: bash
 41                                                    41 
 42         $ echo -e 'CONFIG_KUNIT=y\nCONFIG_KUNI     42         $ echo -e 'CONFIG_KUNIT=y\nCONFIG_KUNIT_ALL_TESTS=y' > .kunit/.kunitconfig
 43         $ ./tools/testing/kunit/kunit.py run '     43         $ ./tools/testing/kunit/kunit.py run 'sysctl*.*write*'
 44                                                    44 
 45 We're paying the cost of building more tests t     45 We're paying the cost of building more tests than we need this way, but it's
 46 easier than fiddling with ``.kunitconfig`` fil     46 easier than fiddling with ``.kunitconfig`` files or commenting out
 47 ``kunit_suite``'s.                                 47 ``kunit_suite``'s.
 48                                                    48 
 49 However, if we wanted to define a set of tests     49 However, if we wanted to define a set of tests in a less ad hoc way, the next
 50 tip is useful.                                     50 tip is useful.
 51                                                    51 
 52 Defining a set of tests                            52 Defining a set of tests
 53 -----------------------                            53 -----------------------
 54                                                    54 
 55 ``kunit.py run`` (along with ``build``, and ``     55 ``kunit.py run`` (along with ``build``, and ``config``) supports a
 56 ``--kunitconfig`` flag. So if you have a set o     56 ``--kunitconfig`` flag. So if you have a set of tests that you want to run on a
 57 regular basis (especially if they have other d     57 regular basis (especially if they have other dependencies), you can create a
 58 specific ``.kunitconfig`` for them.                58 specific ``.kunitconfig`` for them.
 59                                                    59 
 60 E.g. kunit has one for its tests:                  60 E.g. kunit has one for its tests:
 61                                                    61 
 62 .. code-block:: bash                               62 .. code-block:: bash
 63                                                    63 
 64         $ ./tools/testing/kunit/kunit.py run -     64         $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit/.kunitconfig
 65                                                    65 
 66 Alternatively, if you're following the convent     66 Alternatively, if you're following the convention of naming your
 67 file ``.kunitconfig``, you can just pass in th     67 file ``.kunitconfig``, you can just pass in the dir, e.g.
 68                                                    68 
 69 .. code-block:: bash                               69 .. code-block:: bash
 70                                                    70 
 71         $ ./tools/testing/kunit/kunit.py run -     71         $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit
 72                                                    72 
 73 .. note::                                          73 .. note::
 74         This is a relatively new feature (5.12     74         This is a relatively new feature (5.12+) so we don't have any
 75         conventions yet about on what files sh     75         conventions yet about on what files should be checked in versus just
 76         kept around locally. It's up to you an     76         kept around locally. It's up to you and your maintainer to decide if a
 77         config is useful enough to submit (and     77         config is useful enough to submit (and therefore have to maintain).
 78                                                    78 
 79 .. note::                                          79 .. note::
 80         Having ``.kunitconfig`` fragments in a     80         Having ``.kunitconfig`` fragments in a parent and child directory is
 81         iffy. There's discussion about adding      81         iffy. There's discussion about adding an "import" statement in these
 82         files to make it possible to have a to     82         files to make it possible to have a top-level config run tests from all
 83         child directories. But that would mean     83         child directories. But that would mean ``.kunitconfig`` files are no
 84         longer just simple .config fragments.      84         longer just simple .config fragments.
 85                                                    85 
 86         One alternative would be to have kunit     86         One alternative would be to have kunit tool recursively combine configs
 87         automagically, but tests could theoret     87         automagically, but tests could theoretically depend on incompatible
 88         options, so handling that would be tri     88         options, so handling that would be tricky.
 89                                                    89 
 90 Setting kernel commandline parameters              90 Setting kernel commandline parameters
 91 -------------------------------------              91 -------------------------------------
 92                                                    92 
 93 You can use ``--kernel_args`` to pass arbitrar     93 You can use ``--kernel_args`` to pass arbitrary kernel arguments, e.g.
 94                                                    94 
 95 .. code-block:: bash                               95 .. code-block:: bash
 96                                                    96 
 97         $ ./tools/testing/kunit/kunit.py run -     97         $ ./tools/testing/kunit/kunit.py run --kernel_args=param=42 --kernel_args=param2=false
 98                                                    98 
 99                                                    99 
100 Generating code coverage reports under UML        100 Generating code coverage reports under UML
101 ------------------------------------------        101 ------------------------------------------
102                                                   102 
103 .. note::                                         103 .. note::
104         TODO(brendanhiggins@google.com): There    104         TODO(brendanhiggins@google.com): There are various issues with UML and
105         versions of gcc 7 and up. You're likel    105         versions of gcc 7 and up. You're likely to run into missing ``.gcda``
106         files or compile errors.                  106         files or compile errors.
107                                                   107 
108 This is different from the "normal" way of get    108 This is different from the "normal" way of getting coverage information that is
109 documented in Documentation/dev-tools/gcov.rst    109 documented in Documentation/dev-tools/gcov.rst.
110                                                   110 
111 Instead of enabling ``CONFIG_GCOV_KERNEL=y``,     111 Instead of enabling ``CONFIG_GCOV_KERNEL=y``, we can set these options:
112                                                   112 
113 .. code-block:: none                              113 .. code-block:: none
114                                                   114 
115         CONFIG_DEBUG_KERNEL=y                     115         CONFIG_DEBUG_KERNEL=y
116         CONFIG_DEBUG_INFO=y                       116         CONFIG_DEBUG_INFO=y
117         CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFA    117         CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
118         CONFIG_GCOV=y                             118         CONFIG_GCOV=y
119                                                   119 
120                                                   120 
121 Putting it together into a copy-pastable seque    121 Putting it together into a copy-pastable sequence of commands:
122                                                   122 
123 .. code-block:: bash                              123 .. code-block:: bash
124                                                   124 
125         # Append coverage options to the curre    125         # Append coverage options to the current config
126         $ ./tools/testing/kunit/kunit.py run -    126         $ ./tools/testing/kunit/kunit.py run --kunitconfig=.kunit/ --kunitconfig=tools/testing/kunit/configs/coverage_uml.config
127         # Extract the coverage information fro    127         # Extract the coverage information from the build dir (.kunit/)
128         $ lcov -t "my_kunit_tests" -o coverage    128         $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/
129                                                   129 
130         # From here on, it's the same process     130         # From here on, it's the same process as with CONFIG_GCOV_KERNEL=y
131         # E.g. can generate an HTML report in     131         # E.g. can generate an HTML report in a tmp dir like so:
132         $ genhtml -o /tmp/coverage_html covera    132         $ genhtml -o /tmp/coverage_html coverage.info
133                                                   133 
134                                                   134 
135 If your installed version of gcc doesn't work,    135 If your installed version of gcc doesn't work, you can tweak the steps:
136                                                   136 
137 .. code-block:: bash                              137 .. code-block:: bash
138                                                   138 
139         $ ./tools/testing/kunit/kunit.py run -    139         $ ./tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6
140         $ lcov -t "my_kunit_tests" -o coverage    140         $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/ --gcov-tool=/usr/bin/gcov-6
141                                                   141 
142 Alternatively, LLVM-based toolchains can also     142 Alternatively, LLVM-based toolchains can also be used:
143                                                   143 
144 .. code-block:: bash                              144 .. code-block:: bash
145                                                   145 
146         # Build with LLVM and append coverage     146         # Build with LLVM and append coverage options to the current config
147         $ ./tools/testing/kunit/kunit.py run -    147         $ ./tools/testing/kunit/kunit.py run --make_options LLVM=1 --kunitconfig=.kunit/ --kunitconfig=tools/testing/kunit/configs/coverage_uml.config
148         $ llvm-profdata merge -sparse default.    148         $ llvm-profdata merge -sparse default.profraw -o default.profdata
149         $ llvm-cov export --format=lcov .kunit    149         $ llvm-cov export --format=lcov .kunit/vmlinux -instr-profile default.profdata > coverage.info
150         # The coverage.info file is in lcov-co    150         # The coverage.info file is in lcov-compatible format and it can be used to e.g. generate HTML report
151         $ genhtml -o /tmp/coverage_html covera    151         $ genhtml -o /tmp/coverage_html coverage.info
152                                                   152 
153                                                   153 
154 Running tests manually                            154 Running tests manually
155 ======================                            155 ======================
156                                                   156 
157 Running tests without using ``kunit.py run`` i    157 Running tests without using ``kunit.py run`` is also an important use case.
158 Currently it's your only option if you want to    158 Currently it's your only option if you want to test on architectures other than
159 UML.                                              159 UML.
160                                                   160 
161 As running the tests under UML is fairly strai    161 As running the tests under UML is fairly straightforward (configure and compile
162 the kernel, run the ``./linux`` binary), this     162 the kernel, run the ``./linux`` binary), this section will focus on testing
163 non-UML architectures.                            163 non-UML architectures.
164                                                   164 
165                                                   165 
166 Running built-in tests                            166 Running built-in tests
167 ----------------------                            167 ----------------------
168                                                   168 
169 When setting tests to ``=y``, the tests will r    169 When setting tests to ``=y``, the tests will run as part of boot and print
170 results to dmesg in TAP format. So you just ne    170 results to dmesg in TAP format. So you just need to add your tests to your
171 ``.config``, build and boot your kernel as nor    171 ``.config``, build and boot your kernel as normal.
172                                                   172 
173 So if we compiled our kernel with:                173 So if we compiled our kernel with:
174                                                   174 
175 .. code-block:: none                              175 .. code-block:: none
176                                                   176 
177         CONFIG_KUNIT=y                            177         CONFIG_KUNIT=y
178         CONFIG_KUNIT_EXAMPLE_TEST=y               178         CONFIG_KUNIT_EXAMPLE_TEST=y
179                                                   179 
180 Then we'd see output like this in dmesg signal    180 Then we'd see output like this in dmesg signaling the test ran and passed:
181                                                   181 
182 .. code-block:: none                              182 .. code-block:: none
183                                                   183 
184         TAP version 14                            184         TAP version 14
185         1..1                                      185         1..1
186             # Subtest: example                    186             # Subtest: example
187             1..1                                  187             1..1
188             # example_simple_test: initializin    188             # example_simple_test: initializing
189             ok 1 - example_simple_test            189             ok 1 - example_simple_test
190         ok 1 - example                            190         ok 1 - example
191                                                   191 
192 Running tests as modules                          192 Running tests as modules
193 ------------------------                          193 ------------------------
194                                                   194 
195 Depending on the tests, you can build them as     195 Depending on the tests, you can build them as loadable modules.
196                                                   196 
197 For example, we'd change the config options fr    197 For example, we'd change the config options from before to
198                                                   198 
199 .. code-block:: none                              199 .. code-block:: none
200                                                   200 
201         CONFIG_KUNIT=y                            201         CONFIG_KUNIT=y
202         CONFIG_KUNIT_EXAMPLE_TEST=m               202         CONFIG_KUNIT_EXAMPLE_TEST=m
203                                                   203 
204 Then after booting into our kernel, we can run    204 Then after booting into our kernel, we can run the test via
205                                                   205 
206 .. code-block:: none                              206 .. code-block:: none
207                                                   207 
208         $ modprobe kunit-example-test             208         $ modprobe kunit-example-test
209                                                   209 
210 This will then cause it to print TAP output to    210 This will then cause it to print TAP output to stdout.
211                                                   211 
212 .. note::                                         212 .. note::
213         The ``modprobe`` will *not* have a non    213         The ``modprobe`` will *not* have a non-zero exit code if any test
214         failed (as of 5.13). But ``kunit.py pa    214         failed (as of 5.13). But ``kunit.py parse`` would, see below.
215                                                   215 
216 .. note::                                         216 .. note::
217         You can set ``CONFIG_KUNIT=m`` as well    217         You can set ``CONFIG_KUNIT=m`` as well, however, some features will not
218         work and thus some tests might break.     218         work and thus some tests might break. Ideally tests would specify they
219         depend on ``KUNIT=y`` in their ``Kconf    219         depend on ``KUNIT=y`` in their ``Kconfig``'s, but this is an edge case
220         most test authors won't think about.      220         most test authors won't think about.
221         As of 5.13, the only difference is tha    221         As of 5.13, the only difference is that ``current->kunit_test`` will
222         not exist.                                222         not exist.
223                                                   223 
224 Pretty-printing results                           224 Pretty-printing results
225 -----------------------                           225 -----------------------
226                                                   226 
227 You can use ``kunit.py parse`` to parse dmesg     227 You can use ``kunit.py parse`` to parse dmesg for test output and print out
228 results in the same familiar format that ``kun    228 results in the same familiar format that ``kunit.py run`` does.
229                                                   229 
230 .. code-block:: bash                              230 .. code-block:: bash
231                                                   231 
232         $ ./tools/testing/kunit/kunit.py parse    232         $ ./tools/testing/kunit/kunit.py parse /var/log/dmesg
233                                                   233 
234                                                   234 
235 Retrieving per suite results                      235 Retrieving per suite results
236 ----------------------------                      236 ----------------------------
237                                                   237 
238 Regardless of how you're running your tests, y    238 Regardless of how you're running your tests, you can enable
239 ``CONFIG_KUNIT_DEBUGFS`` to expose per-suite T    239 ``CONFIG_KUNIT_DEBUGFS`` to expose per-suite TAP-formatted results:
240                                                   240 
241 .. code-block:: none                              241 .. code-block:: none
242                                                   242 
243         CONFIG_KUNIT=y                            243         CONFIG_KUNIT=y
244         CONFIG_KUNIT_EXAMPLE_TEST=m               244         CONFIG_KUNIT_EXAMPLE_TEST=m
245         CONFIG_KUNIT_DEBUGFS=y                    245         CONFIG_KUNIT_DEBUGFS=y
246                                                   246 
247 The results for each suite will be exposed und    247 The results for each suite will be exposed under
248 ``/sys/kernel/debug/kunit/<suite>/results``.      248 ``/sys/kernel/debug/kunit/<suite>/results``.
249 So using our example config:                      249 So using our example config:
250                                                   250 
251 .. code-block:: bash                              251 .. code-block:: bash
252                                                   252 
253         $ modprobe kunit-example-test > /dev/n    253         $ modprobe kunit-example-test > /dev/null
254         $ cat /sys/kernel/debug/kunit/example/    254         $ cat /sys/kernel/debug/kunit/example/results
255         ... <TAP output> ...                      255         ... <TAP output> ...
256                                                   256 
257         # After removing the module, the corre    257         # After removing the module, the corresponding files will go away
258         $ modprobe -r kunit-example-test          258         $ modprobe -r kunit-example-test
259         $ cat /sys/kernel/debug/kunit/example/    259         $ cat /sys/kernel/debug/kunit/example/results
260         /sys/kernel/debug/kunit/example/result    260         /sys/kernel/debug/kunit/example/results: No such file or directory
261                                                   261 
262 Generating code coverage reports                  262 Generating code coverage reports
263 --------------------------------                  263 --------------------------------
264                                                   264 
265 See Documentation/dev-tools/gcov.rst for detai    265 See Documentation/dev-tools/gcov.rst for details on how to do this.
266                                                   266 
267 The only vaguely KUnit-specific advice here is    267 The only vaguely KUnit-specific advice here is that you probably want to build
268 your tests as modules. That way you can isolat    268 your tests as modules. That way you can isolate the coverage from tests from
269 other code executed during boot, e.g.             269 other code executed during boot, e.g.
270                                                   270 
271 .. code-block:: bash                              271 .. code-block:: bash
272                                                   272 
273         # Reset coverage counters before runni    273         # Reset coverage counters before running the test.
274         $ echo 0 > /sys/kernel/debug/gcov/rese    274         $ echo 0 > /sys/kernel/debug/gcov/reset
275         $ modprobe kunit-example-test             275         $ modprobe kunit-example-test
276                                                   276 
277                                                   277 
278 Test Attributes and Filtering                     278 Test Attributes and Filtering
279 =============================                     279 =============================
280                                                   280 
281 Test suites and cases can be marked with test     281 Test suites and cases can be marked with test attributes, such as speed of
282 test. These attributes will later be printed i    282 test. These attributes will later be printed in test output and can be used to
283 filter test execution.                            283 filter test execution.
284                                                   284 
285 Marking Test Attributes                           285 Marking Test Attributes
286 -----------------------                           286 -----------------------
287                                                   287 
288 Tests are marked with an attribute by includin    288 Tests are marked with an attribute by including a ``kunit_attributes`` object
289 in the test definition.                           289 in the test definition.
290                                                   290 
291 Test cases can be marked using the ``KUNIT_CAS    291 Test cases can be marked using the ``KUNIT_CASE_ATTR(test_name, attributes)``
292 macro to define the test case instead of ``KUN    292 macro to define the test case instead of ``KUNIT_CASE(test_name)``.
293                                                   293 
294 .. code-block:: c                                 294 .. code-block:: c
295                                                   295 
296         static const struct kunit_attributes e    296         static const struct kunit_attributes example_attr = {
297                 .speed = KUNIT_VERY_SLOW,         297                 .speed = KUNIT_VERY_SLOW,
298         };                                        298         };
299                                                   299 
300         static struct kunit_case example_test_    300         static struct kunit_case example_test_cases[] = {
301                 KUNIT_CASE_ATTR(example_test,     301                 KUNIT_CASE_ATTR(example_test, example_attr),
302         };                                        302         };
303                                                   303 
304 .. note::                                         304 .. note::
305         To mark a test case as slow, you can a    305         To mark a test case as slow, you can also use ``KUNIT_CASE_SLOW(test_name)``.
306         This is a helpful macro as the slow at    306         This is a helpful macro as the slow attribute is the most commonly used.
307                                                   307 
308 Test suites can be marked with an attribute by    308 Test suites can be marked with an attribute by setting the "attr" field in the
309 suite definition.                                 309 suite definition.
310                                                   310 
311 .. code-block:: c                                 311 .. code-block:: c
312                                                   312 
313         static const struct kunit_attributes e    313         static const struct kunit_attributes example_attr = {
314                 .speed = KUNIT_VERY_SLOW,         314                 .speed = KUNIT_VERY_SLOW,
315         };                                        315         };
316                                                   316 
317         static struct kunit_suite example_test    317         static struct kunit_suite example_test_suite = {
318                 ...,                              318                 ...,
319                 .attr = example_attr,             319                 .attr = example_attr,
320         };                                        320         };
321                                                   321 
322 .. note::                                         322 .. note::
323         Not all attributes need to be set in a    323         Not all attributes need to be set in a ``kunit_attributes`` object. Unset
324         attributes will remain uninitialized a    324         attributes will remain uninitialized and act as though the attribute is set
325         to 0 or NULL. Thus, if an attribute is    325         to 0 or NULL. Thus, if an attribute is set to 0, it is treated as unset.
326         These unset attributes will not be rep    326         These unset attributes will not be reported and may act as a default value
327         for filtering purposes.                   327         for filtering purposes.
328                                                   328 
329 Reporting Attributes                              329 Reporting Attributes
330 --------------------                              330 --------------------
331                                                   331 
332 When a user runs tests, attributes will be pre    332 When a user runs tests, attributes will be present in the raw kernel output (in
333 KTAP format). Note that attributes will be hid    333 KTAP format). Note that attributes will be hidden by default in kunit.py output
334 for all passing tests but the raw kernel outpu    334 for all passing tests but the raw kernel output can be accessed using the
335 ``--raw_output`` flag. This is an example of h    335 ``--raw_output`` flag. This is an example of how test attributes for test cases
336 will be formatted in kernel output:               336 will be formatted in kernel output:
337                                                   337 
338 .. code-block:: none                              338 .. code-block:: none
339                                                   339 
340         # example_test.speed: slow                340         # example_test.speed: slow
341         ok 1 example_test                         341         ok 1 example_test
342                                                   342 
343 This is an example of how test attributes for     343 This is an example of how test attributes for test suites will be formatted in
344 kernel output:                                    344 kernel output:
345                                                   345 
346 .. code-block:: none                              346 .. code-block:: none
347                                                   347 
348           KTAP version 2                          348           KTAP version 2
349           # Subtest: example_suite                349           # Subtest: example_suite
350           # module: kunit_example_test            350           # module: kunit_example_test
351           1..3                                    351           1..3
352           ...                                     352           ...
353         ok 1 example_suite                        353         ok 1 example_suite
354                                                   354 
355 Additionally, users can output a full attribut    355 Additionally, users can output a full attribute report of tests with their
356 attributes, using the command line flag ``--li    356 attributes, using the command line flag ``--list_tests_attr``:
357                                                   357 
358 .. code-block:: bash                              358 .. code-block:: bash
359                                                   359 
360         kunit.py run "example" --list_tests_at    360         kunit.py run "example" --list_tests_attr
361                                                   361 
362 .. note::                                         362 .. note::
363         This report can be accessed when runni    363         This report can be accessed when running KUnit manually by passing in the
364         module_param ``kunit.action=list_attr`    364         module_param ``kunit.action=list_attr``.
365                                                   365 
366 Filtering                                         366 Filtering
367 ---------                                         367 ---------
368                                                   368 
369 Users can filter tests using the ``--filter``     369 Users can filter tests using the ``--filter`` command line flag when running
370 tests. As an example:                             370 tests. As an example:
371                                                   371 
372 .. code-block:: bash                              372 .. code-block:: bash
373                                                   373 
374         kunit.py run --filter speed=slow          374         kunit.py run --filter speed=slow
375                                                   375 
376                                                   376 
377 You can also use the following operations on f    377 You can also use the following operations on filters: "<", ">", "<=", ">=",
378 "!=", and "=". Example:                           378 "!=", and "=". Example:
379                                                   379 
380 .. code-block:: bash                              380 .. code-block:: bash
381                                                   381 
382         kunit.py run --filter "speed>slow"        382         kunit.py run --filter "speed>slow"
383                                                   383 
384 This example will run all tests with speeds fa    384 This example will run all tests with speeds faster than slow. Note that the
385 characters < and > are often interpreted by th    385 characters < and > are often interpreted by the shell, so they may need to be
386 quoted or escaped, as above.                      386 quoted or escaped, as above.
387                                                   387 
388 Additionally, you can use multiple filters at     388 Additionally, you can use multiple filters at once. Simply separate filters
389 using commas. Example:                            389 using commas. Example:
390                                                   390 
391 .. code-block:: bash                              391 .. code-block:: bash
392                                                   392 
393         kunit.py run --filter "speed>slow, mod    393         kunit.py run --filter "speed>slow, module=kunit_example_test"
394                                                   394 
395 .. note::                                         395 .. note::
396         You can use this filtering feature whe    396         You can use this filtering feature when running KUnit manually by passing
397         the filter as a module param: ``kunit.    397         the filter as a module param: ``kunit.filter="speed>slow, speed<=normal"``.
398                                                   398 
399 Filtered tests will not run or show up in the     399 Filtered tests will not run or show up in the test output. You can use the
400 ``--filter_action=skip`` flag to skip filtered    400 ``--filter_action=skip`` flag to skip filtered tests instead. These tests will be
401 shown in the test output in the test but will     401 shown in the test output in the test but will not run. To use this feature when
402 running KUnit manually, use the module param `    402 running KUnit manually, use the module param ``kunit.filter_action=skip``.
403                                                   403 
404 Rules of Filtering Procedure                      404 Rules of Filtering Procedure
405 ----------------------------                      405 ----------------------------
406                                                   406 
407 Since both suites and test cases can have attr    407 Since both suites and test cases can have attributes, there may be conflicts
408 between attributes during filtering. The proce    408 between attributes during filtering. The process of filtering follows these
409 rules:                                            409 rules:
410                                                   410 
411 - Filtering always operates at a per-test leve    411 - Filtering always operates at a per-test level.
412                                                   412 
413 - If a test has an attribute set, then the tes    413 - If a test has an attribute set, then the test's value is filtered on.
414                                                   414 
415 - Otherwise, the value falls back to the suite    415 - Otherwise, the value falls back to the suite's value.
416                                                   416 
417 - If neither are set, the attribute has a glob    417 - If neither are set, the attribute has a global "default" value, which is used.
418                                                   418 
419 List of Current Attributes                        419 List of Current Attributes
420 --------------------------                        420 --------------------------
421                                                   421 
422 ``speed``                                         422 ``speed``
423                                                   423 
424 This attribute indicates the speed of a test's    424 This attribute indicates the speed of a test's execution (how slow or fast the
425 test is).                                         425 test is).
426                                                   426 
427 This attribute is saved as an enum with the fo    427 This attribute is saved as an enum with the following categories: "normal",
428 "slow", or "very_slow". The assumed default sp    428 "slow", or "very_slow". The assumed default speed for tests is "normal". This
429 indicates that the test takes a relatively tri    429 indicates that the test takes a relatively trivial amount of time (less than
430 1 second), regardless of the machine it is run    430 1 second), regardless of the machine it is running on. Any test slower than
431 this could be marked as "slow" or "very_slow".    431 this could be marked as "slow" or "very_slow".
432                                                   432 
433 The macro ``KUNIT_CASE_SLOW(test_name)`` can b    433 The macro ``KUNIT_CASE_SLOW(test_name)`` can be easily used to set the speed
434 of a test case to "slow".                         434 of a test case to "slow".
435                                                   435 
436 ``module``                                        436 ``module``
437                                                   437 
438 This attribute indicates the name of the modul    438 This attribute indicates the name of the module associated with the test.
439                                                   439 
440 This attribute is automatically saved as a str    440 This attribute is automatically saved as a string and is printed for each suite.
441 Tests can also be filtered using this attribut    441 Tests can also be filtered using this attribute.
442                                                   442 
443 ``is_init``                                       443 ``is_init``
444                                                   444 
445 This attribute indicates whether the test uses    445 This attribute indicates whether the test uses init data or functions.
446                                                   446 
447 This attribute is automatically saved as a boo    447 This attribute is automatically saved as a boolean and tests can also be
448 filtered using this attribute.                    448 filtered using this attribute.
                                                      

~ [ source navigation ] ~ [ diff markup ] ~ [ identifier search ] ~

kernel.org | git.kernel.org | LWN.net | Project Home | SVN repository | Mail admin

Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.

sflogo.php