1 .. SPDX-License-Identifier: GPL-2.0 1 .. SPDX-License-Identifier: GPL-2.0
2 2
3 ========================== 3 ==========================
4 Frequently Asked Questions 4 Frequently Asked Questions
5 ========================== 5 ==========================
6 6
7 How is this different from Autotest, kselftest 7 How is this different from Autotest, kselftest, and so on?
8 ============================================== 8 ==========================================================
9 KUnit is a unit testing framework. Autotest, k 9 KUnit is a unit testing framework. Autotest, kselftest (and some others) are
10 not. 10 not.
11 11
12 A `unit test <https://martinfowler.com/bliki/U 12 A `unit test <https://martinfowler.com/bliki/UnitTest.html>`_ is supposed to
13 test a single unit of code in isolation and he 13 test a single unit of code in isolation and hence the name *unit test*. A unit
14 test should be the finest granularity of testi 14 test should be the finest granularity of testing and should allow all possible
15 code paths to be tested in the code under test 15 code paths to be tested in the code under test. This is only possible if the
16 code under test is small and does not have any 16 code under test is small and does not have any external dependencies outside of
17 the test's control like hardware. 17 the test's control like hardware.
18 18
19 There are no testing frameworks currently avai 19 There are no testing frameworks currently available for the kernel that do not
20 require installing the kernel on a test machin 20 require installing the kernel on a test machine or in a virtual machine. All
21 testing frameworks require tests to be written 21 testing frameworks require tests to be written in userspace and run on the
22 kernel under test. This is true for Autotest, 22 kernel under test. This is true for Autotest, kselftest, and some others,
23 disqualifying any of them from being considere 23 disqualifying any of them from being considered unit testing frameworks.
24 24
25 Does KUnit support running on architectures ot 25 Does KUnit support running on architectures other than UML?
26 ============================================== 26 ===========================================================
27 27
28 Yes, mostly. 28 Yes, mostly.
29 29
30 For the most part, the KUnit core framework (w 30 For the most part, the KUnit core framework (what we use to write the tests)
31 can compile to any architecture. It compiles l 31 can compile to any architecture. It compiles like just another part of the
32 kernel and runs when the kernel boots, or when 32 kernel and runs when the kernel boots, or when built as a module, when the
33 module is loaded. However, there is infrastru 33 module is loaded. However, there is infrastructure, like the KUnit Wrapper
34 (``tools/testing/kunit/kunit.py``) that might 34 (``tools/testing/kunit/kunit.py``) that might not support some architectures
35 (see :ref:`kunit-on-qemu`). 35 (see :ref:`kunit-on-qemu`).
36 36
37 In short, yes, you can run KUnit on other arch 37 In short, yes, you can run KUnit on other architectures, but it might require
38 more work than using KUnit on UML. 38 more work than using KUnit on UML.
39 39
40 For more information, see :ref:`kunit-on-non-u 40 For more information, see :ref:`kunit-on-non-uml`.
41 41
42 .. _kinds-of-tests: 42 .. _kinds-of-tests:
43 43
44 What is the difference between a unit test and 44 What is the difference between a unit test and other kinds of tests?
45 ============================================== 45 ====================================================================
46 Most existing tests for the Linux kernel would 46 Most existing tests for the Linux kernel would be categorized as an integration
47 test, or an end-to-end test. 47 test, or an end-to-end test.
48 48
49 - A unit test is supposed to test a single uni 49 - A unit test is supposed to test a single unit of code in isolation. A unit
50 test should be the finest granularity of tes 50 test should be the finest granularity of testing and, as such, allows all
51 possible code paths to be tested in the code 51 possible code paths to be tested in the code under test. This is only possible
52 if the code under test is small and does not 52 if the code under test is small and does not have any external dependencies
53 outside of the test's control like hardware. 53 outside of the test's control like hardware.
54 - An integration test tests the interaction be 54 - An integration test tests the interaction between a minimal set of components,
55 usually just two or three. For example, some 55 usually just two or three. For example, someone might write an integration
56 test to test the interaction between a drive 56 test to test the interaction between a driver and a piece of hardware, or to
57 test the interaction between the userspace l 57 test the interaction between the userspace libraries the kernel provides and
58 the kernel itself. However, one of these tes 58 the kernel itself. However, one of these tests would probably not test the
59 entire kernel along with hardware interactio 59 entire kernel along with hardware interactions and interactions with the
60 userspace. 60 userspace.
61 - An end-to-end test usually tests the entire 61 - An end-to-end test usually tests the entire system from the perspective of the
62 code under test. For example, someone might 62 code under test. For example, someone might write an end-to-end test for the
63 kernel by installing a production configurat 63 kernel by installing a production configuration of the kernel on production
64 hardware with a production userspace and the 64 hardware with a production userspace and then trying to exercise some behavior
65 that depends on interactions between the har 65 that depends on interactions between the hardware, the kernel, and userspace.
66 66
67 KUnit is not working, what should I do? 67 KUnit is not working, what should I do?
68 ======================================= 68 =======================================
69 69
70 Unfortunately, there are a number of things wh 70 Unfortunately, there are a number of things which can break, but here are some
71 things to try. 71 things to try.
72 72
73 1. Run ``./tools/testing/kunit/kunit.py run`` 73 1. Run ``./tools/testing/kunit/kunit.py run`` with the ``--raw_output``
74 parameter. This might show details or error 74 parameter. This might show details or error messages hidden by the kunit_tool
75 parser. 75 parser.
76 2. Instead of running ``kunit.py run``, try ru 76 2. Instead of running ``kunit.py run``, try running ``kunit.py config``,
77 ``kunit.py build``, and ``kunit.py exec`` i 77 ``kunit.py build``, and ``kunit.py exec`` independently. This can help track
78 down where an issue is occurring. (If you t 78 down where an issue is occurring. (If you think the parser is at fault, you
79 can run it manually against ``stdin`` or a 79 can run it manually against ``stdin`` or a file with ``kunit.py parse``.)
80 3. Running the UML kernel directly can often r 80 3. Running the UML kernel directly can often reveal issues or error messages,
81 ``kunit_tool`` ignores. This should be as s 81 ``kunit_tool`` ignores. This should be as simple as running ``./vmlinux``
82 after building the UML kernel (for example, 82 after building the UML kernel (for example, by using ``kunit.py build``).
83 Note that UML has some unusual requirements 83 Note that UML has some unusual requirements (such as the host having a tmpfs
84 filesystem mounted), and has had issues in 84 filesystem mounted), and has had issues in the past when built statically and
85 the host has KASLR enabled. (On older host 85 the host has KASLR enabled. (On older host kernels, you may need to run
86 ``setarch `uname -m` -R ./vmlinux`` to disa 86 ``setarch `uname -m` -R ./vmlinux`` to disable KASLR.)
87 4. Make sure the kernel .config has ``CONFIG_K 87 4. Make sure the kernel .config has ``CONFIG_KUNIT=y`` and at least one test
88 (e.g. ``CONFIG_KUNIT_EXAMPLE_TEST=y``). kun 88 (e.g. ``CONFIG_KUNIT_EXAMPLE_TEST=y``). kunit_tool will keep its .config
89 around, so you can see what config was used 89 around, so you can see what config was used after running ``kunit.py run``.
90 It also preserves any config changes you mi 90 It also preserves any config changes you might make, so you can
91 enable/disable things with ``make ARCH=um m 91 enable/disable things with ``make ARCH=um menuconfig`` or similar, and then
92 re-run kunit_tool. 92 re-run kunit_tool.
93 5. Try to run ``make ARCH=um defconfig`` befor 93 5. Try to run ``make ARCH=um defconfig`` before running ``kunit.py run``. This
94 may help clean up any residual config items 94 may help clean up any residual config items which could be causing problems.
95 6. Finally, try running KUnit outside UML. KUn 95 6. Finally, try running KUnit outside UML. KUnit and KUnit tests can be
96 built into any kernel, or can be built as a 96 built into any kernel, or can be built as a module and loaded at runtime.
97 Doing so should allow you to determine if U 97 Doing so should allow you to determine if UML is causing the issue you're
98 seeing. When tests are built-in, they will 98 seeing. When tests are built-in, they will execute when the kernel boots, and
99 modules will automatically execute associat 99 modules will automatically execute associated tests when loaded. Test results
100 can be collected from ``/sys/kernel/debug/k 100 can be collected from ``/sys/kernel/debug/kunit/<test suite>/results``, and
101 can be parsed with ``kunit.py parse``. For 101 can be parsed with ``kunit.py parse``. For more details, see :ref:`kunit-on-qemu`.
102 102
103 If none of the above tricks help, you are alwa 103 If none of the above tricks help, you are always welcome to email any issues to
104 kunit-dev@googlegroups.com. 104 kunit-dev@googlegroups.com.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.