1 tdc - Linux Traffic Control (tc) unit testing 1 tdc - Linux Traffic Control (tc) unit testing suite
2 2
3 Author: Lucas Bates - lucasb@mojatatu.com 3 Author: Lucas Bates - lucasb@mojatatu.com
4 4
5 tdc is a Python script to load tc unit tests f 5 tdc is a Python script to load tc unit tests from a separate JSON file and
6 execute them inside a network namespace dedica 6 execute them inside a network namespace dedicated to the task.
7 7
8 8
9 REQUIREMENTS 9 REQUIREMENTS
10 ------------ 10 ------------
11 11
12 * Minimum Python version of 3.8. !! 12 * Minimum Python version of 3.4. Earlier 3.X versions may work but are not
>> 13 guaranteed.
13 14
14 * The kernel must have network namespace supp !! 15 * The kernel must have network namespace support
15 16
16 * The kernel must have veth support available !! 17 * The kernel must have veth support available, as a veth pair is created
17 prior to running the tests when using nsPlu !! 18 prior to running the tests.
18 19
19 * The kernel must have the appropriate infras !! 20 * All tc-related features must be built in or available as modules.
20 unit tests. See the config file in this dir !! 21 To check what is required in current setup run:
21 features. As new tests will be added, confi <<
22 <<
23 * All tc-related features being tested must b <<
24 modules. To check what is required in curr <<
25 ./tdc.py -c 22 ./tdc.py -c
26 23
27 Note: 24 Note:
28 In the current release, tdc run will abort 25 In the current release, tdc run will abort due to a failure in setup or
29 teardown commands - which includes not bein 26 teardown commands - which includes not being able to run a test simply
30 because the kernel did not support a specif 27 because the kernel did not support a specific feature. (This will be
31 handled in a future version - the current w 28 handled in a future version - the current workaround is to run the tests
32 on specific test categories that your kerne 29 on specific test categories that your kernel supports)
33 30
34 31
35 BEFORE YOU RUN 32 BEFORE YOU RUN
36 -------------- 33 --------------
37 34
38 The path to the tc executable that will be mos 35 The path to the tc executable that will be most commonly tested can be defined
39 in the tdc_config.py file. Find the 'TC' entry 36 in the tdc_config.py file. Find the 'TC' entry in the NAMES dictionary and
40 define the path. 37 define the path.
41 38
42 If you need to test a different tc executable 39 If you need to test a different tc executable on the fly, you can do so by
43 using the -p option when running tdc: 40 using the -p option when running tdc:
44 ./tdc.py -p /path/to/tc 41 ./tdc.py -p /path/to/tc
45 42
46 43
47 RUNNING TDC 44 RUNNING TDC
48 ----------- 45 -----------
49 46
50 To use tdc, root privileges are required. Thi !! 47 To use tdc, root privileges are required. tdc will not run otherwise.
51 commands being tested must be run as root. Th !! 48
52 execution by root uid has been moved into a pl !! 49 All tests are executed inside a network namespace to prevent conflicts
53 ARCHITECTURE, below). !! 50 within the host.
54 <<
55 Tests that use a network device should have ns <<
56 requirement for that test. nsPlugin executes a <<
57 network namespace and creates a veth pair whic <<
58 cases. To disable execution within the namespa <<
59 to tdc when starting a test run; the veth pair <<
60 by the plugin. <<
61 51
62 Running tdc without any arguments will run all 52 Running tdc without any arguments will run all tests. Refer to the section
63 on command line arguments for more information 53 on command line arguments for more information, or run:
64 ./tdc.py -h 54 ./tdc.py -h
65 55
66 tdc will list the test names as they are being 56 tdc will list the test names as they are being run, and print a summary in
67 TAP (Test Anything Protocol) format when they 57 TAP (Test Anything Protocol) format when they are done. If tests fail,
68 output captured from the failing test will be 58 output captured from the failing test will be printed immediately following
69 the failed test in the TAP output. 59 the failed test in the TAP output.
70 60
71 61
72 OVERVIEW OF TDC EXECUTION <<
73 ------------------------- <<
74 <<
75 One run of tests is considered a "test suite" <<
76 future). A test suite has one or more test ca <<
77 <<
78 A test case has four stages: <<
79 <<
80 - setup <<
81 - execute <<
82 - verify <<
83 - teardown <<
84 <<
85 The setup and teardown stages can run zero or <<
86 stage does some setup if the test needs it. T <<
87 the setup and returns the system to a "neutral <<
88 can be run next. These two stages require any <<
89 success, but do not otherwise verify the resul <<
90 <<
91 The execute and verify stages each run one com <<
92 tests the return code against one or more acce <<
93 verify stage checks the return code for succes <<
94 the stdout with a regular expression. <<
95 <<
96 Each of the commands in any stage will run in <<
97 <<
98 Each test is an atomic unit. A test that for w <<
99 definitions is a bug. <<
100 <<
101 A test that runs inside a namespace (requires <<
102 with other tests. <<
103 <<
104 Tests that use netdevsim or don't run inside a <<
105 to each other. <<
106 <<
107 <<
108 USER-DEFINED CONSTANTS 62 USER-DEFINED CONSTANTS
109 ---------------------- 63 ----------------------
110 64
111 The tdc_config.py file contains multiple value 65 The tdc_config.py file contains multiple values that can be altered to suit
112 your needs. Any value in the NAMES dictionary 66 your needs. Any value in the NAMES dictionary can be altered without affecting
113 the tests to be run. These values are used in 67 the tests to be run. These values are used in the tc commands that will be
114 executed as part of the test. More will be add 68 executed as part of the test. More will be added as test cases require.
115 69
116 Example: 70 Example:
117 $TC qdisc add dev $DEV1 ingress 71 $TC qdisc add dev $DEV1 ingress
118 72
119 The NAMES values are used to substitute into t <<
120 <<
121 73
122 COMMAND LINE ARGUMENTS 74 COMMAND LINE ARGUMENTS
123 ---------------------- 75 ----------------------
124 76
125 Run tdc.py -h to see the full list of availabl 77 Run tdc.py -h to see the full list of available arguments.
126 78
127 PLUGIN ARCHITECTURE !! 79 -p PATH Specify the tc executable located at PATH to be used on this
128 ------------------- !! 80 test run
129 !! 81 -c Show the available test case categories in this test file
130 There is now a plugin architecture, and some o !! 82 -c CATEGORY Run only tests that belong to CATEGORY
131 was in the tdc.py script has been moved into t !! 83 -f FILE Read test cases from the JSON file named FILE
132 !! 84 -l [CATEGORY] List all test cases in the JSON file. If CATEGORY is
133 The plugins are in the directory plugin-lib. !! 85 specified, list test cases matching that category.
134 directory plugins. Put symbolic links from pl !! 86 -s ID Show the test case matching ID
135 and name them according to the order you want !! 87 -e ID Execute the test case identified by ID
136 necessary if a test case being run requires a !! 88 -i Generate unique ID numbers for test cases with no existing
137 !! 89 ID number
138 Example: <<
139 <<
140 bjb@bee:~/work/tc-testing$ ls -l plugins <<
141 total 4 <<
142 lrwxrwxrwx 1 bjb bjb 27 Oct 4 16:12 10-r <<
143 lrwxrwxrwx 1 bjb bjb 25 Oct 12 17:55 20-n <<
144 -rwxr-xr-x 1 bjb bjb 0 Sep 29 15:56 __in <<
145 <<
146 The plugins are a subclass of TdcPlugin, defin <<
147 must be called "SubPlugin" so tdc can find the <<
148 distinguished from each other in the python pr <<
149 name. <<
150 <<
151 This base class supplies "hooks" to run extra <<
152 <<
153 pre- and post-suite <<
154 pre- and post-case <<
155 pre- and post-execute stage <<
156 adjust-command (runs in all stages and receive <<
157 <<
158 The pre-suite hook receives the number of test <<
159 This allows you to dump out the list of skippe <<
160 failure during setup or teardown stage. <<
161 <<
162 The pre-case hook receives the ordinal number <<
163 <<
164 The adjust-command hook receives the stage id <<
165 full command to be executed. This allows for <<
166 of the command. <<
167 <<
168 The stages are identified by the following str <<
169 <<
170 - pre (pre-suite) <<
171 - setup <<
172 - command <<
173 - verify <<
174 - teardown <<
175 - post (post-suite) <<
176 <<
177 <<
178 To write a plugin, you need to inherit from Td <<
179 TdcPlugin.py. To use the plugin, you have to <<
180 implementation file in plugin-lib, and add a s <<
181 plugins. It will be detected at run time and <<
182 appropriate times. There are a few examples i <<
183 directory: <<
184 <<
185 - rootPlugin.py: <<
186 implements the enforcement of running as <<
187 - nsPlugin.py: <<
188 sets up a network namespace and runs all <<
189 while also setting up dummy devices to b <<
190 - valgrindPlugin.py <<
191 runs each command in the execute stage u <<
192 and checks for leaks. <<
193 This plugin will output an extra test fo <<
194 one is the existing output as to whether <<
195 and the other is a test whether the comm <<
196 (This one is a preliminary version, it m <<
197 but the overall template is there and it <<
198 90
199 91
200 ACKNOWLEDGEMENTS 92 ACKNOWLEDGEMENTS
201 ---------------- 93 ----------------
202 94
203 Thanks to: 95 Thanks to:
204 96
205 Jamal Hadi Salim, for providing valuable test 97 Jamal Hadi Salim, for providing valuable test cases
206 Keara Leibovitz, who wrote the CLI test driver 98 Keara Leibovitz, who wrote the CLI test driver that I used as a base for the
207 first version of the tc testing suite. This 99 first version of the tc testing suite. This work was presented at
208 Netdev 1.2 Tokyo in October 2016. 100 Netdev 1.2 Tokyo in October 2016.
209 Samir Hussain, for providing help while I dove 101 Samir Hussain, for providing help while I dove into Python for the first time
210 and being a second eye for this code. 102 and being a second eye for this code.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.