1 ================== 1 ==================
2 BPF Selftest Notes 2 BPF Selftest Notes
3 ================== 3 ==================
4 General instructions on running selftests can 4 General instructions on running selftests can be found in
5 `Documentation/bpf/bpf_devel_QA.rst`__. 5 `Documentation/bpf/bpf_devel_QA.rst`__.
6 6
7 __ /Documentation/bpf/bpf_devel_QA.rst#q-how-t 7 __ /Documentation/bpf/bpf_devel_QA.rst#q-how-to-run-bpf-selftests
8 8
9 ============= 9 =============
10 BPF CI System 10 BPF CI System
11 ============= 11 =============
12 12
13 BPF employs a continuous integration (CI) syst 13 BPF employs a continuous integration (CI) system to check patch submission in an
14 automated fashion. The system runs selftests f 14 automated fashion. The system runs selftests for each patch in a series. Results
15 are propagated to patchwork, where failures ar 15 are propagated to patchwork, where failures are highlighted similar to
16 violations of other checks (such as additional 16 violations of other checks (such as additional warnings being emitted or a
17 ``scripts/checkpatch.pl`` reported deficiency) 17 ``scripts/checkpatch.pl`` reported deficiency):
18 18
19 https://patchwork.kernel.org/project/netdevb 19 https://patchwork.kernel.org/project/netdevbpf/list/?delegate=121173
20 20
21 The CI system executes tests on multiple archi 21 The CI system executes tests on multiple architectures. It uses a kernel
22 configuration derived from both the generic an 22 configuration derived from both the generic and architecture specific config
23 file fragments below ``tools/testing/selftests 23 file fragments below ``tools/testing/selftests/bpf/`` (e.g., ``config`` and
24 ``config.x86_64``). 24 ``config.x86_64``).
25 25
26 Denylisting Tests 26 Denylisting Tests
27 ================= 27 =================
28 28
29 It is possible for some architectures to not h 29 It is possible for some architectures to not have support for all BPF features.
30 In such a case tests in CI may fail. An exampl 30 In such a case tests in CI may fail. An example of such a shortcoming is BPF
31 trampoline support on IBM's s390x architecture 31 trampoline support on IBM's s390x architecture. For cases like this, an in-tree
32 deny list file, located at ``tools/testing/sel 32 deny list file, located at ``tools/testing/selftests/bpf/DENYLIST.<arch>``, can
33 be used to prevent the test from running on su 33 be used to prevent the test from running on such an architecture.
34 34
35 In addition to that, the generic ``tools/testi 35 In addition to that, the generic ``tools/testing/selftests/bpf/DENYLIST`` is
36 honored on every architecture running tests. 36 honored on every architecture running tests.
37 37
38 These files are organized in three columns. Th 38 These files are organized in three columns. The first column lists the test in
39 question. This can be the name of a test suite 39 question. This can be the name of a test suite or of an individual test. The
40 remaining two columns provide additional meta 40 remaining two columns provide additional meta data that helps identify and
41 classify the entry: column two is a copy and p 41 classify the entry: column two is a copy and paste of the error being reported
42 when running the test in the setting in questi 42 when running the test in the setting in question. The third column, if
43 available, summarizes the underlying problem. 43 available, summarizes the underlying problem. A value of ``trampoline``, for
44 example, indicates that lack of trampoline sup 44 example, indicates that lack of trampoline support is causing the test to fail.
45 This last entry helps identify tests that can 45 This last entry helps identify tests that can be re-enabled once such support is
46 added. 46 added.
47 47
48 ========================= 48 =========================
49 Running Selftests in a VM 49 Running Selftests in a VM
50 ========================= 50 =========================
51 51
52 It's now possible to run the selftests using ` 52 It's now possible to run the selftests using ``tools/testing/selftests/bpf/vmtest.sh``.
53 The script tries to ensure that the tests are 53 The script tries to ensure that the tests are run with the same environment as they
54 would be run post-submit in the CI used by the 54 would be run post-submit in the CI used by the Maintainers, with the exception
55 that deny lists are not automatically honored. 55 that deny lists are not automatically honored.
56 56
57 This script uses the in-tree kernel configurat 57 This script uses the in-tree kernel configuration and downloads a VM userspace
58 image from the system used by the CI. It build 58 image from the system used by the CI. It builds the kernel (without overwriting
59 your existing Kconfig), recompiles the bpf sel 59 your existing Kconfig), recompiles the bpf selftests, runs them (by default
60 ``tools/testing/selftests/bpf/test_progs``) an 60 ``tools/testing/selftests/bpf/test_progs``) and saves the resulting output (by
61 default in ``~/.bpf_selftests``). 61 default in ``~/.bpf_selftests``).
62 62
63 Script dependencies: 63 Script dependencies:
64 - clang (preferably built from sources, https: 64 - clang (preferably built from sources, https://github.com/llvm/llvm-project);
65 - pahole (preferably built from sources, https 65 - pahole (preferably built from sources, https://git.kernel.org/pub/scm/devel/pahole/pahole.git/);
66 - qemu; 66 - qemu;
67 - docutils (for ``rst2man``); 67 - docutils (for ``rst2man``);
68 - libcap-devel. 68 - libcap-devel.
69 69
70 For more information about using the script, r 70 For more information about using the script, run:
71 71
72 .. code-block:: console 72 .. code-block:: console
73 73
74 $ tools/testing/selftests/bpf/vmtest.sh -h 74 $ tools/testing/selftests/bpf/vmtest.sh -h
75 75
76 In case of linker errors when running selftest 76 In case of linker errors when running selftests, try using static linking:
77 77
78 .. code-block:: console 78 .. code-block:: console
79 79
80 $ LDLIBS=-static PKG_CONFIG='pkg-config --st !! 80 $ LDLIBS=-static vmtest.sh
81 81
82 .. note:: Some distros may not support static 82 .. note:: Some distros may not support static linking.
83 83
84 .. note:: The script uses pahole and clang bas 84 .. note:: The script uses pahole and clang based on host environment setting.
85 If you want to change pahole and llv 85 If you want to change pahole and llvm, you can change `PATH` environment
86 variable in the beginning of script. 86 variable in the beginning of script.
87 87
88 Running vmtest on RV64 !! 88 .. note:: The script currently only supports x86_64 and s390x architectures.
89 ====================== <<
90 To speed up testing and avoid various dependen <<
91 run vmtest in a Docker container. Before runni <<
92 Docker container and local rootfs image. The o <<
93 <<
94 1. Create Docker container as shown in link [0 <<
95 <<
96 2. Use mkrootfs_debian.sh script [1] to build <<
97 <<
98 .. code-block:: console <<
99 <<
100 $ sudo ./mkrootfs_debian.sh --arch riscv64 - <<
101 <<
102 3. Start Docker container [0] and run vmtest i <<
103 <<
104 .. code-block:: console <<
105 <<
106 $ PLATFORM=riscv64 CROSS_COMPILE=riscv64-lin <<
107 tools/testing/selftests/bpf/vmtest.sh \ <<
108 -l <path of local rootfs image> -- \ <<
109 ./test_progs -d \ <<
110 \"$(cat tools/testing/selftests/bpf/DE <<
111 | cut -d'#' -f1 \ <<
112 | sed -e 's/^[[:space:]]*//' \ <<
113 -e 's/[[:space:]]*$//' \ <<
114 | tr -s '\n' ',' \ <<
115 )\" <<
116 <<
117 Link: https://github.com/pulehui/riscv-bpf-vmt <<
118 Link: https://github.com/libbpf/ci/blob/main/r <<
119 89
120 Additional information about selftest failures 90 Additional information about selftest failures are
121 documented here. 91 documented here.
122 92
123 profiler[23] test failures with clang/llvm <12 93 profiler[23] test failures with clang/llvm <12.0.0
124 ============================================== 94 ==================================================
125 95
126 With clang/llvm <12.0.0, the profiler[23] test 96 With clang/llvm <12.0.0, the profiler[23] test may fail.
127 The symptom looks like 97 The symptom looks like
128 98
129 .. code-block:: c 99 .. code-block:: c
130 100
131 // r9 is a pointer to map_value 101 // r9 is a pointer to map_value
132 // r7 is a scalar 102 // r7 is a scalar
133 17: bf 96 00 00 00 00 00 00 r6 = r9 103 17: bf 96 00 00 00 00 00 00 r6 = r9
134 18: 0f 76 00 00 00 00 00 00 r6 += r7 104 18: 0f 76 00 00 00 00 00 00 r6 += r7
135 math between map_value pointer and register 105 math between map_value pointer and register with unbounded min value is not allowed
136 106
137 // the instructions below will not be seen i 107 // the instructions below will not be seen in the verifier log
138 19: a5 07 01 00 01 01 00 00 if r7 < 25 108 19: a5 07 01 00 01 01 00 00 if r7 < 257 goto +1
139 20: bf 96 00 00 00 00 00 00 r6 = r9 109 20: bf 96 00 00 00 00 00 00 r6 = r9
140 // r6 is used here 110 // r6 is used here
141 111
142 The verifier will reject such code with above 112 The verifier will reject such code with above error.
143 At insn 18 the r7 is indeed unbounded. The lat 113 At insn 18 the r7 is indeed unbounded. The later insn 19 checks the bounds and
144 the insn 20 undoes map_value addition. It is c 114 the insn 20 undoes map_value addition. It is currently impossible for the
145 verifier to understand such speculative pointe 115 verifier to understand such speculative pointer arithmetic.
146 Hence `this patch`__ addresses it on the compi 116 Hence `this patch`__ addresses it on the compiler side. It was committed on llvm 12.
147 117
148 __ https://github.com/llvm/llvm-project/commit !! 118 __ https://reviews.llvm.org/D85570
149 119
150 The corresponding C code 120 The corresponding C code
151 121
152 .. code-block:: c 122 .. code-block:: c
153 123
154 for (int i = 0; i < MAX_CGROUPS_PATH_DEPTH; 124 for (int i = 0; i < MAX_CGROUPS_PATH_DEPTH; i++) {
155 filepart_length = bpf_probe_read_str 125 filepart_length = bpf_probe_read_str(payload, ...);
156 if (filepart_length <= MAX_PATH) { 126 if (filepart_length <= MAX_PATH) {
157 barrier_var(filepart_length) 127 barrier_var(filepart_length); // workaround
158 payload += filepart_length; 128 payload += filepart_length;
159 } 129 }
160 } 130 }
161 131
162 bpf_iter test failures with clang/llvm 10.0.0 132 bpf_iter test failures with clang/llvm 10.0.0
163 ============================================= 133 =============================================
164 134
165 With clang/llvm 10.0.0, the following two bpf_ 135 With clang/llvm 10.0.0, the following two bpf_iter tests failed:
166 * ``bpf_iter/ipv6_route`` 136 * ``bpf_iter/ipv6_route``
167 * ``bpf_iter/netlink`` 137 * ``bpf_iter/netlink``
168 138
169 The symptom for ``bpf_iter/ipv6_route`` looks 139 The symptom for ``bpf_iter/ipv6_route`` looks like
170 140
171 .. code-block:: c 141 .. code-block:: c
172 142
173 2: (79) r8 = *(u64 *)(r1 +8) 143 2: (79) r8 = *(u64 *)(r1 +8)
174 ... 144 ...
175 14: (bf) r2 = r8 145 14: (bf) r2 = r8
176 15: (0f) r2 += r1 146 15: (0f) r2 += r1
177 ; BPF_SEQ_PRINTF(seq, "%pi6 %02x ", &rt->fib 147 ; BPF_SEQ_PRINTF(seq, "%pi6 %02x ", &rt->fib6_dst.addr, rt->fib6_dst.plen);
178 16: (7b) *(u64 *)(r8 +64) = r2 148 16: (7b) *(u64 *)(r8 +64) = r2
179 only read is supported 149 only read is supported
180 150
181 The symptom for ``bpf_iter/netlink`` looks lik 151 The symptom for ``bpf_iter/netlink`` looks like
182 152
183 .. code-block:: c 153 .. code-block:: c
184 154
185 ; struct netlink_sock *nlk = ctx->sk; 155 ; struct netlink_sock *nlk = ctx->sk;
186 2: (79) r7 = *(u64 *)(r1 +8) 156 2: (79) r7 = *(u64 *)(r1 +8)
187 ... 157 ...
188 15: (bf) r2 = r7 158 15: (bf) r2 = r7
189 16: (0f) r2 += r1 159 16: (0f) r2 += r1
190 ; BPF_SEQ_PRINTF(seq, "%pK %-3d ", s, s->sk_ 160 ; BPF_SEQ_PRINTF(seq, "%pK %-3d ", s, s->sk_protocol);
191 17: (7b) *(u64 *)(r7 +0) = r2 161 17: (7b) *(u64 *)(r7 +0) = r2
192 only read is supported 162 only read is supported
193 163
194 This is due to a llvm BPF backend bug. `The fi 164 This is due to a llvm BPF backend bug. `The fix`__
195 has been pushed to llvm 10.x release branch an 165 has been pushed to llvm 10.x release branch and will be
196 available in 10.0.1. The patch is available in 166 available in 10.0.1. The patch is available in llvm 11.0.0 trunk.
197 167
198 __ https://github.com/llvm/llvm-project/commi !! 168 __ https://reviews.llvm.org/D78466
199 169
200 bpf_verif_scale/loop6.bpf.o test failure with 170 bpf_verif_scale/loop6.bpf.o test failure with Clang 12
201 ============================================== 171 ======================================================
202 172
203 With Clang 12, the following bpf_verif_scale t 173 With Clang 12, the following bpf_verif_scale test failed:
204 * ``bpf_verif_scale/loop6.bpf.o`` 174 * ``bpf_verif_scale/loop6.bpf.o``
205 175
206 The verifier output looks like 176 The verifier output looks like
207 177
208 .. code-block:: c 178 .. code-block:: c
209 179
210 R1 type=ctx expected=fp 180 R1 type=ctx expected=fp
211 The sequence of 8193 jumps is too complex. 181 The sequence of 8193 jumps is too complex.
212 182
213 The reason is compiler generating the followin 183 The reason is compiler generating the following code
214 184
215 .. code-block:: c 185 .. code-block:: c
216 186
217 ; for (i = 0; (i < VIRTIO_MAX_SGS) && 187 ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) {
218 14: 16 05 40 00 00 00 00 00 if w5 188 14: 16 05 40 00 00 00 00 00 if w5 == 0 goto +64 <LBB0_6>
219 15: bc 51 00 00 00 00 00 00 w1 = w 189 15: bc 51 00 00 00 00 00 00 w1 = w5
220 16: 04 01 00 00 ff ff ff ff w1 += 190 16: 04 01 00 00 ff ff ff ff w1 += -1
221 17: 67 05 00 00 20 00 00 00 r5 <<= 191 17: 67 05 00 00 20 00 00 00 r5 <<= 32
222 18: 77 05 00 00 20 00 00 00 r5 >>= 192 18: 77 05 00 00 20 00 00 00 r5 >>= 32
223 19: a6 01 01 00 05 00 00 00 if w1 193 19: a6 01 01 00 05 00 00 00 if w1 < 5 goto +1 <LBB0_4>
224 20: b7 05 00 00 06 00 00 00 r5 = 6 194 20: b7 05 00 00 06 00 00 00 r5 = 6
225 00000000000000a8 <LBB0_4>: 195 00000000000000a8 <LBB0_4>:
226 21: b7 02 00 00 00 00 00 00 r2 = 0 196 21: b7 02 00 00 00 00 00 00 r2 = 0
227 22: b7 01 00 00 00 00 00 00 r1 = 0 197 22: b7 01 00 00 00 00 00 00 r1 = 0
228 ; for (i = 0; (i < VIRTIO_MAX_SGS) && 198 ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) {
229 23: 7b 1a e0 ff 00 00 00 00 *(u64 199 23: 7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1
230 24: 7b 5a c0 ff 00 00 00 00 *(u64 200 24: 7b 5a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r5
231 201
232 Note that insn #15 has w1 = w5 and w1 is refin 202 Note that insn #15 has w1 = w5 and w1 is refined later but
233 r5(w5) is eventually saved on stack at insn #2 203 r5(w5) is eventually saved on stack at insn #24 for later use.
234 This cause later verifier failure. The bug has 204 This cause later verifier failure. The bug has been `fixed`__ in
235 Clang 13. 205 Clang 13.
236 206
237 __ https://github.com/llvm/llvm-project/commi !! 207 __ https://reviews.llvm.org/D97479
238 208
239 BPF CO-RE-based tests and Clang version 209 BPF CO-RE-based tests and Clang version
240 ======================================= 210 =======================================
241 211
242 A set of selftests use BPF target-specific bui 212 A set of selftests use BPF target-specific built-ins, which might require
243 bleeding-edge Clang versions (Clang 12 nightly 213 bleeding-edge Clang versions (Clang 12 nightly at this time).
244 214
245 Few sub-tests of core_reloc test suit (part of 215 Few sub-tests of core_reloc test suit (part of test_progs test runner) require
246 the following built-ins, listed with correspon 216 the following built-ins, listed with corresponding Clang diffs introducing
247 them to Clang/LLVM. These sub-tests are going 217 them to Clang/LLVM. These sub-tests are going to be skipped if Clang is too
248 old to support them, they shouldn't cause buil 218 old to support them, they shouldn't cause build failures or runtime test
249 failures: 219 failures:
250 220
251 - __builtin_btf_type_id() [0_, 1_, 2_]; 221 - __builtin_btf_type_id() [0_, 1_, 2_];
252 - __builtin_preserve_type_info(), __builtin_pr 222 - __builtin_preserve_type_info(), __builtin_preserve_enum_value() [3_, 4_].
253 223
254 .. _0: https://github.com/llvm/llvm-project/co !! 224 .. _0: https://reviews.llvm.org/D74572
255 .. _1: https://github.com/llvm/llvm-project/co !! 225 .. _1: https://reviews.llvm.org/D74668
256 .. _2: https://github.com/llvm/llvm-project/co !! 226 .. _2: https://reviews.llvm.org/D85174
257 .. _3: https://github.com/llvm/llvm-project/co !! 227 .. _3: https://reviews.llvm.org/D83878
258 .. _4: https://github.com/llvm/llvm-project/co !! 228 .. _4: https://reviews.llvm.org/D83242
259 229
260 Floating-point tests and Clang version 230 Floating-point tests and Clang version
261 ====================================== 231 ======================================
262 232
263 Certain selftests, e.g. core_reloc, require su 233 Certain selftests, e.g. core_reloc, require support for the floating-point
264 types, which was introduced in `Clang 13`__. T 234 types, which was introduced in `Clang 13`__. The older Clang versions will
265 either crash when compiling these tests, or ge 235 either crash when compiling these tests, or generate an incorrect BTF.
266 236
267 __ https://github.com/llvm/llvm-project/commi !! 237 __ https://reviews.llvm.org/D83289
268 238
269 Kernel function call test and Clang version 239 Kernel function call test and Clang version
270 =========================================== 240 ===========================================
271 241
272 Some selftests (e.g. kfunc_call and bpf_tcp_ca 242 Some selftests (e.g. kfunc_call and bpf_tcp_ca) require a LLVM support
273 to generate extern function in BTF. It was in 243 to generate extern function in BTF. It was introduced in `Clang 13`__.
274 244
275 Without it, the error from compiling bpf selft 245 Without it, the error from compiling bpf selftests looks like:
276 246
277 .. code-block:: console 247 .. code-block:: console
278 248
279 libbpf: failed to find BTF for extern 'tcp_s 249 libbpf: failed to find BTF for extern 'tcp_slow_start' [25] section: -2
280 250
281 __ https://github.com/llvm/llvm-project/commit !! 251 __ https://reviews.llvm.org/D93563
282 252
283 btf_tag test and Clang version 253 btf_tag test and Clang version
284 ============================== 254 ==============================
285 255
286 The btf_tag selftest requires LLVM support to 256 The btf_tag selftest requires LLVM support to recognize the btf_decl_tag and
287 btf_type_tag attributes. They are introduced i 257 btf_type_tag attributes. They are introduced in `Clang 14` [0_, 1_].
288 The subtests ``btf_type_tag_user_{mod1, mod2, 258 The subtests ``btf_type_tag_user_{mod1, mod2, vmlinux}`` also requires
289 pahole version ``1.23``. 259 pahole version ``1.23``.
290 260
291 Without them, the btf_tag selftest will be ski 261 Without them, the btf_tag selftest will be skipped and you will observe:
292 262
293 .. code-block:: console 263 .. code-block:: console
294 264
295 #<test_num> btf_tag:SKIP 265 #<test_num> btf_tag:SKIP
296 266
297 .. _0: https://github.com/llvm/llvm-project/co !! 267 .. _0: https://reviews.llvm.org/D111588
298 .. _1: https://github.com/llvm/llvm-project/co !! 268 .. _1: https://reviews.llvm.org/D111199
299 269
300 Clang dependencies for static linking tests 270 Clang dependencies for static linking tests
301 =========================================== 271 ===========================================
302 272
303 linked_vars, linked_maps, and linked_funcs tes 273 linked_vars, linked_maps, and linked_funcs tests depend on `Clang fix`__ to
304 generate valid BTF information for weak variab 274 generate valid BTF information for weak variables. Please make sure you use
305 Clang that contains the fix. 275 Clang that contains the fix.
306 276
307 __ https://github.com/llvm/llvm-project/commit !! 277 __ https://reviews.llvm.org/D100362
308 278
309 Clang relocation changes 279 Clang relocation changes
310 ======================== 280 ========================
311 281
312 Clang 13 patch `clang reloc patch`_ made some 282 Clang 13 patch `clang reloc patch`_ made some changes on relocations such
313 that existing relocation types are broken into 283 that existing relocation types are broken into more types and
314 each new type corresponds to only one way to r 284 each new type corresponds to only one way to resolve relocation.
315 See `kernel llvm reloc`_ for more explanation 285 See `kernel llvm reloc`_ for more explanation and some examples.
316 Using clang 13 to compile old libbpf which has 286 Using clang 13 to compile old libbpf which has static linker support,
317 there will be a compilation failure:: 287 there will be a compilation failure::
318 288
319 libbpf: ELF relo #0 in section #6 has unexpe 289 libbpf: ELF relo #0 in section #6 has unexpected type 2 in .../bpf_tcp_nogpl.bpf.o
320 290
321 Here, ``type 2`` refers to new relocation type 291 Here, ``type 2`` refers to new relocation type ``R_BPF_64_ABS64``.
322 To fix this issue, user newer libbpf. 292 To fix this issue, user newer libbpf.
323 293
324 .. Links 294 .. Links
325 .. _clang reloc patch: https://github.com/llvm !! 295 .. _clang reloc patch: https://reviews.llvm.org/D102712
326 .. _kernel llvm reloc: /Documentation/bpf/llvm 296 .. _kernel llvm reloc: /Documentation/bpf/llvm_reloc.rst
327 297
328 Clang dependencies for the u32 spill test (xdp 298 Clang dependencies for the u32 spill test (xdpwall)
329 ============================================== 299 ===================================================
330 The xdpwall selftest requires a change in `Cla 300 The xdpwall selftest requires a change in `Clang 14`__.
331 301
332 Without it, the xdpwall selftest will fail and 302 Without it, the xdpwall selftest will fail and the error message
333 from running test_progs will look like: 303 from running test_progs will look like:
334 304
335 .. code-block:: console 305 .. code-block:: console
336 306
337 test_xdpwall:FAIL:Does LLVM have https://git !! 307 test_xdpwall:FAIL:Does LLVM have https://reviews.llvm.org/D109073? unexpected error: -4007
338 308
339 __ https://github.com/llvm/llvm-project/commit !! 309 __ https://reviews.llvm.org/D109073
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.