1 ================== 1 ================== 2 BPF Selftest Notes 2 BPF Selftest Notes 3 ================== 3 ================== 4 General instructions on running selftests can 4 General instructions on running selftests can be found in 5 `Documentation/bpf/bpf_devel_QA.rst`__. 5 `Documentation/bpf/bpf_devel_QA.rst`__. 6 6 7 __ /Documentation/bpf/bpf_devel_QA.rst#q-how-t 7 __ /Documentation/bpf/bpf_devel_QA.rst#q-how-to-run-bpf-selftests 8 8 9 ============= 9 ============= 10 BPF CI System 10 BPF CI System 11 ============= 11 ============= 12 12 13 BPF employs a continuous integration (CI) syst 13 BPF employs a continuous integration (CI) system to check patch submission in an 14 automated fashion. The system runs selftests f 14 automated fashion. The system runs selftests for each patch in a series. Results 15 are propagated to patchwork, where failures ar 15 are propagated to patchwork, where failures are highlighted similar to 16 violations of other checks (such as additional 16 violations of other checks (such as additional warnings being emitted or a 17 ``scripts/checkpatch.pl`` reported deficiency) 17 ``scripts/checkpatch.pl`` reported deficiency): 18 18 19 https://patchwork.kernel.org/project/netdevb 19 https://patchwork.kernel.org/project/netdevbpf/list/?delegate=121173 20 20 21 The CI system executes tests on multiple archi 21 The CI system executes tests on multiple architectures. It uses a kernel 22 configuration derived from both the generic an 22 configuration derived from both the generic and architecture specific config 23 file fragments below ``tools/testing/selftests 23 file fragments below ``tools/testing/selftests/bpf/`` (e.g., ``config`` and 24 ``config.x86_64``). 24 ``config.x86_64``). 25 25 26 Denylisting Tests 26 Denylisting Tests 27 ================= 27 ================= 28 28 29 It is possible for some architectures to not h 29 It is possible for some architectures to not have support for all BPF features. 30 In such a case tests in CI may fail. An exampl 30 In such a case tests in CI may fail. An example of such a shortcoming is BPF 31 trampoline support on IBM's s390x architecture 31 trampoline support on IBM's s390x architecture. For cases like this, an in-tree 32 deny list file, located at ``tools/testing/sel 32 deny list file, located at ``tools/testing/selftests/bpf/DENYLIST.<arch>``, can 33 be used to prevent the test from running on su 33 be used to prevent the test from running on such an architecture. 34 34 35 In addition to that, the generic ``tools/testi 35 In addition to that, the generic ``tools/testing/selftests/bpf/DENYLIST`` is 36 honored on every architecture running tests. 36 honored on every architecture running tests. 37 37 38 These files are organized in three columns. Th 38 These files are organized in three columns. The first column lists the test in 39 question. This can be the name of a test suite 39 question. This can be the name of a test suite or of an individual test. The 40 remaining two columns provide additional meta 40 remaining two columns provide additional meta data that helps identify and 41 classify the entry: column two is a copy and p 41 classify the entry: column two is a copy and paste of the error being reported 42 when running the test in the setting in questi 42 when running the test in the setting in question. The third column, if 43 available, summarizes the underlying problem. 43 available, summarizes the underlying problem. A value of ``trampoline``, for 44 example, indicates that lack of trampoline sup 44 example, indicates that lack of trampoline support is causing the test to fail. 45 This last entry helps identify tests that can 45 This last entry helps identify tests that can be re-enabled once such support is 46 added. 46 added. 47 47 48 ========================= 48 ========================= 49 Running Selftests in a VM 49 Running Selftests in a VM 50 ========================= 50 ========================= 51 51 52 It's now possible to run the selftests using ` 52 It's now possible to run the selftests using ``tools/testing/selftests/bpf/vmtest.sh``. 53 The script tries to ensure that the tests are 53 The script tries to ensure that the tests are run with the same environment as they 54 would be run post-submit in the CI used by the 54 would be run post-submit in the CI used by the Maintainers, with the exception 55 that deny lists are not automatically honored. 55 that deny lists are not automatically honored. 56 56 57 This script uses the in-tree kernel configurat 57 This script uses the in-tree kernel configuration and downloads a VM userspace 58 image from the system used by the CI. It build 58 image from the system used by the CI. It builds the kernel (without overwriting 59 your existing Kconfig), recompiles the bpf sel 59 your existing Kconfig), recompiles the bpf selftests, runs them (by default 60 ``tools/testing/selftests/bpf/test_progs``) an 60 ``tools/testing/selftests/bpf/test_progs``) and saves the resulting output (by 61 default in ``~/.bpf_selftests``). 61 default in ``~/.bpf_selftests``). 62 62 63 Script dependencies: 63 Script dependencies: 64 - clang (preferably built from sources, https: 64 - clang (preferably built from sources, https://github.com/llvm/llvm-project); 65 - pahole (preferably built from sources, https 65 - pahole (preferably built from sources, https://git.kernel.org/pub/scm/devel/pahole/pahole.git/); 66 - qemu; 66 - qemu; 67 - docutils (for ``rst2man``); 67 - docutils (for ``rst2man``); 68 - libcap-devel. 68 - libcap-devel. 69 69 70 For more information about using the script, r 70 For more information about using the script, run: 71 71 72 .. code-block:: console 72 .. code-block:: console 73 73 74 $ tools/testing/selftests/bpf/vmtest.sh -h 74 $ tools/testing/selftests/bpf/vmtest.sh -h 75 75 76 In case of linker errors when running selftest 76 In case of linker errors when running selftests, try using static linking: 77 77 78 .. code-block:: console 78 .. code-block:: console 79 79 80 $ LDLIBS=-static PKG_CONFIG='pkg-config --st 80 $ LDLIBS=-static PKG_CONFIG='pkg-config --static' vmtest.sh 81 81 82 .. note:: Some distros may not support static 82 .. note:: Some distros may not support static linking. 83 83 84 .. note:: The script uses pahole and clang bas 84 .. note:: The script uses pahole and clang based on host environment setting. 85 If you want to change pahole and llv 85 If you want to change pahole and llvm, you can change `PATH` environment 86 variable in the beginning of script. 86 variable in the beginning of script. 87 87 88 Running vmtest on RV64 !! 88 .. note:: The script currently only supports x86_64 and s390x architectures. 89 ====================== << 90 To speed up testing and avoid various dependen << 91 run vmtest in a Docker container. Before runni << 92 Docker container and local rootfs image. The o << 93 << 94 1. Create Docker container as shown in link [0 << 95 << 96 2. Use mkrootfs_debian.sh script [1] to build << 97 << 98 .. code-block:: console << 99 << 100 $ sudo ./mkrootfs_debian.sh --arch riscv64 - << 101 << 102 3. Start Docker container [0] and run vmtest i << 103 << 104 .. code-block:: console << 105 << 106 $ PLATFORM=riscv64 CROSS_COMPILE=riscv64-lin << 107 tools/testing/selftests/bpf/vmtest.sh \ << 108 -l <path of local rootfs image> -- \ << 109 ./test_progs -d \ << 110 \"$(cat tools/testing/selftests/bpf/DE << 111 | cut -d'#' -f1 \ << 112 | sed -e 's/^[[:space:]]*//' \ << 113 -e 's/[[:space:]]*$//' \ << 114 | tr -s '\n' ',' \ << 115 )\" << 116 << 117 Link: https://github.com/pulehui/riscv-bpf-vmt << 118 Link: https://github.com/libbpf/ci/blob/main/r << 119 89 120 Additional information about selftest failures 90 Additional information about selftest failures are 121 documented here. 91 documented here. 122 92 123 profiler[23] test failures with clang/llvm <12 93 profiler[23] test failures with clang/llvm <12.0.0 124 ============================================== 94 ================================================== 125 95 126 With clang/llvm <12.0.0, the profiler[23] test 96 With clang/llvm <12.0.0, the profiler[23] test may fail. 127 The symptom looks like 97 The symptom looks like 128 98 129 .. code-block:: c 99 .. code-block:: c 130 100 131 // r9 is a pointer to map_value 101 // r9 is a pointer to map_value 132 // r7 is a scalar 102 // r7 is a scalar 133 17: bf 96 00 00 00 00 00 00 r6 = r9 103 17: bf 96 00 00 00 00 00 00 r6 = r9 134 18: 0f 76 00 00 00 00 00 00 r6 += r7 104 18: 0f 76 00 00 00 00 00 00 r6 += r7 135 math between map_value pointer and register 105 math between map_value pointer and register with unbounded min value is not allowed 136 106 137 // the instructions below will not be seen i 107 // the instructions below will not be seen in the verifier log 138 19: a5 07 01 00 01 01 00 00 if r7 < 25 108 19: a5 07 01 00 01 01 00 00 if r7 < 257 goto +1 139 20: bf 96 00 00 00 00 00 00 r6 = r9 109 20: bf 96 00 00 00 00 00 00 r6 = r9 140 // r6 is used here 110 // r6 is used here 141 111 142 The verifier will reject such code with above 112 The verifier will reject such code with above error. 143 At insn 18 the r7 is indeed unbounded. The lat 113 At insn 18 the r7 is indeed unbounded. The later insn 19 checks the bounds and 144 the insn 20 undoes map_value addition. It is c 114 the insn 20 undoes map_value addition. It is currently impossible for the 145 verifier to understand such speculative pointe 115 verifier to understand such speculative pointer arithmetic. 146 Hence `this patch`__ addresses it on the compi 116 Hence `this patch`__ addresses it on the compiler side. It was committed on llvm 12. 147 117 148 __ https://github.com/llvm/llvm-project/commit 118 __ https://github.com/llvm/llvm-project/commit/ddf1864ace484035e3cde5e83b3a31ac81e059c6 149 119 150 The corresponding C code 120 The corresponding C code 151 121 152 .. code-block:: c 122 .. code-block:: c 153 123 154 for (int i = 0; i < MAX_CGROUPS_PATH_DEPTH; 124 for (int i = 0; i < MAX_CGROUPS_PATH_DEPTH; i++) { 155 filepart_length = bpf_probe_read_str 125 filepart_length = bpf_probe_read_str(payload, ...); 156 if (filepart_length <= MAX_PATH) { 126 if (filepart_length <= MAX_PATH) { 157 barrier_var(filepart_length) 127 barrier_var(filepart_length); // workaround 158 payload += filepart_length; 128 payload += filepart_length; 159 } 129 } 160 } 130 } 161 131 162 bpf_iter test failures with clang/llvm 10.0.0 132 bpf_iter test failures with clang/llvm 10.0.0 163 ============================================= 133 ============================================= 164 134 165 With clang/llvm 10.0.0, the following two bpf_ 135 With clang/llvm 10.0.0, the following two bpf_iter tests failed: 166 * ``bpf_iter/ipv6_route`` 136 * ``bpf_iter/ipv6_route`` 167 * ``bpf_iter/netlink`` 137 * ``bpf_iter/netlink`` 168 138 169 The symptom for ``bpf_iter/ipv6_route`` looks 139 The symptom for ``bpf_iter/ipv6_route`` looks like 170 140 171 .. code-block:: c 141 .. code-block:: c 172 142 173 2: (79) r8 = *(u64 *)(r1 +8) 143 2: (79) r8 = *(u64 *)(r1 +8) 174 ... 144 ... 175 14: (bf) r2 = r8 145 14: (bf) r2 = r8 176 15: (0f) r2 += r1 146 15: (0f) r2 += r1 177 ; BPF_SEQ_PRINTF(seq, "%pi6 %02x ", &rt->fib 147 ; BPF_SEQ_PRINTF(seq, "%pi6 %02x ", &rt->fib6_dst.addr, rt->fib6_dst.plen); 178 16: (7b) *(u64 *)(r8 +64) = r2 148 16: (7b) *(u64 *)(r8 +64) = r2 179 only read is supported 149 only read is supported 180 150 181 The symptom for ``bpf_iter/netlink`` looks lik 151 The symptom for ``bpf_iter/netlink`` looks like 182 152 183 .. code-block:: c 153 .. code-block:: c 184 154 185 ; struct netlink_sock *nlk = ctx->sk; 155 ; struct netlink_sock *nlk = ctx->sk; 186 2: (79) r7 = *(u64 *)(r1 +8) 156 2: (79) r7 = *(u64 *)(r1 +8) 187 ... 157 ... 188 15: (bf) r2 = r7 158 15: (bf) r2 = r7 189 16: (0f) r2 += r1 159 16: (0f) r2 += r1 190 ; BPF_SEQ_PRINTF(seq, "%pK %-3d ", s, s->sk_ 160 ; BPF_SEQ_PRINTF(seq, "%pK %-3d ", s, s->sk_protocol); 191 17: (7b) *(u64 *)(r7 +0) = r2 161 17: (7b) *(u64 *)(r7 +0) = r2 192 only read is supported 162 only read is supported 193 163 194 This is due to a llvm BPF backend bug. `The fi 164 This is due to a llvm BPF backend bug. `The fix`__ 195 has been pushed to llvm 10.x release branch an 165 has been pushed to llvm 10.x release branch and will be 196 available in 10.0.1. The patch is available in 166 available in 10.0.1. The patch is available in llvm 11.0.0 trunk. 197 167 198 __ https://github.com/llvm/llvm-project/commi 168 __ https://github.com/llvm/llvm-project/commit/3cb7e7bf959dcd3b8080986c62e10a75c7af43f0 199 169 200 bpf_verif_scale/loop6.bpf.o test failure with 170 bpf_verif_scale/loop6.bpf.o test failure with Clang 12 201 ============================================== 171 ====================================================== 202 172 203 With Clang 12, the following bpf_verif_scale t 173 With Clang 12, the following bpf_verif_scale test failed: 204 * ``bpf_verif_scale/loop6.bpf.o`` 174 * ``bpf_verif_scale/loop6.bpf.o`` 205 175 206 The verifier output looks like 176 The verifier output looks like 207 177 208 .. code-block:: c 178 .. code-block:: c 209 179 210 R1 type=ctx expected=fp 180 R1 type=ctx expected=fp 211 The sequence of 8193 jumps is too complex. 181 The sequence of 8193 jumps is too complex. 212 182 213 The reason is compiler generating the followin 183 The reason is compiler generating the following code 214 184 215 .. code-block:: c 185 .. code-block:: c 216 186 217 ; for (i = 0; (i < VIRTIO_MAX_SGS) && 187 ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) { 218 14: 16 05 40 00 00 00 00 00 if w5 188 14: 16 05 40 00 00 00 00 00 if w5 == 0 goto +64 <LBB0_6> 219 15: bc 51 00 00 00 00 00 00 w1 = w 189 15: bc 51 00 00 00 00 00 00 w1 = w5 220 16: 04 01 00 00 ff ff ff ff w1 += 190 16: 04 01 00 00 ff ff ff ff w1 += -1 221 17: 67 05 00 00 20 00 00 00 r5 <<= 191 17: 67 05 00 00 20 00 00 00 r5 <<= 32 222 18: 77 05 00 00 20 00 00 00 r5 >>= 192 18: 77 05 00 00 20 00 00 00 r5 >>= 32 223 19: a6 01 01 00 05 00 00 00 if w1 193 19: a6 01 01 00 05 00 00 00 if w1 < 5 goto +1 <LBB0_4> 224 20: b7 05 00 00 06 00 00 00 r5 = 6 194 20: b7 05 00 00 06 00 00 00 r5 = 6 225 00000000000000a8 <LBB0_4>: 195 00000000000000a8 <LBB0_4>: 226 21: b7 02 00 00 00 00 00 00 r2 = 0 196 21: b7 02 00 00 00 00 00 00 r2 = 0 227 22: b7 01 00 00 00 00 00 00 r1 = 0 197 22: b7 01 00 00 00 00 00 00 r1 = 0 228 ; for (i = 0; (i < VIRTIO_MAX_SGS) && 198 ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) { 229 23: 7b 1a e0 ff 00 00 00 00 *(u64 199 23: 7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1 230 24: 7b 5a c0 ff 00 00 00 00 *(u64 200 24: 7b 5a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r5 231 201 232 Note that insn #15 has w1 = w5 and w1 is refin 202 Note that insn #15 has w1 = w5 and w1 is refined later but 233 r5(w5) is eventually saved on stack at insn #2 203 r5(w5) is eventually saved on stack at insn #24 for later use. 234 This cause later verifier failure. The bug has 204 This cause later verifier failure. The bug has been `fixed`__ in 235 Clang 13. 205 Clang 13. 236 206 237 __ https://github.com/llvm/llvm-project/commi 207 __ https://github.com/llvm/llvm-project/commit/1959ead525b8830cc8a345f45e1c3ef9902d3229 238 208 239 BPF CO-RE-based tests and Clang version 209 BPF CO-RE-based tests and Clang version 240 ======================================= 210 ======================================= 241 211 242 A set of selftests use BPF target-specific bui 212 A set of selftests use BPF target-specific built-ins, which might require 243 bleeding-edge Clang versions (Clang 12 nightly 213 bleeding-edge Clang versions (Clang 12 nightly at this time). 244 214 245 Few sub-tests of core_reloc test suit (part of 215 Few sub-tests of core_reloc test suit (part of test_progs test runner) require 246 the following built-ins, listed with correspon 216 the following built-ins, listed with corresponding Clang diffs introducing 247 them to Clang/LLVM. These sub-tests are going 217 them to Clang/LLVM. These sub-tests are going to be skipped if Clang is too 248 old to support them, they shouldn't cause buil 218 old to support them, they shouldn't cause build failures or runtime test 249 failures: 219 failures: 250 220 251 - __builtin_btf_type_id() [0_, 1_, 2_]; 221 - __builtin_btf_type_id() [0_, 1_, 2_]; 252 - __builtin_preserve_type_info(), __builtin_pr 222 - __builtin_preserve_type_info(), __builtin_preserve_enum_value() [3_, 4_]. 253 223 254 .. _0: https://github.com/llvm/llvm-project/co 224 .. _0: https://github.com/llvm/llvm-project/commit/6b01b465388b204d543da3cf49efd6080db094a9 255 .. _1: https://github.com/llvm/llvm-project/co 225 .. _1: https://github.com/llvm/llvm-project/commit/072cde03aaa13a2c57acf62d79876bf79aa1919f 256 .. _2: https://github.com/llvm/llvm-project/co 226 .. _2: https://github.com/llvm/llvm-project/commit/00602ee7ef0bf6c68d690a2bd729c12b95c95c99 257 .. _3: https://github.com/llvm/llvm-project/co 227 .. _3: https://github.com/llvm/llvm-project/commit/6d218b4adb093ff2e9764febbbc89f429412006c 258 .. _4: https://github.com/llvm/llvm-project/co 228 .. _4: https://github.com/llvm/llvm-project/commit/6d6750696400e7ce988d66a1a00e1d0cb32815f8 259 229 260 Floating-point tests and Clang version 230 Floating-point tests and Clang version 261 ====================================== 231 ====================================== 262 232 263 Certain selftests, e.g. core_reloc, require su 233 Certain selftests, e.g. core_reloc, require support for the floating-point 264 types, which was introduced in `Clang 13`__. T 234 types, which was introduced in `Clang 13`__. The older Clang versions will 265 either crash when compiling these tests, or ge 235 either crash when compiling these tests, or generate an incorrect BTF. 266 236 267 __ https://github.com/llvm/llvm-project/commi 237 __ https://github.com/llvm/llvm-project/commit/a7137b238a07d9399d3ae96c0b461571bd5aa8b2 268 238 269 Kernel function call test and Clang version 239 Kernel function call test and Clang version 270 =========================================== 240 =========================================== 271 241 272 Some selftests (e.g. kfunc_call and bpf_tcp_ca 242 Some selftests (e.g. kfunc_call and bpf_tcp_ca) require a LLVM support 273 to generate extern function in BTF. It was in 243 to generate extern function in BTF. It was introduced in `Clang 13`__. 274 244 275 Without it, the error from compiling bpf selft 245 Without it, the error from compiling bpf selftests looks like: 276 246 277 .. code-block:: console 247 .. code-block:: console 278 248 279 libbpf: failed to find BTF for extern 'tcp_s 249 libbpf: failed to find BTF for extern 'tcp_slow_start' [25] section: -2 280 250 281 __ https://github.com/llvm/llvm-project/commit 251 __ https://github.com/llvm/llvm-project/commit/886f9ff53155075bd5f1e994f17b85d1e1b7470c 282 252 283 btf_tag test and Clang version 253 btf_tag test and Clang version 284 ============================== 254 ============================== 285 255 286 The btf_tag selftest requires LLVM support to 256 The btf_tag selftest requires LLVM support to recognize the btf_decl_tag and 287 btf_type_tag attributes. They are introduced i 257 btf_type_tag attributes. They are introduced in `Clang 14` [0_, 1_]. 288 The subtests ``btf_type_tag_user_{mod1, mod2, 258 The subtests ``btf_type_tag_user_{mod1, mod2, vmlinux}`` also requires 289 pahole version ``1.23``. 259 pahole version ``1.23``. 290 260 291 Without them, the btf_tag selftest will be ski 261 Without them, the btf_tag selftest will be skipped and you will observe: 292 262 293 .. code-block:: console 263 .. code-block:: console 294 264 295 #<test_num> btf_tag:SKIP 265 #<test_num> btf_tag:SKIP 296 266 297 .. _0: https://github.com/llvm/llvm-project/co 267 .. _0: https://github.com/llvm/llvm-project/commit/a162b67c98066218d0d00aa13b99afb95d9bb5e6 298 .. _1: https://github.com/llvm/llvm-project/co 268 .. _1: https://github.com/llvm/llvm-project/commit/3466e00716e12e32fdb100e3fcfca5c2b3e8d784 299 269 300 Clang dependencies for static linking tests 270 Clang dependencies for static linking tests 301 =========================================== 271 =========================================== 302 272 303 linked_vars, linked_maps, and linked_funcs tes 273 linked_vars, linked_maps, and linked_funcs tests depend on `Clang fix`__ to 304 generate valid BTF information for weak variab 274 generate valid BTF information for weak variables. Please make sure you use 305 Clang that contains the fix. 275 Clang that contains the fix. 306 276 307 __ https://github.com/llvm/llvm-project/commit 277 __ https://github.com/llvm/llvm-project/commit/968292cb93198442138128d850fd54dc7edc0035 308 278 309 Clang relocation changes 279 Clang relocation changes 310 ======================== 280 ======================== 311 281 312 Clang 13 patch `clang reloc patch`_ made some 282 Clang 13 patch `clang reloc patch`_ made some changes on relocations such 313 that existing relocation types are broken into 283 that existing relocation types are broken into more types and 314 each new type corresponds to only one way to r 284 each new type corresponds to only one way to resolve relocation. 315 See `kernel llvm reloc`_ for more explanation 285 See `kernel llvm reloc`_ for more explanation and some examples. 316 Using clang 13 to compile old libbpf which has 286 Using clang 13 to compile old libbpf which has static linker support, 317 there will be a compilation failure:: 287 there will be a compilation failure:: 318 288 319 libbpf: ELF relo #0 in section #6 has unexpe 289 libbpf: ELF relo #0 in section #6 has unexpected type 2 in .../bpf_tcp_nogpl.bpf.o 320 290 321 Here, ``type 2`` refers to new relocation type 291 Here, ``type 2`` refers to new relocation type ``R_BPF_64_ABS64``. 322 To fix this issue, user newer libbpf. 292 To fix this issue, user newer libbpf. 323 293 324 .. Links 294 .. Links 325 .. _clang reloc patch: https://github.com/llvm 295 .. _clang reloc patch: https://github.com/llvm/llvm-project/commit/6a2ea84600ba4bd3b2733bd8f08f5115eb32164b 326 .. _kernel llvm reloc: /Documentation/bpf/llvm 296 .. _kernel llvm reloc: /Documentation/bpf/llvm_reloc.rst 327 297 328 Clang dependencies for the u32 spill test (xdp 298 Clang dependencies for the u32 spill test (xdpwall) 329 ============================================== 299 =================================================== 330 The xdpwall selftest requires a change in `Cla 300 The xdpwall selftest requires a change in `Clang 14`__. 331 301 332 Without it, the xdpwall selftest will fail and 302 Without it, the xdpwall selftest will fail and the error message 333 from running test_progs will look like: 303 from running test_progs will look like: 334 304 335 .. code-block:: console 305 .. code-block:: console 336 306 337 test_xdpwall:FAIL:Does LLVM have https://git 307 test_xdpwall:FAIL:Does LLVM have https://github.com/llvm/llvm-project/commit/ea72b0319d7b0f0c2fcf41d121afa5d031b319d5? unexpected error: -4007 338 308 339 __ https://github.com/llvm/llvm-project/commit 309 __ https://github.com/llvm/llvm-project/commit/ea72b0319d7b0f0c2fcf41d121afa5d031b319d5
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.