1 =================== 1 ===================
2 Testing BPF on s390 2 Testing BPF on s390
3 =================== 3 ===================
4 4
5 1. Introduction 5 1. Introduction
6 *************** 6 ***************
7 7
8 IBM Z are mainframe computers, which are desce 8 IBM Z are mainframe computers, which are descendants of IBM System/360 from
9 year 1964. They are supported by the Linux ker 9 year 1964. They are supported by the Linux kernel under the name "s390". This
10 document describes how to test BPF in an s390 10 document describes how to test BPF in an s390 QEMU guest.
11 11
12 2. One-time setup 12 2. One-time setup
13 ***************** 13 *****************
14 14
15 The following is required to build and run the 15 The following is required to build and run the test suite:
16 16
17 * s390 GCC 17 * s390 GCC
18 * s390 development headers and libraries 18 * s390 development headers and libraries
19 * Clang with BPF support 19 * Clang with BPF support
20 * QEMU with s390 support 20 * QEMU with s390 support
21 * Disk image with s390 rootfs 21 * Disk image with s390 rootfs
22 22
23 Debian supports installing compiler and librar 23 Debian supports installing compiler and libraries for s390 out of the box.
24 Users of other distros may use debootstrap in 24 Users of other distros may use debootstrap in order to set up a Debian chroot::
25 25
26 sudo debootstrap \ 26 sudo debootstrap \
27 --variant=minbase \ 27 --variant=minbase \
28 --include=sudo \ 28 --include=sudo \
29 testing \ 29 testing \
30 ./s390-toolchain 30 ./s390-toolchain
31 sudo mount --rbind /dev ./s390-toolchain/dev 31 sudo mount --rbind /dev ./s390-toolchain/dev
32 sudo mount --rbind /proc ./s390-toolchain/pr 32 sudo mount --rbind /proc ./s390-toolchain/proc
33 sudo mount --rbind /sys ./s390-toolchain/sys 33 sudo mount --rbind /sys ./s390-toolchain/sys
34 sudo chroot ./s390-toolchain 34 sudo chroot ./s390-toolchain
35 35
36 Once on Debian, the build prerequisites can be 36 Once on Debian, the build prerequisites can be installed as follows::
37 37
38 sudo dpkg --add-architecture s390x 38 sudo dpkg --add-architecture s390x
39 sudo apt-get update 39 sudo apt-get update
40 sudo apt-get install \ 40 sudo apt-get install \
41 bc \ 41 bc \
42 bison \ 42 bison \
43 cmake \ 43 cmake \
44 debootstrap \ 44 debootstrap \
45 dwarves \ 45 dwarves \
46 flex \ 46 flex \
47 g++ \ 47 g++ \
48 gcc \ 48 gcc \
49 g++-s390x-linux-gnu \ 49 g++-s390x-linux-gnu \
50 gcc-s390x-linux-gnu \ 50 gcc-s390x-linux-gnu \
51 gdb-multiarch \ 51 gdb-multiarch \
52 git \ 52 git \
53 make \ 53 make \
54 python3 \ 54 python3 \
55 qemu-system-misc \ 55 qemu-system-misc \
56 qemu-utils \ 56 qemu-utils \
57 rsync \ 57 rsync \
58 libcap-dev:s390x \ 58 libcap-dev:s390x \
59 libelf-dev:s390x \ 59 libelf-dev:s390x \
60 libncurses-dev 60 libncurses-dev
61 61
62 Latest Clang targeting BPF can be installed as 62 Latest Clang targeting BPF can be installed as follows::
63 63
64 git clone https://github.com/llvm/llvm-proje 64 git clone https://github.com/llvm/llvm-project.git
65 ln -s ../../clang llvm-project/llvm/tools/ 65 ln -s ../../clang llvm-project/llvm/tools/
66 mkdir llvm-project-build 66 mkdir llvm-project-build
67 cd llvm-project-build 67 cd llvm-project-build
68 cmake \ 68 cmake \
69 -DLLVM_TARGETS_TO_BUILD=BPF \ 69 -DLLVM_TARGETS_TO_BUILD=BPF \
70 -DCMAKE_BUILD_TYPE=Release \ 70 -DCMAKE_BUILD_TYPE=Release \
71 -DCMAKE_INSTALL_PREFIX=/opt/clang-bpf \ 71 -DCMAKE_INSTALL_PREFIX=/opt/clang-bpf \
72 ../llvm-project/llvm 72 ../llvm-project/llvm
73 make 73 make
74 sudo make install 74 sudo make install
75 export PATH=/opt/clang-bpf/bin:$PATH 75 export PATH=/opt/clang-bpf/bin:$PATH
76 76
77 The disk image can be prepared using a loopbac 77 The disk image can be prepared using a loopback mount and debootstrap::
78 78
79 qemu-img create -f raw ./s390.img 1G 79 qemu-img create -f raw ./s390.img 1G
80 sudo losetup -f ./s390.img 80 sudo losetup -f ./s390.img
81 sudo mkfs.ext4 /dev/loopX 81 sudo mkfs.ext4 /dev/loopX
82 mkdir ./s390.rootfs 82 mkdir ./s390.rootfs
83 sudo mount /dev/loopX ./s390.rootfs 83 sudo mount /dev/loopX ./s390.rootfs
84 sudo debootstrap \ 84 sudo debootstrap \
85 --foreign \ 85 --foreign \
86 --arch=s390x \ 86 --arch=s390x \
87 --variant=minbase \ 87 --variant=minbase \
88 --include=" \ 88 --include=" \
89 iproute2, \ 89 iproute2, \
90 iputils-ping, \ 90 iputils-ping, \
91 isc-dhcp-client, \ 91 isc-dhcp-client, \
92 kmod, \ 92 kmod, \
93 libcap2, \ 93 libcap2, \
94 libelf1, \ 94 libelf1, \
95 netcat, \ 95 netcat, \
96 procps" \ 96 procps" \
97 testing \ 97 testing \
98 ./s390.rootfs 98 ./s390.rootfs
99 sudo umount ./s390.rootfs 99 sudo umount ./s390.rootfs
100 sudo losetup -d /dev/loopX 100 sudo losetup -d /dev/loopX
101 101
102 3. Compilation 102 3. Compilation
103 ************** 103 **************
104 104
105 In addition to the usual Kconfig options requi 105 In addition to the usual Kconfig options required to run the BPF test suite, it
106 is also helpful to select:: 106 is also helpful to select::
107 107
108 CONFIG_NET_9P=y 108 CONFIG_NET_9P=y
109 CONFIG_9P_FS=y 109 CONFIG_9P_FS=y
110 CONFIG_NET_9P_VIRTIO=y 110 CONFIG_NET_9P_VIRTIO=y
111 CONFIG_VIRTIO_PCI=y 111 CONFIG_VIRTIO_PCI=y
112 112
113 as that would enable a very easy way to share 113 as that would enable a very easy way to share files with the s390 virtual
114 machine. 114 machine.
115 115
116 Compiling kernel, modules and testsuite, as we 116 Compiling kernel, modules and testsuite, as well as preparing gdb scripts to
117 simplify debugging, can be done using the foll 117 simplify debugging, can be done using the following commands::
118 118
119 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu 119 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- menuconfig
120 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu 120 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- bzImage modules scripts_gdb
121 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu 121 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- \
122 -C tools/testing/selftests \ 122 -C tools/testing/selftests \
123 TARGETS=bpf \ 123 TARGETS=bpf \
124 INSTALL_PATH=$PWD/tools/testing/selftests/ 124 INSTALL_PATH=$PWD/tools/testing/selftests/kselftest_install \
125 install 125 install
126 126
127 4. Running the test suite 127 4. Running the test suite
128 ************************* 128 *************************
129 129
130 The virtual machine can be started as follows: 130 The virtual machine can be started as follows::
131 131
132 qemu-system-s390x \ 132 qemu-system-s390x \
133 -cpu max,zpci=on \ 133 -cpu max,zpci=on \
134 -smp 2 \ 134 -smp 2 \
135 -m 4G \ 135 -m 4G \
136 -kernel linux/arch/s390/boot/compressed/vm 136 -kernel linux/arch/s390/boot/compressed/vmlinux \
137 -drive file=./s390.img,if=virtio,format=ra 137 -drive file=./s390.img,if=virtio,format=raw \
138 -nographic \ 138 -nographic \
139 -append 'root=/dev/vda rw console=ttyS1' \ 139 -append 'root=/dev/vda rw console=ttyS1' \
140 -virtfs local,path=./linux,security_model= 140 -virtfs local,path=./linux,security_model=none,mount_tag=linux \
141 -object rng-random,filename=/dev/urandom,i 141 -object rng-random,filename=/dev/urandom,id=rng0 \
142 -device virtio-rng-ccw,rng=rng0 \ 142 -device virtio-rng-ccw,rng=rng0 \
143 -netdev user,id=net0 \ 143 -netdev user,id=net0 \
144 -device virtio-net-ccw,netdev=net0 144 -device virtio-net-ccw,netdev=net0
145 145
146 When using this on a real IBM Z, ``-enable-kvm 146 When using this on a real IBM Z, ``-enable-kvm`` may be added for better
147 performance. When starting the virtual machine 147 performance. When starting the virtual machine for the first time, disk image
148 setup must be finalized using the following co 148 setup must be finalized using the following command::
149 149
150 /debootstrap/debootstrap --second-stage 150 /debootstrap/debootstrap --second-stage
151 151
152 Directory with the code built on the host as w 152 Directory with the code built on the host as well as ``/proc`` and ``/sys``
153 need to be mounted as follows:: 153 need to be mounted as follows::
154 154
155 mkdir -p /linux 155 mkdir -p /linux
156 mount -t 9p linux /linux 156 mount -t 9p linux /linux
157 mount -t proc proc /proc 157 mount -t proc proc /proc
158 mount -t sysfs sys /sys 158 mount -t sysfs sys /sys
159 159
160 After that, the test suite can be run using th 160 After that, the test suite can be run using the following commands::
161 161
162 cd /linux/tools/testing/selftests/kselftest_ 162 cd /linux/tools/testing/selftests/kselftest_install
163 ./run_kselftest.sh 163 ./run_kselftest.sh
164 164
165 As usual, tests can be also run individually:: 165 As usual, tests can be also run individually::
166 166
167 cd /linux/tools/testing/selftests/bpf 167 cd /linux/tools/testing/selftests/bpf
168 ./test_verifier 168 ./test_verifier
169 169
170 5. Debugging 170 5. Debugging
171 ************ 171 ************
172 172
173 It is possible to debug the s390 kernel using 173 It is possible to debug the s390 kernel using QEMU GDB stub, which is activated
174 by passing ``-s`` to QEMU. 174 by passing ``-s`` to QEMU.
175 175
176 It is preferable to turn KASLR off, so that gd 176 It is preferable to turn KASLR off, so that gdb would know where to find the
177 kernel image in memory, by building the kernel 177 kernel image in memory, by building the kernel with::
178 178
179 RANDOMIZE_BASE=n 179 RANDOMIZE_BASE=n
180 180
181 GDB can then be attached using the following c 181 GDB can then be attached using the following command::
182 182
183 gdb-multiarch -ex 'target remote localhost:1 183 gdb-multiarch -ex 'target remote localhost:1234' ./vmlinux
184 184
185 6. Network 185 6. Network
186 ********** 186 **********
187 187
188 In case one needs to use the network in the vi 188 In case one needs to use the network in the virtual machine in order to e.g.
189 install additional packages, it can be configu 189 install additional packages, it can be configured using::
190 190
191 dhclient eth0 191 dhclient eth0
192 192
193 7. Links 193 7. Links
194 ******** 194 ********
195 195
196 This document is a compilation of techniques, 196 This document is a compilation of techniques, whose more comprehensive
197 descriptions can be found by following these l 197 descriptions can be found by following these links:
198 198
199 - `Debootstrap <https://wiki.debian.org/EmDebi 199 - `Debootstrap <https://wiki.debian.org/EmDebian/CrossDebootstrap>`_
200 - `Multiarch <https://wiki.debian.org/Multiarc 200 - `Multiarch <https://wiki.debian.org/Multiarch/HOWTO>`_
201 - `Building LLVM <https://llvm.org/docs/CMake. 201 - `Building LLVM <https://llvm.org/docs/CMake.html>`_
202 - `Cross-compiling the kernel <https://wiki.ge 202 - `Cross-compiling the kernel <https://wiki.gentoo.org/wiki/Embedded_Handbook/General/Cross-compiling_the_kernel>`_
203 - `QEMU s390x Guest Support <https://wiki.qemu 203 - `QEMU s390x Guest Support <https://wiki.qemu.org/Documentation/Platforms/S390X>`_
204 - `Plan 9 folder sharing over Virtio <https:// 204 - `Plan 9 folder sharing over Virtio <https://wiki.qemu.org/Documentation/9psetup>`_
205 - `Using GDB with QEMU <https://wiki.osdev.org 205 - `Using GDB with QEMU <https://wiki.osdev.org/Kernel_Debugging#Use_GDB_with_QEMU>`_
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.