1 =================== 2 Testing BPF on s390 3 =================== 4 5 1. Introduction 6 *************** 7 8 IBM Z are mainframe computers, which are descendants of IBM System/360 from 9 year 1964. They are supported by the Linux kernel under the name "s390". This 10 document describes how to test BPF in an s390 QEMU guest. 11 12 2. One-time setup 13 ***************** 14 15 The following is required to build and run the test suite: 16 17 * s390 GCC 18 * s390 development headers and libraries 19 * Clang with BPF support 20 * QEMU with s390 support 21 * Disk image with s390 rootfs 22 23 Debian supports installing compiler and libraries for s390 out of the box. 24 Users of other distros may use debootstrap in order to set up a Debian chroot:: 25 26 sudo debootstrap \ 27 --variant=minbase \ 28 --include=sudo \ 29 testing \ 30 ./s390-toolchain 31 sudo mount --rbind /dev ./s390-toolchain/dev 32 sudo mount --rbind /proc ./s390-toolchain/proc 33 sudo mount --rbind /sys ./s390-toolchain/sys 34 sudo chroot ./s390-toolchain 35 36 Once on Debian, the build prerequisites can be installed as follows:: 37 38 sudo dpkg --add-architecture s390x 39 sudo apt-get update 40 sudo apt-get install \ 41 bc \ 42 bison \ 43 cmake \ 44 debootstrap \ 45 dwarves \ 46 flex \ 47 g++ \ 48 gcc \ 49 g++-s390x-linux-gnu \ 50 gcc-s390x-linux-gnu \ 51 gdb-multiarch \ 52 git \ 53 make \ 54 python3 \ 55 qemu-system-misc \ 56 qemu-utils \ 57 rsync \ 58 libcap-dev:s390x \ 59 libelf-dev:s390x \ 60 libncurses-dev 61 62 Latest Clang targeting BPF can be installed as follows:: 63 64 git clone https://github.com/llvm/llvm-project.git 65 ln -s ../../clang llvm-project/llvm/tools/ 66 mkdir llvm-project-build 67 cd llvm-project-build 68 cmake \ 69 -DLLVM_TARGETS_TO_BUILD=BPF \ 70 -DCMAKE_BUILD_TYPE=Release \ 71 -DCMAKE_INSTALL_PREFIX=/opt/clang-bpf \ 72 ../llvm-project/llvm 73 make 74 sudo make install 75 export PATH=/opt/clang-bpf/bin:$PATH 76 77 The disk image can be prepared using a loopback mount and debootstrap:: 78 79 qemu-img create -f raw ./s390.img 1G 80 sudo losetup -f ./s390.img 81 sudo mkfs.ext4 /dev/loopX 82 mkdir ./s390.rootfs 83 sudo mount /dev/loopX ./s390.rootfs 84 sudo debootstrap \ 85 --foreign \ 86 --arch=s390x \ 87 --variant=minbase \ 88 --include=" \ 89 iproute2, \ 90 iputils-ping, \ 91 isc-dhcp-client, \ 92 kmod, \ 93 libcap2, \ 94 libelf1, \ 95 netcat, \ 96 procps" \ 97 testing \ 98 ./s390.rootfs 99 sudo umount ./s390.rootfs 100 sudo losetup -d /dev/loopX 101 102 3. Compilation 103 ************** 104 105 In addition to the usual Kconfig options required to run the BPF test suite, it 106 is also helpful to select:: 107 108 CONFIG_NET_9P=y 109 CONFIG_9P_FS=y 110 CONFIG_NET_9P_VIRTIO=y 111 CONFIG_VIRTIO_PCI=y 112 113 as that would enable a very easy way to share files with the s390 virtual 114 machine. 115 116 Compiling kernel, modules and testsuite, as well as preparing gdb scripts to 117 simplify debugging, can be done using the following commands:: 118 119 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- menuconfig 120 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- bzImage modules scripts_gdb 121 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- \ 122 -C tools/testing/selftests \ 123 TARGETS=bpf \ 124 INSTALL_PATH=$PWD/tools/testing/selftests/kselftest_install \ 125 install 126 127 4. Running the test suite 128 ************************* 129 130 The virtual machine can be started as follows:: 131 132 qemu-system-s390x \ 133 -cpu max,zpci=on \ 134 -smp 2 \ 135 -m 4G \ 136 -kernel linux/arch/s390/boot/compressed/vmlinux \ 137 -drive file=./s390.img,if=virtio,format=raw \ 138 -nographic \ 139 -append 'root=/dev/vda rw console=ttyS1' \ 140 -virtfs local,path=./linux,security_model=none,mount_tag=linux \ 141 -object rng-random,filename=/dev/urandom,id=rng0 \ 142 -device virtio-rng-ccw,rng=rng0 \ 143 -netdev user,id=net0 \ 144 -device virtio-net-ccw,netdev=net0 145 146 When using this on a real IBM Z, ``-enable-kvm`` may be added for better 147 performance. When starting the virtual machine for the first time, disk image 148 setup must be finalized using the following command:: 149 150 /debootstrap/debootstrap --second-stage 151 152 Directory with the code built on the host as well as ``/proc`` and ``/sys`` 153 need to be mounted as follows:: 154 155 mkdir -p /linux 156 mount -t 9p linux /linux 157 mount -t proc proc /proc 158 mount -t sysfs sys /sys 159 160 After that, the test suite can be run using the following commands:: 161 162 cd /linux/tools/testing/selftests/kselftest_install 163 ./run_kselftest.sh 164 165 As usual, tests can be also run individually:: 166 167 cd /linux/tools/testing/selftests/bpf 168 ./test_verifier 169 170 5. Debugging 171 ************ 172 173 It is possible to debug the s390 kernel using QEMU GDB stub, which is activated 174 by passing ``-s`` to QEMU. 175 176 It is preferable to turn KASLR off, so that gdb would know where to find the 177 kernel image in memory, by building the kernel with:: 178 179 RANDOMIZE_BASE=n 180 181 GDB can then be attached using the following command:: 182 183 gdb-multiarch -ex 'target remote localhost:1234' ./vmlinux 184 185 6. Network 186 ********** 187 188 In case one needs to use the network in the virtual machine in order to e.g. 189 install additional packages, it can be configured using:: 190 191 dhclient eth0 192 193 7. Links 194 ******** 195 196 This document is a compilation of techniques, whose more comprehensive 197 descriptions can be found by following these links: 198 199 - `Debootstrap <https://wiki.debian.org/EmDebian/CrossDebootstrap>`_ 200 - `Multiarch <https://wiki.debian.org/Multiarch/HOWTO>`_ 201 - `Building LLVM <https://llvm.org/docs/CMake.html>`_ 202 - `Cross-compiling the kernel <https://wiki.gentoo.org/wiki/Embedded_Handbook/General/Cross-compiling_the_kernel>`_ 203 - `QEMU s390x Guest Support <https://wiki.qemu.org/Documentation/Platforms/S390X>`_ 204 - `Plan 9 folder sharing over Virtio <https://wiki.qemu.org/Documentation/9psetup>`_ 205 - `Using GDB with QEMU <https://wiki.osdev.org/Kernel_Debugging#Use_GDB_with_QEMU>`_
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.