1 ======================
2 Userspace verbs access
3 ======================
4
5 The ib_uverbs module, built by enabling CONF
6 enables direct userspace access to IB hardwa
7 described in chapter 11 of the InfiniBand Ar
8
9 To use the verbs, the libibverbs library, av
10 https://github.com/linux-rdma/rdma-core, is
11 device-independent API for using the ib_uver
12 libibverbs also requires appropriate device-
13 userspace driver for your InfiniBand hardwar
14 a Mellanox HCA, you will need the ib_mthca k
15 libmthca userspace driver be installed.
16
17 User-kernel communication
18 =========================
19
20 Userspace communicates with the kernel for s
21 management operations via the /dev/infiniban
22 devices. Fast path operations are typically
23 directly to hardware registers mmap()ed into
24 system call or context switch into the kerne
25
26 Commands are sent to the kernel via write()s
27 The ABI is defined in drivers/infiniband/inc
28 The structs for commands that require a resp
29 contain a 64-bit field used to pass a pointe
30 Status is returned to userspace as the retur
31 system call.
32
33 Resource management
34 ===================
35
36 Since creation and destruction of all IB res
37 commands passed through a file descriptor, t
38 of which resources are attached to a given u
39 ib_uverbs module maintains idr tables that a
40 between kernel pointers and opaque userspace
41 pointers are never exposed to userspace and
42 the kernel into following a bogus pointer.
43
44 This also allows the kernel to clean up when
45 prevent one process from touching another pr
46
47 Memory pinning
48 ==============
49
50 Direct userspace I/O requires that memory re
51 I/O targets be kept resident at the same phy
52 ib_uverbs module manages pinning and unpinni
53 get_user_pages() and put_page() calls. It a
54 amount of memory pinned in the process's pin
55 unprivileged processes do not exceed their R
56
57 Pages that are pinned multiple times are cou
58 pinned, so the value of pinned_vm may be an
59 number of pages pinned by a process.
60
61 /dev files
62 ==========
63
64 To create the appropriate character device f
65 udev, a rule like::
66
67 KERNEL=="uverbs*", NAME="infiniband/%k"
68
69 can be used. This will create device nodes
70
71 /dev/infiniband/uverbs0
72
73 and so on. Since the InfiniBand userspace v
74 use by non-privileged processes, it may be u
75 appropriate MODE or GROUP to the udev rule.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.