1 .. SPDX-License-Identifier: GPL-2.0
2 .. _VAS-API:
3
4 ==============================================
5 Virtual Accelerator Switchboard (VAS) userspac
6 ==============================================
7
8 Introduction
9 ============
10
11 Power9 processor introduced Virtual Accelerato
12 allows both userspace and kernel communicate t
13 (hardware accelerator) referred to as the Nest
14 unit comprises of one or more hardware engines
15 such as 842 compression, GZIP compression and
16 userspace applications will have access to onl
17 which supports ZLIB and GZIP compression algor
18
19 To communicate with NX, kernel has to establis
20 then requests can be submitted directly withou
21 Requests to the GZIP engine must be formatted
22 Block (CRB) and these CRBs must be submitted t
23 instructions to paste the CRB to hardware addr
24 the engine's request queue.
25
26 The GZIP engine provides two priority levels o
27 High. Only Normal requests are supported from
28
29 This document explains userspace API that is u
30 kernel to setup channel / window which can be
31 requests directly to NX accelerator.
32
33
34 Overview
35 ========
36
37 Application access to the GZIP engine is provi
38 /dev/crypto/nx-gzip device node implemented by
39 An application must open the /dev/crypto/nx-gz
40 descriptor (fd). Then should issue VAS_TX_WIN_
41 establish connection to the engine. It means s
42 engine for this process. Once a connection is
43 should use the mmap() system call to map the h
44 request queue into the application's virtual a
45
46 The application can then submit one or more re
47 using copy/paste instructions and pasting the
48 (aka paste_address) returned by mmap(). User s
49 established connection or send window by closi
50 (close(fd)) or upon the process exit.
51
52 Note that applications can send several reques
53 can establish multiple windows, but one window
54
55 Following sections provide additional details
56 individual steps.
57
58 NX-GZIP Device Node
59 ===================
60
61 There is one /dev/crypto/nx-gzip node in the s
62 access to all GZIP engines in the system. The
63 /dev/crypto/nx-gzip are:
64
65 * open() the device for read and write
66 * issue VAS_TX_WIN_OPEN ioctl
67 * mmap() the engine's request queue in
68 address space (i.e. get a paste_addr
69 engine).
70 * close the device node.
71
72 Other file operations on this device node are
73
74 Note that the copy and paste operations go dir
75 do not go through this device. Refer COPY/PAST
76 details.
77
78 Although a system may have several instances o
79 engines (typically, one per P9 chip) there is
80 /dev/crypto/nx-gzip device node in the system.
81 node is opened, Kernel opens send window on a
82 accelerator. It finds CPU on which the user pr
83 determine the NX instance for the correspondin
84 belongs.
85
86 Applications may chose a specific instance of
87 the vas_id field in the VAS_TX_WIN_OPEN ioctl
88
89 A userspace library libnxz is available here b
90
91 https://github.com/abalib/power-gzip
92
93 Applications that use inflate / deflate calls
94 instead of libz and use NX GZIP compression wi
95
96 Open /dev/crypto/nx-gzip
97 ========================
98
99 The nx-gzip device should be opened for read a
100 privileges are needed to open the device. Each
101 file descriptor. So if the userspace process n
102 several open calls have to be issued.
103
104 See open(2) system call man pages for other de
105 error codes and restrictions.
106
107 VAS_TX_WIN_OPEN ioctl
108 =====================
109
110 Applications should use the VAS_TX_WIN_OPEN io
111 a connection with NX co-processor engine:
112
113 ::
114
115 struct vas_tx_win_open_attr {
116 __u32 version;
117 __s16 vas_id; /* spe
118
119 __u16 reserved1;
120 __u64 flags; /* For
121 __u64 reserved2[6];
122 };
123
124 version:
125 The version field must be curr
126 vas_id:
127 If '-1' is passed, kernel will
128 to assign an optimal instance
129 select the specific VAS instan
130 "Discovery of available VAS en
131
132 flags, reserved1 and reserved2[6] fiel
133 and must be set to 0.
134
135 The attributes attr for the VAS_TX_WIN
136 follows::
137
138 #define VAS_MAGIC 'v'
139 #define VAS_TX_WIN_OPEN _IOW(V
140
141
142 struct vas_tx_win_open_attr at
143 rc = ioctl(fd, VAS_TX_WIN_OPEN
144
145 The VAS_TX_WIN_OPEN ioctl returns 0 on
146 returns -1 and sets the errno variable
147
148 Error conditions:
149
150 ====== ======================
151 EINVAL fd does not refer to a
152 EINVAL Invalid vas ID
153 EINVAL version is not set wit
154 EEXIST Window is already open
155 ENOMEM Memory is not availabl
156 ENOSPC System has too many ac
157 opened
158 EINVAL reserved fields are no
159 ====== ======================
160
161 See the ioctl(2) man page for more det
162 restrictions.
163
164 mmap() NX-GZIP device
165 =====================
166
167 The mmap() system call for a NX-GZIP device fd
168 that the application can use to copy/paste its
169
170 ::
171
172 paste_addr = mmap(addr, size,
173
174 Only restrictions on mmap for a NX-GZI
175
176 * size should be PAGE_SIZE
177 * offset parameter should be 0
178
179 Refer to mmap(2) man page for addition
180 In addition to the error conditions li
181 page, can also fail with one of the fo
182
183 ====== ======================
184 EINVAL fd is not associated w
185 (i.e mmap() does not f
186 to the VAS_TX_WIN_OPEN
187 EINVAL offset field is not 0U
188 ====== ======================
189
190 Discovery of available VAS engines
191 ==================================
192
193 Each available VAS instance in the system will
194 like /proc/device-tree/vas@* or /proc/device-t
195 Determine the chip or VAS instance and use the
196 property value in this node to select specific
197
198 Copy/Paste operations
199 =====================
200
201 Applications should use the copy and paste ins
202 Refer section 4.4 in PowerISA for Copy/Paste i
203 https://openpowerfoundation.org/?resource_lib=
204
205 CRB Specification and use NX
206 ============================
207
208 Applications should format requests to the co-
209 co-processor Request Block (CRBs). Refer NX-GZ
210 of CRB and use NX from userspace such as sendi
211 request status.
212
213 NX Fault handling
214 =================
215
216 Applications send requests to NX and wait for
217 co-processor Status Block (CSB) flags. NX upda
218 request is processed. Refer NX-GZIP user's man
219 status flags.
220
221 In case if NX encounters translation error (ca
222 address or any request buffer, raises an inter
223 fault. Page fault can happen if an application
224 request buffers are not in memory. The operati
225 updating CSB with the following data::
226
227 csb.flags = CSB_V;
228 csb.cc = CSB_CC_FAULT_ADDRESS;
229 csb.ce = CSB_CE_TERMINATION;
230 csb.address = fault_address;
231
232 When an application receives translation error
233 the page that has a fault address so that this
234 the application can resend this request to NX.
235
236 If the OS can not update CSB due to invalid CS
237 to the process who opened the send window on w
238 issued. This signal returns with the following
239
240 siginfo.si_signo = SIGSEGV;
241 siginfo.si_errno = EFAULT;
242 siginfo.si_code = SEGV_MAPERR;
243 siginfo.si_addr = CSB address;
244
245 In the case of multi-thread applications, NX s
246 across all threads. For example, a child threa
247 but other threads can send requests to NX usin
248 requests will be successful even in the case o
249 as CSB address is valid. If the NX request con
250 the signal will be sent to the child thread th
251 the thread is exited without closing the windo
252 using this window. the signal will be issued t
253 (tgid). It is up to the application whether to
254 signals.
255
256 NX-GZIP User's Manual:
257 https://github.com/libnxz/power-gzip/blob/mast
258
259 Simple example
260 ==============
261
262 ::
263
264 int use_nx_gzip()
265 {
266 int rc, fd;
267 void *addr;
268 struct vas_setup_attr
269
270 fd = open("/dev/crypto
271 if (fd < 0) {
272 fprintf(stderr
273 return -1;
274 }
275 memset(&txattr, 0, siz
276 txattr.version = 1;
277 txattr.vas_id = -1
278 rc = ioctl(fd, VAS_TX_
279 (unsig
280 if (rc < 0) {
281 fprintf(stderr
282
283 return rc;
284 }
285 addr = mmap(NULL, 4096
286 MAP_SH
287 if (addr == MAP_FAILED
288 fprintf(stderr
289
290 return -errno;
291 }
292 do {
293 //Format CRB r
294 //uncompressio
295 // Refer tests
296 vas_copy((&crb
297 vas_paste(addr
298 // Poll on csb
299 // csb address
300 } while (true)
301 close(fd) or window ca
302 }
303
304 Refer https://github.com/libnxz/power-
305 use cases.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.