1 ==================== 1 ====================
2 TCM Userspace Design 2 TCM Userspace Design
3 ==================== 3 ====================
4 4
5 5
6 .. Contents: 6 .. Contents:
7 7
8 1) Design 8 1) Design
9 a) Background 9 a) Background
10 b) Benefits 10 b) Benefits
11 c) Design constraints 11 c) Design constraints
12 d) Implementation overview 12 d) Implementation overview
13 i. Mailbox 13 i. Mailbox
14 ii. Command ring 14 ii. Command ring
15 iii. Data Area 15 iii. Data Area
16 e) Device discovery 16 e) Device discovery
17 f) Device events 17 f) Device events
18 g) Other contingencies 18 g) Other contingencies
19 2) Writing a user pass-through handler 19 2) Writing a user pass-through handler
20 a) Discovering and configuring TCMU uio d 20 a) Discovering and configuring TCMU uio devices
21 b) Waiting for events on the device(s) 21 b) Waiting for events on the device(s)
22 c) Managing the command ring 22 c) Managing the command ring
23 3) A final note 23 3) A final note
24 24
25 25
26 Design 26 Design
27 ====== 27 ======
28 28
29 TCM is another name for LIO, an in-kernel iSCS 29 TCM is another name for LIO, an in-kernel iSCSI target (server).
30 Existing TCM targets run in the kernel. TCMU 30 Existing TCM targets run in the kernel. TCMU (TCM in Userspace)
31 allows userspace programs to be written which 31 allows userspace programs to be written which act as iSCSI targets.
32 This document describes the design. 32 This document describes the design.
33 33
34 The existing kernel provides modules for diffe 34 The existing kernel provides modules for different SCSI transport
35 protocols. TCM also modularizes the data stor 35 protocols. TCM also modularizes the data storage. There are existing
36 modules for file, block device, RAM or using a 36 modules for file, block device, RAM or using another SCSI device as
37 storage. These are called "backstores" or "st 37 storage. These are called "backstores" or "storage engines". These
38 built-in modules are implemented entirely as k 38 built-in modules are implemented entirely as kernel code.
39 39
40 Background 40 Background
41 ---------- 41 ----------
42 42
43 In addition to modularizing the transport prot 43 In addition to modularizing the transport protocol used for carrying
44 SCSI commands ("fabrics"), the Linux kernel ta 44 SCSI commands ("fabrics"), the Linux kernel target, LIO, also modularizes
45 the actual data storage as well. These are ref 45 the actual data storage as well. These are referred to as "backstores"
46 or "storage engines". The target comes with ba 46 or "storage engines". The target comes with backstores that allow a
47 file, a block device, RAM, or another SCSI dev 47 file, a block device, RAM, or another SCSI device to be used for the
48 local storage needed for the exported SCSI LUN 48 local storage needed for the exported SCSI LUN. Like the rest of LIO,
49 these are implemented entirely as kernel code. 49 these are implemented entirely as kernel code.
50 50
51 These backstores cover the most common use cas 51 These backstores cover the most common use cases, but not all. One new
52 use case that other non-kernel target solution 52 use case that other non-kernel target solutions, such as tgt, are able
53 to support is using Gluster's GLFS or Ceph's R 53 to support is using Gluster's GLFS or Ceph's RBD as a backstore. The
54 target then serves as a translator, allowing i 54 target then serves as a translator, allowing initiators to store data
55 in these non-traditional networked storage sys 55 in these non-traditional networked storage systems, while still only
56 using standard protocols themselves. 56 using standard protocols themselves.
57 57
58 If the target is a userspace process, supporti 58 If the target is a userspace process, supporting these is easy. tgt,
59 for example, needs only a small adapter module 59 for example, needs only a small adapter module for each, because the
60 modules just use the available userspace libra 60 modules just use the available userspace libraries for RBD and GLFS.
61 61
62 Adding support for these backstores in LIO is 62 Adding support for these backstores in LIO is considerably more
63 difficult, because LIO is entirely kernel code 63 difficult, because LIO is entirely kernel code. Instead of undertaking
64 the significant work to port the GLFS or RBD A 64 the significant work to port the GLFS or RBD APIs and protocols to the
65 kernel, another approach is to create a usersp 65 kernel, another approach is to create a userspace pass-through
66 backstore for LIO, "TCMU". 66 backstore for LIO, "TCMU".
67 67
68 68
69 Benefits 69 Benefits
70 -------- 70 --------
71 71
72 In addition to allowing relatively easy suppor 72 In addition to allowing relatively easy support for RBD and GLFS, TCMU
73 will also allow easier development of new back 73 will also allow easier development of new backstores. TCMU combines
74 with the LIO loopback fabric to become somethi 74 with the LIO loopback fabric to become something similar to FUSE
75 (Filesystem in Userspace), but at the SCSI lay 75 (Filesystem in Userspace), but at the SCSI layer instead of the
76 filesystem layer. A SUSE, if you will. 76 filesystem layer. A SUSE, if you will.
77 77
78 The disadvantage is there are more distinct co 78 The disadvantage is there are more distinct components to configure, and
79 potentially to malfunction. This is unavoidabl 79 potentially to malfunction. This is unavoidable, but hopefully not
80 fatal if we're careful to keep things as simpl 80 fatal if we're careful to keep things as simple as possible.
81 81
82 Design constraints 82 Design constraints
83 ------------------ 83 ------------------
84 84
85 - Good performance: high throughput, low laten 85 - Good performance: high throughput, low latency
86 - Cleanly handle if userspace: 86 - Cleanly handle if userspace:
87 87
88 1) never attaches 88 1) never attaches
89 2) hangs 89 2) hangs
90 3) dies 90 3) dies
91 4) misbehaves 91 4) misbehaves
92 92
93 - Allow future flexibility in user & kernel im 93 - Allow future flexibility in user & kernel implementations
94 - Be reasonably memory-efficient 94 - Be reasonably memory-efficient
95 - Simple to configure & run 95 - Simple to configure & run
96 - Simple to write a userspace backend 96 - Simple to write a userspace backend
97 97
98 98
99 Implementation overview 99 Implementation overview
100 ----------------------- 100 -----------------------
101 101
102 The core of the TCMU interface is a memory reg 102 The core of the TCMU interface is a memory region that is shared
103 between kernel and userspace. Within this regi 103 between kernel and userspace. Within this region is: a control area
104 (mailbox); a lockless producer/consumer circul 104 (mailbox); a lockless producer/consumer circular buffer for commands
105 to be passed up, and status returned; and an i 105 to be passed up, and status returned; and an in/out data buffer area.
106 106
107 TCMU uses the pre-existing UIO subsystem. UIO 107 TCMU uses the pre-existing UIO subsystem. UIO allows device driver
108 development in userspace, and this is conceptu 108 development in userspace, and this is conceptually very close to the
109 TCMU use case, except instead of a physical de 109 TCMU use case, except instead of a physical device, TCMU implements a
110 memory-mapped layout designed for SCSI command 110 memory-mapped layout designed for SCSI commands. Using UIO also
111 benefits TCMU by handling device introspection 111 benefits TCMU by handling device introspection (e.g. a way for
112 userspace to determine how large the shared re 112 userspace to determine how large the shared region is) and signaling
113 mechanisms in both directions. 113 mechanisms in both directions.
114 114
115 There are no embedded pointers in the memory r 115 There are no embedded pointers in the memory region. Everything is
116 expressed as an offset from the region's start 116 expressed as an offset from the region's starting address. This allows
117 the ring to still work if the user process die 117 the ring to still work if the user process dies and is restarted with
118 the region mapped at a different virtual addre 118 the region mapped at a different virtual address.
119 119
120 See target_core_user.h for the struct definiti 120 See target_core_user.h for the struct definitions.
121 121
122 The Mailbox 122 The Mailbox
123 ----------- 123 -----------
124 124
125 The mailbox is always at the start of the shar 125 The mailbox is always at the start of the shared memory region, and
126 contains a version, details about the starting 126 contains a version, details about the starting offset and size of the
127 command ring, and head and tail pointers to be 127 command ring, and head and tail pointers to be used by the kernel and
128 userspace (respectively) to put commands on th 128 userspace (respectively) to put commands on the ring, and indicate
129 when the commands are completed. 129 when the commands are completed.
130 130
131 version - 1 (userspace should abort if otherwi 131 version - 1 (userspace should abort if otherwise)
132 132
133 flags: 133 flags:
134 - TCMU_MAILBOX_FLAG_CAP_OOOC: 134 - TCMU_MAILBOX_FLAG_CAP_OOOC:
135 indicates out-of-order completion is s 135 indicates out-of-order completion is supported.
136 See "The Command Ring" for details. 136 See "The Command Ring" for details.
137 137
138 cmdr_off 138 cmdr_off
139 The offset of the start of the command 139 The offset of the start of the command ring from the start
140 of the memory region, to account for t 140 of the memory region, to account for the mailbox size.
141 cmdr_size 141 cmdr_size
142 The size of the command ring. This doe 142 The size of the command ring. This does *not* need to be a
143 power of two. 143 power of two.
144 cmd_head 144 cmd_head
145 Modified by the kernel to indicate whe 145 Modified by the kernel to indicate when a command has been
146 placed on the ring. 146 placed on the ring.
147 cmd_tail 147 cmd_tail
148 Modified by userspace to indicate when 148 Modified by userspace to indicate when it has completed
149 processing of a command. 149 processing of a command.
150 150
151 The Command Ring 151 The Command Ring
152 ---------------- 152 ----------------
153 153
154 Commands are placed on the ring by the kernel 154 Commands are placed on the ring by the kernel incrementing
155 mailbox.cmd_head by the size of the command, m 155 mailbox.cmd_head by the size of the command, modulo cmdr_size, and
156 then signaling userspace via uio_event_notify( 156 then signaling userspace via uio_event_notify(). Once the command is
157 completed, userspace updates mailbox.cmd_tail 157 completed, userspace updates mailbox.cmd_tail in the same way and
158 signals the kernel via a 4-byte write(). When 158 signals the kernel via a 4-byte write(). When cmd_head equals
159 cmd_tail, the ring is empty -- no commands are 159 cmd_tail, the ring is empty -- no commands are currently waiting to be
160 processed by userspace. 160 processed by userspace.
161 161
162 TCMU commands are 8-byte aligned. They start w 162 TCMU commands are 8-byte aligned. They start with a common header
163 containing "len_op", a 32-bit value that store 163 containing "len_op", a 32-bit value that stores the length, as well as
164 the opcode in the lowest unused bits. It also 164 the opcode in the lowest unused bits. It also contains cmd_id and
165 flags fields for setting by the kernel (kflags 165 flags fields for setting by the kernel (kflags) and userspace
166 (uflags). 166 (uflags).
167 167
168 Currently only two opcodes are defined, TCMU_O 168 Currently only two opcodes are defined, TCMU_OP_CMD and TCMU_OP_PAD.
169 169
170 When the opcode is CMD, the entry in the comma 170 When the opcode is CMD, the entry in the command ring is a struct
171 tcmu_cmd_entry. Userspace finds the SCSI CDB ( 171 tcmu_cmd_entry. Userspace finds the SCSI CDB (Command Data Block) via
172 tcmu_cmd_entry.req.cdb_off. This is an offset 172 tcmu_cmd_entry.req.cdb_off. This is an offset from the start of the
173 overall shared memory region, not the entry. T 173 overall shared memory region, not the entry. The data in/out buffers
174 are accessible via the req.iov[] array. iov_cn !! 174 are accessible via tht req.iov[] array. iov_cnt contains the number of
175 entries in iov[] needed to describe either the 175 entries in iov[] needed to describe either the Data-In or Data-Out
176 buffers. For bidirectional commands, iov_cnt s 176 buffers. For bidirectional commands, iov_cnt specifies how many iovec
177 entries cover the Data-Out area, and iov_bidi_ 177 entries cover the Data-Out area, and iov_bidi_cnt specifies how many
178 iovec entries immediately after that in iov[] 178 iovec entries immediately after that in iov[] cover the Data-In
179 area. Just like other fields, iov.iov_base is 179 area. Just like other fields, iov.iov_base is an offset from the start
180 of the region. 180 of the region.
181 181
182 When completing a command, userspace sets rsp. 182 When completing a command, userspace sets rsp.scsi_status, and
183 rsp.sense_buffer if necessary. Userspace then 183 rsp.sense_buffer if necessary. Userspace then increments
184 mailbox.cmd_tail by entry.hdr.length (mod cmdr 184 mailbox.cmd_tail by entry.hdr.length (mod cmdr_size) and signals the
185 kernel via the UIO method, a 4-byte write to t 185 kernel via the UIO method, a 4-byte write to the file descriptor.
186 186
187 If TCMU_MAILBOX_FLAG_CAP_OOOC is set for mailb 187 If TCMU_MAILBOX_FLAG_CAP_OOOC is set for mailbox->flags, kernel is
188 capable of handling out-of-order completions. 188 capable of handling out-of-order completions. In this case, userspace can
189 handle command in different order other than o 189 handle command in different order other than original. Since kernel would
190 still process the commands in the same order i 190 still process the commands in the same order it appeared in the command
191 ring, userspace need to update the cmd->id whe 191 ring, userspace need to update the cmd->id when completing the
192 command(a.k.a steal the original command's ent 192 command(a.k.a steal the original command's entry).
193 193
194 When the opcode is PAD, userspace only updates 194 When the opcode is PAD, userspace only updates cmd_tail as above --
195 it's a no-op. (The kernel inserts PAD entries 195 it's a no-op. (The kernel inserts PAD entries to ensure each CMD entry
196 is contiguous within the command ring.) 196 is contiguous within the command ring.)
197 197
198 More opcodes may be added in the future. If us 198 More opcodes may be added in the future. If userspace encounters an
199 opcode it does not handle, it must set UNKNOWN 199 opcode it does not handle, it must set UNKNOWN_OP bit (bit 0) in
200 hdr.uflags, update cmd_tail, and proceed with 200 hdr.uflags, update cmd_tail, and proceed with processing additional
201 commands, if any. 201 commands, if any.
202 202
203 The Data Area 203 The Data Area
204 ------------- 204 -------------
205 205
206 This is shared-memory space after the command 206 This is shared-memory space after the command ring. The organization
207 of this area is not defined in the TCMU interf 207 of this area is not defined in the TCMU interface, and userspace
208 should access only the parts referenced by pen 208 should access only the parts referenced by pending iovs.
209 209
210 210
211 Device Discovery 211 Device Discovery
212 ---------------- 212 ----------------
213 213
214 Other devices may be using UIO besides TCMU. U 214 Other devices may be using UIO besides TCMU. Unrelated user processes
215 may also be handling different sets of TCMU de 215 may also be handling different sets of TCMU devices. TCMU userspace
216 processes must find their devices by scanning 216 processes must find their devices by scanning sysfs
217 class/uio/uio*/name. For TCMU devices, these n 217 class/uio/uio*/name. For TCMU devices, these names will be of the
218 format:: 218 format::
219 219
220 tcm-user/<hba_num>/<device_name>/<subt 220 tcm-user/<hba_num>/<device_name>/<subtype>/<path>
221 221
222 where "tcm-user" is common for all TCMU-backed 222 where "tcm-user" is common for all TCMU-backed UIO devices. <hba_num>
223 and <device_name> allow userspace to find the 223 and <device_name> allow userspace to find the device's path in the
224 kernel target's configfs tree. Assuming the us 224 kernel target's configfs tree. Assuming the usual mount point, it is
225 found at:: 225 found at::
226 226
227 /sys/kernel/config/target/core/user_<h 227 /sys/kernel/config/target/core/user_<hba_num>/<device_name>
228 228
229 This location contains attributes such as "hw_ 229 This location contains attributes such as "hw_block_size", that
230 userspace needs to know for correct operation. 230 userspace needs to know for correct operation.
231 231
232 <subtype> will be a userspace-process-unique s 232 <subtype> will be a userspace-process-unique string to identify the
233 TCMU device as expecting to be backed by a cer 233 TCMU device as expecting to be backed by a certain handler, and <path>
234 will be an additional handler-specific string 234 will be an additional handler-specific string for the user process to
235 configure the device, if needed. The name cann 235 configure the device, if needed. The name cannot contain ':', due to
236 LIO limitations. 236 LIO limitations.
237 237
238 For all devices so discovered, the user handle 238 For all devices so discovered, the user handler opens /dev/uioX and
239 calls mmap():: 239 calls mmap()::
240 240
241 mmap(NULL, size, PROT_READ|PROT_WRITE, 241 mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0)
242 242
243 where size must be equal to the value read fro 243 where size must be equal to the value read from
244 /sys/class/uio/uioX/maps/map0/size. 244 /sys/class/uio/uioX/maps/map0/size.
245 245
246 246
247 Device Events 247 Device Events
248 ------------- 248 -------------
249 249
250 If a new device is added or removed, a notific 250 If a new device is added or removed, a notification will be broadcast
251 over netlink, using a generic netlink family n 251 over netlink, using a generic netlink family name of "TCM-USER" and a
252 multicast group named "config". This will incl 252 multicast group named "config". This will include the UIO name as
253 described in the previous section, as well as 253 described in the previous section, as well as the UIO minor
254 number. This should allow userspace to identif 254 number. This should allow userspace to identify both the UIO device and
255 the LIO device, so that after determining the 255 the LIO device, so that after determining the device is supported
256 (based on subtype) it can take the appropriate 256 (based on subtype) it can take the appropriate action.
257 257
258 258
259 Other contingencies 259 Other contingencies
260 ------------------- 260 -------------------
261 261
262 Userspace handler process never attaches: 262 Userspace handler process never attaches:
263 263
264 - TCMU will post commands, and then abort them 264 - TCMU will post commands, and then abort them after a timeout period
265 (30 seconds.) 265 (30 seconds.)
266 266
267 Userspace handler process is killed: 267 Userspace handler process is killed:
268 268
269 - It is still possible to restart and re-conne 269 - It is still possible to restart and re-connect to TCMU
270 devices. Command ring is preserved. However, 270 devices. Command ring is preserved. However, after the timeout period,
271 the kernel will abort pending tasks. 271 the kernel will abort pending tasks.
272 272
273 Userspace handler process hangs: 273 Userspace handler process hangs:
274 274
275 - The kernel will abort pending tasks after a 275 - The kernel will abort pending tasks after a timeout period.
276 276
277 Userspace handler process is malicious: 277 Userspace handler process is malicious:
278 278
279 - The process can trivially break the handling 279 - The process can trivially break the handling of devices it controls,
280 but should not be able to access kernel memo 280 but should not be able to access kernel memory outside its shared
281 memory areas. 281 memory areas.
282 282
283 283
284 Writing a user pass-through handler (with exam 284 Writing a user pass-through handler (with example code)
285 ============================================== 285 =======================================================
286 286
287 A user process handing a TCMU device must supp 287 A user process handing a TCMU device must support the following:
288 288
289 a) Discovering and configuring TCMU uio device 289 a) Discovering and configuring TCMU uio devices
290 b) Waiting for events on the device(s) 290 b) Waiting for events on the device(s)
291 c) Managing the command ring: Parsing operatio 291 c) Managing the command ring: Parsing operations and commands,
292 performing work as needed, setting response 292 performing work as needed, setting response fields (scsi_status and
293 possibly sense_buffer), updating cmd_tail, 293 possibly sense_buffer), updating cmd_tail, and notifying the kernel
294 that work has been finished 294 that work has been finished
295 295
296 First, consider instead writing a plugin for t 296 First, consider instead writing a plugin for tcmu-runner. tcmu-runner
297 implements all of this, and provides a higher- 297 implements all of this, and provides a higher-level API for plugin
298 authors. 298 authors.
299 299
300 TCMU is designed so that multiple unrelated pr 300 TCMU is designed so that multiple unrelated processes can manage TCMU
301 devices separately. All handlers should make s 301 devices separately. All handlers should make sure to only open their
302 devices, based opon a known subtype string. 302 devices, based opon a known subtype string.
303 303
304 a) Discovering and configuring TCMU UIO device 304 a) Discovering and configuring TCMU UIO devices::
305 305
306 /* error checking omitted for brevity */ 306 /* error checking omitted for brevity */
307 307
308 int fd, dev_fd; 308 int fd, dev_fd;
309 char buf[256]; 309 char buf[256];
310 unsigned long long map_len; 310 unsigned long long map_len;
311 void *map; 311 void *map;
312 312
313 fd = open("/sys/class/uio/uio0/name", O_ 313 fd = open("/sys/class/uio/uio0/name", O_RDONLY);
314 ret = read(fd, buf, sizeof(buf)); 314 ret = read(fd, buf, sizeof(buf));
315 close(fd); 315 close(fd);
316 buf[ret-1] = '\0'; /* null-terminate and 316 buf[ret-1] = '\0'; /* null-terminate and chop off the \n */
317 317
318 /* we only want uio devices whose name i 318 /* we only want uio devices whose name is a format we expect */
319 if (strncmp(buf, "tcm-user", 8)) 319 if (strncmp(buf, "tcm-user", 8))
320 exit(-1); 320 exit(-1);
321 321
322 /* Further checking for subtype also nee 322 /* Further checking for subtype also needed here */
323 323
324 fd = open(/sys/class/uio/%s/maps/map0/si 324 fd = open(/sys/class/uio/%s/maps/map0/size, O_RDONLY);
325 ret = read(fd, buf, sizeof(buf)); 325 ret = read(fd, buf, sizeof(buf));
326 close(fd); 326 close(fd);
327 str_buf[ret-1] = '\0'; /* null-terminate 327 str_buf[ret-1] = '\0'; /* null-terminate and chop off the \n */
328 328
329 map_len = strtoull(buf, NULL, 0); 329 map_len = strtoull(buf, NULL, 0);
330 330
331 dev_fd = open("/dev/uio0", O_RDWR); 331 dev_fd = open("/dev/uio0", O_RDWR);
332 map = mmap(NULL, map_len, PROT_READ|PROT 332 map = mmap(NULL, map_len, PROT_READ|PROT_WRITE, MAP_SHARED, dev_fd, 0);
333 333
334 334
335 b) Waiting for events on the device(s) 335 b) Waiting for events on the device(s)
336 336
337 while (1) { 337 while (1) {
338 char buf[4]; 338 char buf[4];
339 339
340 int ret = read(dev_fd, buf, 4); /* wil 340 int ret = read(dev_fd, buf, 4); /* will block */
341 341
342 handle_device_events(dev_fd, map); 342 handle_device_events(dev_fd, map);
343 } 343 }
344 344
345 345
346 c) Managing the command ring:: 346 c) Managing the command ring::
347 347
348 #include <linux/target_core_user.h> 348 #include <linux/target_core_user.h>
349 349
350 int handle_device_events(int fd, void *m 350 int handle_device_events(int fd, void *map)
351 { 351 {
352 struct tcmu_mailbox *mb = map; 352 struct tcmu_mailbox *mb = map;
353 struct tcmu_cmd_entry *ent = (void *) 353 struct tcmu_cmd_entry *ent = (void *) mb + mb->cmdr_off + mb->cmd_tail;
354 int did_some_work = 0; 354 int did_some_work = 0;
355 355
356 /* Process events from cmd ring until 356 /* Process events from cmd ring until we catch up with cmd_head */
357 while (ent != (void *)mb + mb->cmdr_of 357 while (ent != (void *)mb + mb->cmdr_off + mb->cmd_head) {
358 358
359 if (tcmu_hdr_get_op(ent->hdr.len_op) 359 if (tcmu_hdr_get_op(ent->hdr.len_op) == TCMU_OP_CMD) {
360 uint8_t *cdb = (void *)mb + ent->r 360 uint8_t *cdb = (void *)mb + ent->req.cdb_off;
361 bool success = true; 361 bool success = true;
362 362
363 /* Handle command here. */ 363 /* Handle command here. */
364 printf("SCSI opcode: 0x%x\n", cdb[ 364 printf("SCSI opcode: 0x%x\n", cdb[0]);
365 365
366 /* Set response fields */ 366 /* Set response fields */
367 if (success) 367 if (success)
368 ent->rsp.scsi_status = SCSI_NO_S 368 ent->rsp.scsi_status = SCSI_NO_SENSE;
369 else { 369 else {
370 /* Also fill in rsp->sense_buffe 370 /* Also fill in rsp->sense_buffer here */
371 ent->rsp.scsi_status = SCSI_CHEC 371 ent->rsp.scsi_status = SCSI_CHECK_CONDITION;
372 } 372 }
373 } 373 }
374 else if (tcmu_hdr_get_op(ent->hdr.le 374 else if (tcmu_hdr_get_op(ent->hdr.len_op) != TCMU_OP_PAD) {
375 /* Tell the kernel we didn't handl 375 /* Tell the kernel we didn't handle unknown opcodes */
376 ent->hdr.uflags |= TCMU_UFLAG_UNKN 376 ent->hdr.uflags |= TCMU_UFLAG_UNKNOWN_OP;
377 } 377 }
378 else { 378 else {
379 /* Do nothing for PAD entries exce 379 /* Do nothing for PAD entries except update cmd_tail */
380 } 380 }
381 381
382 /* update cmd_tail */ 382 /* update cmd_tail */
383 mb->cmd_tail = (mb->cmd_tail + tcmu_ 383 mb->cmd_tail = (mb->cmd_tail + tcmu_hdr_get_len(&ent->hdr)) % mb->cmdr_size;
384 ent = (void *) mb + mb->cmdr_off + m 384 ent = (void *) mb + mb->cmdr_off + mb->cmd_tail;
385 did_some_work = 1; 385 did_some_work = 1;
386 } 386 }
387 387
388 /* Notify the kernel that work has bee 388 /* Notify the kernel that work has been finished */
389 if (did_some_work) { 389 if (did_some_work) {
390 uint32_t buf = 0; 390 uint32_t buf = 0;
391 391
392 write(fd, &buf, 4); 392 write(fd, &buf, 4);
393 } 393 }
394 394
395 return 0; 395 return 0;
396 } 396 }
397 397
398 398
399 A final note 399 A final note
400 ============ 400 ============
401 401
402 Please be careful to return codes as defined b 402 Please be careful to return codes as defined by the SCSI
403 specifications. These are different than some 403 specifications. These are different than some values defined in the
404 scsi/scsi.h include file. For example, CHECK C 404 scsi/scsi.h include file. For example, CHECK CONDITION's status code
405 is 2, not 1. 405 is 2, not 1.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.