1 .. SPDX-License-Identifier: GPL-2.0
2
3 =========
4 SAS Layer
5 =========
6
7 The SAS Layer is a management infrastructure w
8 SAS LLDDs. It sits between SCSI Core and SAS
9 layout is as follows: while SCSI Core is conce
10 SAM/SPC issues, and a SAS LLDD+sequencer is co
11 phy/OOB/link management, the SAS layer is conc
12
13 * SAS Phy/Port/HA event management (LLDD
14 SAS Layer processes),
15 * SAS Port management (creation/destruct
16 * SAS Domain discovery and revalidation,
17 * SAS Domain device management,
18 * SCSI Host registration/unregistration,
19 * Device registration with SCSI Core (SA
20 (SATA), and
21 * Expander management and exporting expa
22 to user space.
23
24 A SAS LLDD is a PCI device driver. It is conc
25 phy/OOB management, and vendor specific tasks
26 events to the SAS layer.
27
28 The SAS Layer does most SAS tasks as outlined
29 spec.
30
31 The sas_ha_struct describes the SAS LLDD to th
32 Most of it is used by the SAS Layer but a few
33 be initialized by the LLDDs.
34
35 After initializing your hardware, from the pro
36 you call sas_register_ha(). It will register y
37 the SCSI subsystem, creating a SCSI host and i
38 register your SAS driver with the sysfs SAS tr
39 It will then return. Then you enable your phy
40 start OOB (at which point your driver will sta
41 notify_* event callbacks).
42
43 Structure descriptions
44 ======================
45
46 ``struct sas_phy``
47 ------------------
48
49 Normally this is statically embedded to your d
50 phy structure::
51
52 struct my_phy {
53 blah;
54 struct sas_phy sas_phy;
55 bleh;
56 };
57
58 And then all the phys are an array of my_phy i
59 struct (shown below).
60
61 Then as you go along and initialize your phys
62 initialize the sas_phy struct, along with your
63 phy structure.
64
65 In general, the phys are managed by the LLDD a
66 are managed by the SAS layer. So the phys are
67 and updated by the LLDD and the ports are init
68 updated by the SAS layer.
69
70 There is a scheme where the LLDD can RW certai
71 and the SAS layer can only read such ones, and
72 The idea is to avoid unnecessary locking.
73
74 enabled
75 - must be set (0/1)
76
77 id
78 - must be set [0,MAX_PHYS)]
79
80 class, proto, type, role, oob_mode, linkrate
81 - must be set
82
83 oob_mode
84 - you set this when OOB has finished and t
85 the SAS Layer.
86
87 sas_addr
88 - this normally points to an array holding
89 address of the phy, possibly somewhere i
90 struct.
91
92 attached_sas_addr
93 - set this when you (LLDD) receive an
94 IDENTIFY frame or a FIS frame, _before_
95 layer. The idea is that sometimes the L
96 or provide a different SAS address on th
97 allows it to do this. At best you shoul
98 address from the IDENTIFY frame or maybe
99 address for SATA directly attached devic
100 process may later change this.
101
102 frame_rcvd
103 - this is where you copy the IDENTIFY/FIS
104 when you get it; you lock, copy, set fra
105 unlock the lock, and then call the event
106 since there's no way to know your hw fra
107 so you define the actual array in your p
108 this pointer point to it. You copy the
109 DMAable memory to that area holding the
110
111 sas_prim
112 - this is where primitives go when they're
113 received. See sas.h. Grab the lock, set
114 release the lock, notify.
115
116 port
117 - this points to the sas_port if the phy b
118 to a port -- the LLDD only reads this. I
119 sas_port this phy is part of. Set by th
120
121 ha
122 - may be set; the SAS layer sets it anyway
123
124 lldd_phy
125 - you should set this to point to your phy
126 can find your way around faster when the
127 of your callbacks and passes you a phy.
128 embedded you can also use container_of -
129 prefer.
130
131
132 ``struct sas_port``
133 -------------------
134
135 The LLDD doesn't set any fields of this struct
136 reads them. They should be self explanatory.
137
138 phy_mask is 32 bit, this should be enough for
139 haven't heard of a HA having more than 8 phys.
140
141 lldd_port
142 - I haven't found use for that -- maybe ot
143 LLDD who wish to have internal port repr
144 use of this.
145
146 ``struct sas_ha_struct``
147 ------------------------
148
149 It normally is statically declared in your own
150 structure describing your adapter::
151
152 struct my_sas_ha {
153 blah;
154 struct sas_ha_struct sas_ha;
155 struct my_phy phys[MAX_PHYS];
156 struct sas_port sas_ports[MAX_PHYS]; /
157 bleh;
158 };
159
160 (1) If your LLDD doesn't have its own port
161
162 What needs to be initialized (sample function
163
164 pcidev
165 ^^^^^^
166
167 sas_addr
168 - since the SAS layer doesn't want to m
169 memory allocation, etc, this points t
170 allocated array somewhere (say in you
171 structure) and holds the SAS address
172 adapter as given by you or the manufa
173
174 sas_port
175 ^^^^^^^^
176
177 sas_phy
178 - an array of pointers to structures. (s
179 note above on sas_addr).
180 These must be set. See more notes bel
181
182 num_phys
183 - the number of phys present in the sas
184 and the number of ports present in th
185 array. There can be a maximum num_ph
186 port) so we drop the num_ports, and o
187 num_phys.
188
189 The event interface::
190
191 /* LLDD calls these to notify the clas
192 void sas_notify_port_event(struct sas_
193 void sas_notify_phy_event(struct sas_p
194
195 The port notification::
196
197 /* The class calls these to notify the
198 void (*lldd_port_formed)(struct sas_ph
199 void (*lldd_port_deformed)(struct sas_
200
201 If the LLDD wants notification when a port has
202 or deformed it sets those to a function satisf
203
204 A SAS LLDD should also implement at least one
205 Management Functions (TMFs) described in SAM::
206
207 /* Task Management Functions. Must be
208 int (*lldd_abort_task)(struct sas_task
209 int (*lldd_abort_task_set)(struct doma
210 int (*lldd_clear_task_set)(struct doma
211 int (*lldd_I_T_nexus_reset)(struct dom
212 int (*lldd_lu_reset)(struct domain_dev
213 int (*lldd_query_task)(struct sas_task
214
215 For more information please read SAM from T10.
216
217 Port and Adapter management::
218
219 /* Port and Adapter management */
220 int (*lldd_clear_nexus_port)(struct sa
221 int (*lldd_clear_nexus_ha)(struct sas_
222
223 A SAS LLDD should implement at least one of th
224
225 Phy management::
226
227 /* Phy management */
228 int (*lldd_control_phy)(struct sas_phy
229
230 lldd_ha
231 - set this to point to your HA struct. You
232 use container_of if you embedded it as s
233
234 A sample initialization and registration funct
235 can look like this (called last thing from pro
236 *but* before you enable the phys to do OOB::
237
238 static int register_sas_ha(struct my_sas_h
239 {
240 int i;
241 static struct sas_phy *sas_phys[
242 static struct sas_port *sas_ports
243
244 my_ha->sas_ha.sas_addr = &my_ha->s
245
246 for (i = 0; i < MAX_PHYS; i++) {
247 sas_phys[i] = &my_ha->phys
248 sas_ports[i] = &my_ha->sas
249 }
250
251 my_ha->sas_ha.sas_phy = sas_phys;
252 my_ha->sas_ha.sas_port = sas_ports
253 my_ha->sas_ha.num_phys = MAX_PHYS;
254
255 my_ha->sas_ha.lldd_port_formed = m
256
257 my_ha->sas_ha.lldd_dev_found = my_
258 my_ha->sas_ha.lldd_dev_gone = my_d
259
260 my_ha->sas_ha.lldd_execute_task =
261
262 my_ha->sas_ha.lldd_abort_task
263 my_ha->sas_ha.lldd_abort_task_set
264 my_ha->sas_ha.lldd_clear_task_set
265 my_ha->sas_ha.lldd_I_T_nexus_reset
266 my_ha->sas_ha.lldd_lu_reset
267 my_ha->sas_ha.lldd_query_task
268
269 my_ha->sas_ha.lldd_clear_nexus_por
270 my_ha->sas_ha.lldd_clear_nexus_ha
271
272 my_ha->sas_ha.lldd_control_phy = m
273
274 return sas_register_ha(&my_ha->sas
275 }
276
277 (2) SAS 1.1 does not define I_T Nexus Reset TM
278
279 Events
280 ======
281
282 Events are **the only way** a SAS LLDD notifie
283 of anything. There is no other method or way
284 the SAS layer of anything happening internally
285 domain.
286
287 Phy events::
288
289 PHYE_LOSS_OF_SIGNAL, (C)
290 PHYE_OOB_DONE,
291 PHYE_OOB_ERROR, (C)
292 PHYE_SPINUP_HOLD.
293
294 Port events, passed on a _phy_::
295
296 PORTE_BYTES_DMAED, (M)
297 PORTE_BROADCAST_RCVD, (E)
298 PORTE_LINK_RESET_ERR, (C)
299 PORTE_TIMER_EVENT, (C)
300 PORTE_HARD_RESET.
301
302 Host Adapter event:
303 HAE_RESET
304
305 A SAS LLDD should be able to generate
306
307 - at least one event from group C (cho
308 - events marked M (mandatory) are mand
309 - events marked E (expander) if it wan
310 to handle domain revalidation (only
311 - Unmarked events are optional.
312
313 Meaning:
314
315 HAE_RESET
316 - when your HA got internal error and was
317
318 PORTE_BYTES_DMAED
319 - on receiving an IDENTIFY/FIS frame
320
321 PORTE_BROADCAST_RCVD
322 - on receiving a primitive
323
324 PORTE_LINK_RESET_ERR
325 - timer expired, loss of signal, loss of D
326
327 PORTE_TIMER_EVENT
328 - DWS reset timeout timer expired [1]_
329
330 PORTE_HARD_RESET
331 - Hard Reset primitive received.
332
333 PHYE_LOSS_OF_SIGNAL
334 - the device is gone [1]_
335
336 PHYE_OOB_DONE
337 - OOB went fine and oob_mode is valid
338
339 PHYE_OOB_ERROR
340 - Error while doing OOB, the device probab
341 got disconnected. [1]_
342
343 PHYE_SPINUP_HOLD
344 - SATA is present, COMWAKE not sent.
345
346 .. [1] should set/clear the appropriate fields
347 or alternatively call the inlined sas_p
348 which is just a helper, from their task
349
350 The Execute Command SCSI RPC::
351
352 int (*lldd_execute_task)(struct sas_ta
353
354 Used to queue a task to the SAS LLDD. @task i
355 @gfp_mask is the gfp_mask defining the context
356
357 This function should implement the Execute Com
358
359 That is, when lldd_execute_task() is called, t
360 go out on the transport *immediately*. There
361 queuing of any sort and at any level in a SAS
362
363 Returns:
364
365 * -SAS_QUEUE_FULL, -ENOMEM, nothing was que
366 * 0, the task(s) were queued.
367
368 ::
369
370 struct sas_task {
371 dev -- the device this task is des
372 task_proto -- _one_ of enum sas_pr
373 scatter -- pointer to scatter gath
374 num_scatter -- number of elements
375 total_xfer_len -- total number of
376 data_dir -- PCI_DMA_...
377 task_done -- callback when the tas
378 };
379
380 Discovery
381 =========
382
383 The sysfs tree has the following purposes:
384
385 a) It shows you the physical layout of the
386 the current time, i.e. how the domain l
387 physical world right now.
388 b) Shows some device parameters _at_discov
389
390 This is a link to the tree(1) program, very us
391 viewing the SAS domain:
392 ftp://mama.indstate.edu/linux/tree/
393
394 I expect user space applications to actually c
395 graphical interface of this.
396
397 That is, the sysfs domain tree doesn't show or
398 you e.g., change the meaning of the READY LED
399 setting, but it does show you the current conn
400 of the domain device.
401
402 Keeping internal device state changes is respo
403 upper layers (Command set drivers) and user sp
404
405 When a device or devices are unplugged from th
406 is reflected in the sysfs tree immediately, an
407 removed from the system.
408
409 The structure domain_device describes any devi
410 domain. It is completely managed by the SAS l
411 points to a domain device, this is how the SAS
412 where to send the task(s) to. A SAS LLDD only
413 contents of the domain_device structure, but i
414 or destroys one.
415
416 Expander management from User Space
417 ===================================
418
419 In each expander directory in sysfs, there is
420 "smp_portal". It is a binary sysfs attribute
421 implements an SMP portal (Note: this is *NOT*
422 to which user space applications can send SMP
423 receive SMP responses.
424
425 Functionality is deceptively simple:
426
427 1. Build the SMP frame you want to send. The f
428 is described in the SAS spec. Leave the CR
429
430 open(2)
431
432 2. Open the expander's SMP portal sysfs file i
433
434 write(2)
435
436 3. Write the frame you built in 1.
437
438 read(2)
439
440 4. Read the amount of data you expect to recei
441 If you receive different amount of data you
442 then there was some kind of error.
443
444 close(2)
445
446 All this process is shown in detail in the fun
447 and its callers, in the file "expander_conf.c"
448
449 The kernel functionality is implemented in the
450 "sas_expander.c".
451
452 The program "expander_conf.c" implements this.
453 argument, the sysfs file name of the SMP porta
454 expander, and gives expander information, incl
455 tables.
456
457 The SMP portal gives you complete control of t
458 so please be careful.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.