1 What: /sys/bus/cxl/flush
2 Date: Januarry, 2022
3 KernelVersion: v5.18
4 Contact: linux-cxl@vger.kernel.org
5 Description:
6 (WO) If userspace manually unbinds a port the kernel schedules
7 all descendant memdevs for unbind. Writing '1' to this attribute
8 flushes that work.
9
10
11 What: /sys/bus/cxl/devices/memX/firmware_version
12 Date: December, 2020
13 KernelVersion: v5.12
14 Contact: linux-cxl@vger.kernel.org
15 Description:
16 (RO) "FW Revision" string as reported by the Identify
17 Memory Device Output Payload in the CXL-2.0
18 specification.
19
20
21 What: /sys/bus/cxl/devices/memX/ram/size
22 Date: December, 2020
23 KernelVersion: v5.12
24 Contact: linux-cxl@vger.kernel.org
25 Description:
26 (RO) "Volatile Only Capacity" as bytes. Represents the
27 identically named field in the Identify Memory Device Output
28 Payload in the CXL-2.0 specification.
29
30
31 What: /sys/bus/cxl/devices/memX/ram/qos_class
32 Date: May, 2023
33 KernelVersion: v6.8
34 Contact: linux-cxl@vger.kernel.org
35 Description:
36 (RO) For CXL host platforms that support "QoS Telemmetry"
37 this attribute conveys a comma delimited list of platform
38 specific cookies that identifies a QoS performance class
39 for the volatile partition of the CXL mem device. These
40 class-ids can be compared against a similar "qos_class"
41 published for a root decoder. While it is not required
42 that the endpoints map their local memory-class to a
43 matching platform class, mismatches are not recommended
44 and there are platform specific performance related
45 side-effects that may result. First class-id is displayed.
46
47
48 What: /sys/bus/cxl/devices/memX/pmem/size
49 Date: December, 2020
50 KernelVersion: v5.12
51 Contact: linux-cxl@vger.kernel.org
52 Description:
53 (RO) "Persistent Only Capacity" as bytes. Represents the
54 identically named field in the Identify Memory Device Output
55 Payload in the CXL-2.0 specification.
56
57
58 What: /sys/bus/cxl/devices/memX/pmem/qos_class
59 Date: May, 2023
60 KernelVersion: v6.8
61 Contact: linux-cxl@vger.kernel.org
62 Description:
63 (RO) For CXL host platforms that support "QoS Telemmetry"
64 this attribute conveys a comma delimited list of platform
65 specific cookies that identifies a QoS performance class
66 for the persistent partition of the CXL mem device. These
67 class-ids can be compared against a similar "qos_class"
68 published for a root decoder. While it is not required
69 that the endpoints map their local memory-class to a
70 matching platform class, mismatches are not recommended
71 and there are platform specific performance related
72 side-effects that may result. First class-id is displayed.
73
74
75 What: /sys/bus/cxl/devices/memX/serial
76 Date: January, 2022
77 KernelVersion: v5.18
78 Contact: linux-cxl@vger.kernel.org
79 Description:
80 (RO) 64-bit serial number per the PCIe Device Serial Number
81 capability. Mandatory for CXL devices, see CXL 2.0 8.1.12.2
82 Memory Device PCIe Capabilities and Extended Capabilities.
83
84
85 What: /sys/bus/cxl/devices/memX/numa_node
86 Date: January, 2022
87 KernelVersion: v5.18
88 Contact: linux-cxl@vger.kernel.org
89 Description:
90 (RO) If NUMA is enabled and the platform has affinitized the
91 host PCI device for this memory device, emit the CPU node
92 affinity for this device.
93
94
95 What: /sys/bus/cxl/devices/memX/security/state
96 Date: June, 2023
97 KernelVersion: v6.5
98 Contact: linux-cxl@vger.kernel.org
99 Description:
100 (RO) Reading this file will display the CXL security state for
101 that device. Such states can be: 'disabled', 'sanitize', when
102 a sanitization is currently underway; or those available only
103 for persistent memory: 'locked', 'unlocked' or 'frozen'. This
104 sysfs entry is select/poll capable from userspace to notify
105 upon completion of a sanitize operation.
106
107
108 What: /sys/bus/cxl/devices/memX/security/sanitize
109 Date: June, 2023
110 KernelVersion: v6.5
111 Contact: linux-cxl@vger.kernel.org
112 Description:
113 (WO) Write a boolean 'true' string value to this attribute to
114 sanitize the device to securely re-purpose or decommission it.
115 This is done by ensuring that all user data and meta-data,
116 whether it resides in persistent capacity, volatile capacity,
117 or the LSA, is made permanently unavailable by whatever means
118 is appropriate for the media type. This functionality requires
119 the device to be disabled, that is, not actively decoding any
120 HPA ranges. This permits avoiding explicit global CPU cache
121 management, relying instead for it to be done when a region
122 transitions between software programmed and hardware committed
123 states. If this file is not present, then there is no hardware
124 support for the operation.
125
126
127 What /sys/bus/cxl/devices/memX/security/erase
128 Date: June, 2023
129 KernelVersion: v6.5
130 Contact: linux-cxl@vger.kernel.org
131 Description:
132 (WO) Write a boolean 'true' string value to this attribute to
133 secure erase user data by changing the media encryption keys for
134 all user data areas of the device. This functionality requires
135 the device to be disabled, that is, not actively decoding any
136 HPA ranges. This permits avoiding explicit global CPU cache
137 management, relying instead for it to be done when a region
138 transitions between software programmed and hardware committed
139 states. If this file is not present, then there is no hardware
140 support for the operation.
141
142
143 What: /sys/bus/cxl/devices/memX/firmware/
144 Date: April, 2023
145 KernelVersion: v6.5
146 Contact: linux-cxl@vger.kernel.org
147 Description:
148 (RW) Firmware uploader mechanism. The different files under
149 this directory can be used to upload and activate new
150 firmware for CXL devices. The interfaces under this are
151 documented in sysfs-class-firmware.
152
153
154 What: /sys/bus/cxl/devices/*/devtype
155 Date: June, 2021
156 KernelVersion: v5.14
157 Contact: linux-cxl@vger.kernel.org
158 Description:
159 (RO) CXL device objects export the devtype attribute which
160 mirrors the same value communicated in the DEVTYPE environment
161 variable for uevents for devices on the "cxl" bus.
162
163
164 What: /sys/bus/cxl/devices/*/modalias
165 Date: December, 2021
166 KernelVersion: v5.18
167 Contact: linux-cxl@vger.kernel.org
168 Description:
169 (RO) CXL device objects export the modalias attribute which
170 mirrors the same value communicated in the MODALIAS environment
171 variable for uevents for devices on the "cxl" bus.
172
173
174 What: /sys/bus/cxl/devices/portX/uport
175 Date: June, 2021
176 KernelVersion: v5.14
177 Contact: linux-cxl@vger.kernel.org
178 Description:
179 (RO) CXL port objects are enumerated from either a platform
180 firmware device (ACPI0017 and ACPI0016) or PCIe switch upstream
181 port with CXL component registers. The 'uport' symlink connects
182 the CXL portX object to the device that published the CXL port
183 capability.
184
185
186 What: /sys/bus/cxl/devices/{port,endpoint}X/parent_dport
187 Date: January, 2023
188 KernelVersion: v6.3
189 Contact: linux-cxl@vger.kernel.org
190 Description:
191 (RO) CXL port objects are instantiated for each upstream port in
192 a CXL/PCIe switch, and for each endpoint to map the
193 corresponding memory device into the CXL port hierarchy. When a
194 descendant CXL port (switch or endpoint) is enumerated it is
195 useful to know which 'dport' object in the parent CXL port
196 routes to this descendant. The 'parent_dport' symlink points to
197 the device representing the downstream port of a CXL switch that
198 routes to {port,endpoint}X.
199
200
201 What: /sys/bus/cxl/devices/portX/dportY
202 Date: June, 2021
203 KernelVersion: v5.14
204 Contact: linux-cxl@vger.kernel.org
205 Description:
206 (RO) CXL port objects are enumerated from either a platform
207 firmware device (ACPI0017 and ACPI0016) or PCIe switch upstream
208 port with CXL component registers. The 'dportY' symlink
209 identifies one or more downstream ports that the upstream port
210 may target in its decode of CXL memory resources. The 'Y'
211 integer reflects the hardware port unique-id used in the
212 hardware decoder target list.
213
214
215 What: /sys/bus/cxl/devices/portX/decoders_committed
216 Date: October, 2023
217 KernelVersion: v6.7
218 Contact: linux-cxl@vger.kernel.org
219 Description:
220 (RO) A memory device is considered active when any of its
221 decoders are in the "committed" state (See CXL 3.0 8.2.4.19.7
222 CXL HDM Decoder n Control Register). Hotplug and destructive
223 operations like "sanitize" are blocked while device is actively
224 decoding a Host Physical Address range. Note that this number
225 may be elevated without any regionX objects active or even
226 enumerated, as this may be due to decoders established by
227 platform firwmare or a previous kernel (kexec).
228
229
230 What: /sys/bus/cxl/devices/decoderX.Y
231 Date: June, 2021
232 KernelVersion: v5.14
233 Contact: linux-cxl@vger.kernel.org
234 Description:
235 (RO) CXL decoder objects are enumerated from either a platform
236 firmware description, or a CXL HDM decoder register set in a
237 PCIe device (see CXL 2.0 section 8.2.5.12 CXL HDM Decoder
238 Capability Structure). The 'X' in decoderX.Y represents the
239 cxl_port container of this decoder, and 'Y' represents the
240 instance id of a given decoder resource.
241
242
243 What: /sys/bus/cxl/devices/decoderX.Y/{start,size}
244 Date: June, 2021
245 KernelVersion: v5.14
246 Contact: linux-cxl@vger.kernel.org
247 Description:
248 (RO) The 'start' and 'size' attributes together convey the
249 physical address base and number of bytes mapped in the
250 decoder's decode window. For decoders of devtype
251 "cxl_decoder_root" the address range is fixed. For decoders of
252 devtype "cxl_decoder_switch" the address is bounded by the
253 decode range of the cxl_port ancestor of the decoder's cxl_port,
254 and dynamically updates based on the active memory regions in
255 that address space.
256
257
258 What: /sys/bus/cxl/devices/decoderX.Y/locked
259 Date: June, 2021
260 KernelVersion: v5.14
261 Contact: linux-cxl@vger.kernel.org
262 Description:
263 (RO) CXL HDM decoders have the capability to lock the
264 configuration until the next device reset. For decoders of
265 devtype "cxl_decoder_root" there is no standard facility to
266 unlock them. For decoders of devtype "cxl_decoder_switch" a
267 secondary bus reset, of the PCIe bridge that provides the bus
268 for this decoders uport, unlocks / resets the decoder.
269
270
271 What: /sys/bus/cxl/devices/decoderX.Y/target_list
272 Date: June, 2021
273 KernelVersion: v5.14
274 Contact: linux-cxl@vger.kernel.org
275 Description:
276 (RO) Display a comma separated list of the current decoder
277 target configuration. The list is ordered by the current
278 configured interleave order of the decoder's dport instances.
279 Each entry in the list is a dport id.
280
281
282 What: /sys/bus/cxl/devices/decoderX.Y/cap_{pmem,ram,type2,type3}
283 Date: June, 2021
284 KernelVersion: v5.14
285 Contact: linux-cxl@vger.kernel.org
286 Description:
287 (RO) When a CXL decoder is of devtype "cxl_decoder_root", it
288 represents a fixed memory window identified by platform
289 firmware. A fixed window may only support a subset of memory
290 types. The 'cap_*' attributes indicate whether persistent
291 memory, volatile memory, accelerator memory, and / or expander
292 memory may be mapped behind this decoder's memory window.
293
294
295 What: /sys/bus/cxl/devices/decoderX.Y/target_type
296 Date: June, 2021
297 KernelVersion: v5.14
298 Contact: linux-cxl@vger.kernel.org
299 Description:
300 (RO) When a CXL decoder is of devtype "cxl_decoder_switch", it
301 can optionally decode either accelerator memory (type-2) or
302 expander memory (type-3). The 'target_type' attribute indicates
303 the current setting which may dynamically change based on what
304 memory regions are activated in this decode hierarchy.
305
306
307 What: /sys/bus/cxl/devices/endpointX/CDAT
308 Date: July, 2022
309 KernelVersion: v6.0
310 Contact: linux-cxl@vger.kernel.org
311 Description:
312 (RO) If this sysfs entry is not present no DOE mailbox was
313 found to support CDAT data. If it is present and the length of
314 the data is 0 reading the CDAT data failed. Otherwise the CDAT
315 data is reported.
316
317
318 What: /sys/bus/cxl/devices/decoderX.Y/mode
319 Date: May, 2022
320 KernelVersion: v6.0
321 Contact: linux-cxl@vger.kernel.org
322 Description:
323 (RW) When a CXL decoder is of devtype "cxl_decoder_endpoint" it
324 translates from a host physical address range, to a device local
325 address range. Device-local address ranges are further split
326 into a 'ram' (volatile memory) range and 'pmem' (persistent
327 memory) range. The 'mode' attribute emits one of 'ram', 'pmem',
328 'mixed', or 'none'. The 'mixed' indication is for error cases
329 when a decoder straddles the volatile/persistent partition
330 boundary, and 'none' indicates the decoder is not actively
331 decoding, or no DPA allocation policy has been set.
332
333 'mode' can be written, when the decoder is in the 'disabled'
334 state, with either 'ram' or 'pmem' to set the boundaries for the
335 next allocation.
336
337
338 What: /sys/bus/cxl/devices/decoderX.Y/dpa_resource
339 Date: May, 2022
340 KernelVersion: v6.0
341 Contact: linux-cxl@vger.kernel.org
342 Description:
343 (RO) When a CXL decoder is of devtype "cxl_decoder_endpoint",
344 and its 'dpa_size' attribute is non-zero, this attribute
345 indicates the device physical address (DPA) base address of the
346 allocation.
347
348
349 What: /sys/bus/cxl/devices/decoderX.Y/dpa_size
350 Date: May, 2022
351 KernelVersion: v6.0
352 Contact: linux-cxl@vger.kernel.org
353 Description:
354 (RW) When a CXL decoder is of devtype "cxl_decoder_endpoint" it
355 translates from a host physical address range, to a device local
356 address range. The range, base address plus length in bytes, of
357 DPA allocated to this decoder is conveyed in these 2 attributes.
358 Allocations can be mutated as long as the decoder is in the
359 disabled state. A write to 'dpa_size' releases the previous DPA
360 allocation and then attempts to allocate from the free capacity
361 in the device partition referred to by 'decoderX.Y/mode'.
362 Allocate and free requests can only be performed on the highest
363 instance number disabled decoder with non-zero size. I.e.
364 allocations are enforced to occur in increasing 'decoderX.Y/id'
365 order and frees are enforced to occur in decreasing
366 'decoderX.Y/id' order.
367
368
369 What: /sys/bus/cxl/devices/decoderX.Y/interleave_ways
370 Date: May, 2022
371 KernelVersion: v6.0
372 Contact: linux-cxl@vger.kernel.org
373 Description:
374 (RO) The number of targets across which this decoder's host
375 physical address (HPA) memory range is interleaved. The device
376 maps every Nth block of HPA (of size ==
377 'interleave_granularity') to consecutive DPA addresses. The
378 decoder's position in the interleave is determined by the
379 device's (endpoint or switch) switch ancestry. For root
380 decoders their interleave is specified by platform firmware and
381 they only specify a downstream target order for host bridges.
382
383
384 What: /sys/bus/cxl/devices/decoderX.Y/interleave_granularity
385 Date: May, 2022
386 KernelVersion: v6.0
387 Contact: linux-cxl@vger.kernel.org
388 Description:
389 (RO) The number of consecutive bytes of host physical address
390 space this decoder claims at address N before the decode rotates
391 to the next target in the interleave at address N +
392 interleave_granularity (assuming N is aligned to
393 interleave_granularity).
394
395
396 What: /sys/bus/cxl/devices/decoderX.Y/create_{pmem,ram}_region
397 Date: May, 2022, January, 2023
398 KernelVersion: v6.0 (pmem), v6.3 (ram)
399 Contact: linux-cxl@vger.kernel.org
400 Description:
401 (RW) Write a string in the form 'regionZ' to start the process
402 of defining a new persistent, or volatile memory region
403 (interleave-set) within the decode range bounded by root decoder
404 'decoderX.Y'. The value written must match the current value
405 returned from reading this attribute. An atomic compare exchange
406 operation is done on write to assign the requested id to a
407 region and allocate the region-id for the next creation attempt.
408 EBUSY is returned if the region name written does not match the
409 current cached value.
410
411
412 What: /sys/bus/cxl/devices/decoderX.Y/delete_region
413 Date: May, 2022
414 KernelVersion: v6.0
415 Contact: linux-cxl@vger.kernel.org
416 Description:
417 (WO) Write a string in the form 'regionZ' to delete that region,
418 provided it is currently idle / not bound to a driver.
419
420
421 What: /sys/bus/cxl/devices/decoderX.Y/qos_class
422 Date: May, 2023
423 KernelVersion: v6.5
424 Contact: linux-cxl@vger.kernel.org
425 Description:
426 (RO) For CXL host platforms that support "QoS Telemmetry" this
427 root-decoder-only attribute conveys a platform specific cookie
428 that identifies a QoS performance class for the CXL Window.
429 This class-id can be compared against a similar "qos_class"
430 published for each memory-type that an endpoint supports. While
431 it is not required that endpoints map their local memory-class
432 to a matching platform class, mismatches are not recommended and
433 there are platform specific side-effects that may result.
434
435
436 What: /sys/bus/cxl/devices/regionZ/uuid
437 Date: May, 2022
438 KernelVersion: v6.0
439 Contact: linux-cxl@vger.kernel.org
440 Description:
441 (RW) Write a unique identifier for the region. This field must
442 be set for persistent regions and it must not conflict with the
443 UUID of another region. For volatile ram regions this
444 attribute is a read-only empty string.
445
446
447 What: /sys/bus/cxl/devices/regionZ/interleave_granularity
448 Date: May, 2022
449 KernelVersion: v6.0
450 Contact: linux-cxl@vger.kernel.org
451 Description:
452 (RW) Set the number of consecutive bytes each device in the
453 interleave set will claim. The possible interleave granularity
454 values are determined by the CXL spec and the participating
455 devices.
456
457
458 What: /sys/bus/cxl/devices/regionZ/interleave_ways
459 Date: May, 2022
460 KernelVersion: v6.0
461 Contact: linux-cxl@vger.kernel.org
462 Description:
463 (RW) Configures the number of devices participating in the
464 region is set by writing this value. Each device will provide
465 1/interleave_ways of storage for the region.
466
467
468 What: /sys/bus/cxl/devices/regionZ/size
469 Date: May, 2022
470 KernelVersion: v6.0
471 Contact: linux-cxl@vger.kernel.org
472 Description:
473 (RW) System physical address space to be consumed by the region.
474 When written trigger the driver to allocate space out of the
475 parent root decoder's address space. When read the size of the
476 address space is reported and should match the span of the
477 region's resource attribute. Size shall be set after the
478 interleave configuration parameters. Once set it cannot be
479 changed, only freed by writing 0. The kernel makes no guarantees
480 that data is maintained over an address space freeing event, and
481 there is no guarantee that a free followed by an allocate
482 results in the same address being allocated.
483
484
485 What: /sys/bus/cxl/devices/regionZ/mode
486 Date: January, 2023
487 KernelVersion: v6.3
488 Contact: linux-cxl@vger.kernel.org
489 Description:
490 (RO) The mode of a region is established at region creation time
491 and dictates the mode of the endpoint decoder that comprise the
492 region. For more details on the possible modes see
493 /sys/bus/cxl/devices/decoderX.Y/mode
494
495
496 What: /sys/bus/cxl/devices/regionZ/resource
497 Date: May, 2022
498 KernelVersion: v6.0
499 Contact: linux-cxl@vger.kernel.org
500 Description:
501 (RO) A region is a contiguous partition of a CXL root decoder
502 address space. Region capacity is allocated by writing to the
503 size attribute, the resulting physical address space determined
504 by the driver is reflected here. It is therefore not useful to
505 read this before writing a value to the size attribute.
506
507
508 What: /sys/bus/cxl/devices/regionZ/target[0..N]
509 Date: May, 2022
510 KernelVersion: v6.0
511 Contact: linux-cxl@vger.kernel.org
512 Description:
513 (RW) Write an endpoint decoder object name to 'targetX' where X
514 is the intended position of the endpoint device in the region
515 interleave and N is the 'interleave_ways' setting for the
516 region. ENXIO is returned if the write results in an impossible
517 to map decode scenario, like the endpoint is unreachable at that
518 position relative to the root decoder interleave. EBUSY is
519 returned if the position in the region is already occupied, or
520 if the region is not in a state to accept interleave
521 configuration changes. EINVAL is returned if the object name is
522 not an endpoint decoder. Once all positions have been
523 successfully written a final validation for decode conflicts is
524 performed before activating the region.
525
526
527 What: /sys/bus/cxl/devices/regionZ/commit
528 Date: May, 2022
529 KernelVersion: v6.0
530 Contact: linux-cxl@vger.kernel.org
531 Description:
532 (RW) Write a boolean 'true' string value to this attribute to
533 trigger the region to transition from the software programmed
534 state to the actively decoding in hardware state. The commit
535 operation in addition to validating that the region is in proper
536 configured state, validates that the decoders are being
537 committed in spec mandated order (last committed decoder id +
538 1), and checks that the hardware accepts the commit request.
539 Reading this value indicates whether the region is committed or
540 not.
541
542
543 What: /sys/bus/cxl/devices/memX/trigger_poison_list
544 Date: April, 2023
545 KernelVersion: v6.4
546 Contact: linux-cxl@vger.kernel.org
547 Description:
548 (WO) When a boolean 'true' is written to this attribute the
549 memdev driver retrieves the poison list from the device. The
550 list consists of addresses that are poisoned, or would result
551 in poison if accessed, and the source of the poison. This
552 attribute is only visible for devices supporting the
553 capability. The retrieved errors are logged as kernel
554 events when cxl_poison event tracing is enabled.
555
556
557 What: /sys/bus/cxl/devices/regionZ/accessY/read_bandwidth
558 /sys/bus/cxl/devices/regionZ/accessY/write_banwidth
559 Date: Jan, 2024
560 KernelVersion: v6.9
561 Contact: linux-cxl@vger.kernel.org
562 Description:
563 (RO) The aggregated read or write bandwidth of the region. The
564 number is the accumulated read or write bandwidth of all CXL memory
565 devices that contributes to the region in MB/s. It is
566 identical data that should appear in
567 /sys/devices/system/node/nodeX/accessY/initiators/read_bandwidth or
568 /sys/devices/system/node/nodeX/accessY/initiators/write_bandwidth.
569 See Documentation/ABI/stable/sysfs-devices-node. access0 provides
570 the number to the closest initiator and access1 provides the
571 number to the closest CPU.
572
573
574 What: /sys/bus/cxl/devices/regionZ/accessY/read_latency
575 /sys/bus/cxl/devices/regionZ/accessY/write_latency
576 Date: Jan, 2024
577 KernelVersion: v6.9
578 Contact: linux-cxl@vger.kernel.org
579 Description:
580 (RO) The read or write latency of the region. The number is
581 the worst read or write latency of all CXL memory devices that
582 contributes to the region in nanoseconds. It is identical data
583 that should appear in
584 /sys/devices/system/node/nodeX/accessY/initiators/read_latency or
585 /sys/devices/system/node/nodeX/accessY/initiators/write_latency.
586 See Documentation/ABI/stable/sysfs-devices-node. access0 provides
587 the number to the closest initiator and access1 provides the
588 number to the closest CPU.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.