1 What: /sys/kernel/debug/qat_<device> 1 What: /sys/kernel/debug/qat_<device>_<BDF>/telemetry/control
2 Date: March 2024 2 Date: March 2024
3 KernelVersion: 6.8 3 KernelVersion: 6.8
4 Contact: qat-linux@intel.com 4 Contact: qat-linux@intel.com
5 Description: (RW) Enables/disables the repo 5 Description: (RW) Enables/disables the reporting of telemetry metrics.
6 6
7 Allowed values to write: 7 Allowed values to write:
8 ======================== 8 ========================
9 * 0: disable telemetry 9 * 0: disable telemetry
10 * 1: enable telemetry 10 * 1: enable telemetry
11 * 2, 3, 4: enable telemetry an 11 * 2, 3, 4: enable telemetry and calculate minimum, maximum
12 and average for each counter 12 and average for each counter over 2, 3 or 4 samples
13 13
14 Returned values: 14 Returned values:
15 ================ 15 ================
16 * 1-4: telemetry is enabled an 16 * 1-4: telemetry is enabled and running
17 * 0: telemetry is disabled 17 * 0: telemetry is disabled
18 18
19 Example. 19 Example.
20 20
21 Writing '3' to this file start 21 Writing '3' to this file starts the collection of
22 telemetry metrics. Samples are 22 telemetry metrics. Samples are collected every second and
23 stored in a circular buffer of 23 stored in a circular buffer of size 3. These values are then
24 used to calculate the minimum, 24 used to calculate the minimum, maximum and average for each
25 counter. After enabling, count 25 counter. After enabling, counters can be retrieved through
26 the ``device_data`` file:: 26 the ``device_data`` file::
27 27
28 echo 3 > /sys/kernel/debug/q 28 echo 3 > /sys/kernel/debug/qat_4xxx_0000:6b:00.0/telemetry/control
29 29
30 Writing '0' to this file stops 30 Writing '0' to this file stops the collection of telemetry
31 metrics:: 31 metrics::
32 32
33 echo 0 > /sys/kernel/debug/q 33 echo 0 > /sys/kernel/debug/qat_4xxx_0000:6b:00.0/telemetry/control
34 34
35 This attribute is only availab 35 This attribute is only available for qat_4xxx devices.
36 36
37 What: /sys/kernel/debug/qat_<device> 37 What: /sys/kernel/debug/qat_<device>_<BDF>/telemetry/device_data
38 Date: March 2024 38 Date: March 2024
39 KernelVersion: 6.8 39 KernelVersion: 6.8
40 Contact: qat-linux@intel.com 40 Contact: qat-linux@intel.com
41 Description: (RO) Reports device telemetry 41 Description: (RO) Reports device telemetry counters.
42 Reads report metrics about per 42 Reads report metrics about performance and utilization of
43 a QAT device: 43 a QAT device:
44 44
45 ======================= ====== 45 ======================= ========================================
46 Field Descri 46 Field Description
47 ======================= ====== 47 ======================= ========================================
48 sample_cnt number 48 sample_cnt number of acquisitions of telemetry data
49 from t 49 from the device. Reads are performed
50 every 50 every 1000 ms.
51 pci_trans_cnt number 51 pci_trans_cnt number of PCIe partial transactions
52 max_rd_lat maximu 52 max_rd_lat maximum logged read latency [ns] (could
53 be any 53 be any read operation)
54 rd_lat_acc_avg averag 54 rd_lat_acc_avg average read latency [ns]
55 max_gp_lat max ge 55 max_gp_lat max get to put latency [ns] (only takes
56 sample 56 samples for AE0)
57 gp_lat_acc_avg averag 57 gp_lat_acc_avg average get to put latency [ns]
58 bw_in PCIe, 58 bw_in PCIe, write bandwidth [Mbps]
59 bw_out PCIe, 59 bw_out PCIe, read bandwidth [Mbps]
60 at_page_req_lat_avg Addres 60 at_page_req_lat_avg Address Translator(AT), average page
61 reques 61 request latency [ns]
62 at_trans_lat_avg AT, av 62 at_trans_lat_avg AT, average page translation latency [ns]
63 at_max_tlb_used AT, ma 63 at_max_tlb_used AT, maximum uTLB used
64 util_cpr<N> utiliz 64 util_cpr<N> utilization of Compression slice N [%]
65 exec_cpr<N> execut 65 exec_cpr<N> execution count of Compression slice N
66 util_xlt<N> utiliz 66 util_xlt<N> utilization of Translator slice N [%]
67 exec_xlt<N> execut 67 exec_xlt<N> execution count of Translator slice N
68 util_dcpr<N> utiliz 68 util_dcpr<N> utilization of Decompression slice N [%]
69 exec_dcpr<N> execut 69 exec_dcpr<N> execution count of Decompression slice N
70 util_pke<N> utiliz 70 util_pke<N> utilization of PKE N [%]
71 exec_pke<N> execut 71 exec_pke<N> execution count of PKE N
72 util_ucs<N> utiliz 72 util_ucs<N> utilization of UCS slice N [%]
73 exec_ucs<N> execut 73 exec_ucs<N> execution count of UCS slice N
74 util_wat<N> utiliz 74 util_wat<N> utilization of Wireless Authentication
75 slice 75 slice N [%]
76 exec_wat<N> execut 76 exec_wat<N> execution count of Wireless Authentication
77 slice 77 slice N
78 util_wcp<N> utiliz 78 util_wcp<N> utilization of Wireless Cipher slice N [%]
79 exec_wcp<N> execut 79 exec_wcp<N> execution count of Wireless Cipher slice N
80 util_cph<N> utiliz 80 util_cph<N> utilization of Cipher slice N [%]
81 exec_cph<N> execut 81 exec_cph<N> execution count of Cipher slice N
82 util_ath<N> utiliz 82 util_ath<N> utilization of Authentication slice N [%]
83 exec_ath<N> execut 83 exec_ath<N> execution count of Authentication slice N
84 ======================= ====== 84 ======================= ========================================
85 85
86 The telemetry report file can 86 The telemetry report file can be read with the following command::
87 87
88 cat /sys/kernel/debug/qat_4x 88 cat /sys/kernel/debug/qat_4xxx_0000:6b:00.0/telemetry/device_data
89 89
90 If ``control`` is set to 1, on 90 If ``control`` is set to 1, only the current values of the
91 counters are displayed:: 91 counters are displayed::
92 92
93 <counter_name> <current> 93 <counter_name> <current>
94 94
95 If ``control`` is 2, 3 or 4, c 95 If ``control`` is 2, 3 or 4, counters are displayed in the
96 following format:: 96 following format::
97 97
98 <counter_name> <current> <mi 98 <counter_name> <current> <min> <max> <avg>
99 99
100 If a device lacks of a specifi 100 If a device lacks of a specific accelerator, the corresponding
101 attribute is not reported. 101 attribute is not reported.
102 102
103 This attribute is only availab 103 This attribute is only available for qat_4xxx devices.
104 104
105 What: /sys/kernel/debug/qat_<device> 105 What: /sys/kernel/debug/qat_<device>_<BDF>/telemetry/rp_<A/B/C/D>_data
106 Date: March 2024 106 Date: March 2024
107 KernelVersion: 6.8 107 KernelVersion: 6.8
108 Contact: qat-linux@intel.com 108 Contact: qat-linux@intel.com
109 Description: (RW) Selects up to 4 Ring Pair 109 Description: (RW) Selects up to 4 Ring Pairs (RP) to monitor, one per file,
110 and report telemetry counters 110 and report telemetry counters related to each.
111 111
112 Allowed values to write: 112 Allowed values to write:
113 ======================== 113 ========================
114 * 0 to ``<num_rps - 1>``: 114 * 0 to ``<num_rps - 1>``:
115 Ring pair to be monitored. T 115 Ring pair to be monitored. The value of ``num_rps`` can be
116 retrieved through ``/sys/bus 116 retrieved through ``/sys/bus/pci/devices/<BDF>/qat/num_rps``.
117 See Documentation/ABI/testin 117 See Documentation/ABI/testing/sysfs-driver-qat.
118 118
119 Reads report metrics about per 119 Reads report metrics about performance and utilization of
120 the selected RP: 120 the selected RP:
121 121
122 ======================= ====== 122 ======================= ========================================
123 Field Descri 123 Field Description
124 ======================= ====== 124 ======================= ========================================
125 sample_cnt number 125 sample_cnt number of acquisitions of telemetry data
126 from t 126 from the device. Reads are performed
127 every 127 every 1000 ms
128 rp_num RP num 128 rp_num RP number associated with slot <A/B/C/D>
129 service_type servic 129 service_type service associated to the RP
130 pci_trans_cnt number 130 pci_trans_cnt number of PCIe partial transactions
131 gp_lat_acc_avg averag 131 gp_lat_acc_avg average get to put latency [ns]
132 bw_in PCIe, 132 bw_in PCIe, write bandwidth [Mbps]
133 bw_out PCIe, 133 bw_out PCIe, read bandwidth [Mbps]
134 at_glob_devtlb_hit Messag 134 at_glob_devtlb_hit Message descriptor DevTLB hit rate
135 at_glob_devtlb_miss Messag 135 at_glob_devtlb_miss Message descriptor DevTLB miss rate
136 tl_at_payld_devtlb_hit Payloa 136 tl_at_payld_devtlb_hit Payload DevTLB hit rate
137 tl_at_payld_devtlb_miss Payloa 137 tl_at_payld_devtlb_miss Payload DevTLB miss rate
138 ======================= ====== 138 ======================= ========================================
139 139
140 Example. 140 Example.
141 141
142 Writing the value '32' to the 142 Writing the value '32' to the file ``rp_C_data`` starts the
143 collection of telemetry metric 143 collection of telemetry metrics for ring pair 32::
144 144
145 echo 32 > /sys/kernel/debug/ 145 echo 32 > /sys/kernel/debug/qat_4xxx_0000:6b:00.0/telemetry/rp_C_data
146 146
147 Once a ring pair is selected, 147 Once a ring pair is selected, statistics can be read accessing
148 the file:: 148 the file::
149 149
150 cat /sys/kernel/debug/qat_4x 150 cat /sys/kernel/debug/qat_4xxx_0000:6b:00.0/telemetry/rp_C_data
151 151
152 If ``control`` is set to 1, on 152 If ``control`` is set to 1, only the current values of the
153 counters are displayed:: 153 counters are displayed::
154 154
155 <counter_name> <current> 155 <counter_name> <current>
156 156
157 If ``control`` is 2, 3 or 4, c 157 If ``control`` is 2, 3 or 4, counters are displayed in the
158 following format:: 158 following format::
159 159
160 <counter_name> <current> <mi 160 <counter_name> <current> <min> <max> <avg>
161 161
162 162
163 On QAT GEN4 devices there are 163 On QAT GEN4 devices there are 64 RPs on a PF, so the allowed
164 values are 0..63. This number 164 values are 0..63. This number is absolute to the device.
165 If Virtual Functions (VF) are 165 If Virtual Functions (VF) are used, the ring pair number can
166 be derived from the Bus, Devic 166 be derived from the Bus, Device, Function of the VF:
167 167
168 ============ ====== ====== === 168 ============ ====== ====== ====== ======
169 PCI BDF/VF RP0 RP1 RP2 169 PCI BDF/VF RP0 RP1 RP2 RP3
170 ============ ====== ====== === 170 ============ ====== ====== ====== ======
171 0000:6b:0.1 RP 0 RP 1 RP 171 0000:6b:0.1 RP 0 RP 1 RP 2 RP 3
172 0000:6b:0.2 RP 4 RP 5 RP 172 0000:6b:0.2 RP 4 RP 5 RP 6 RP 7
173 0000:6b:0.3 RP 8 RP 9 RP 173 0000:6b:0.3 RP 8 RP 9 RP 10 RP 11
174 0000:6b:0.4 RP 12 RP 13 RP 174 0000:6b:0.4 RP 12 RP 13 RP 14 RP 15
175 0000:6b:0.5 RP 16 RP 17 RP 175 0000:6b:0.5 RP 16 RP 17 RP 18 RP 19
176 0000:6b:0.6 RP 20 RP 21 RP 176 0000:6b:0.6 RP 20 RP 21 RP 22 RP 23
177 0000:6b:0.7 RP 24 RP 25 RP 177 0000:6b:0.7 RP 24 RP 25 RP 26 RP 27
178 0000:6b:1.0 RP 28 RP 29 RP 178 0000:6b:1.0 RP 28 RP 29 RP 30 RP 31
179 0000:6b:1.1 RP 32 RP 33 RP 179 0000:6b:1.1 RP 32 RP 33 RP 34 RP 35
180 0000:6b:1.2 RP 36 RP 37 RP 180 0000:6b:1.2 RP 36 RP 37 RP 38 RP 39
181 0000:6b:1.3 RP 40 RP 41 RP 181 0000:6b:1.3 RP 40 RP 41 RP 42 RP 43
182 0000:6b:1.4 RP 44 RP 45 RP 182 0000:6b:1.4 RP 44 RP 45 RP 46 RP 47
183 0000:6b:1.5 RP 48 RP 49 RP 183 0000:6b:1.5 RP 48 RP 49 RP 50 RP 51
184 0000:6b:1.6 RP 52 RP 53 RP 184 0000:6b:1.6 RP 52 RP 53 RP 54 RP 55
185 0000:6b:1.7 RP 56 RP 57 RP 185 0000:6b:1.7 RP 56 RP 57 RP 58 RP 59
186 0000:6b:2.0 RP 60 RP 61 RP 186 0000:6b:2.0 RP 60 RP 61 RP 62 RP 63
187 ============ ====== ====== === 187 ============ ====== ====== ====== ======
188 188
189 The mapping is only valid for 189 The mapping is only valid for the BDFs of VFs on the host.
190 190
191 191
192 The service provided on a ring 192 The service provided on a ring-pair varies depending on the
193 configuration. The configurati 193 configuration. The configuration for a given device can be
194 queried and set using ``cfg_se 194 queried and set using ``cfg_services``.
195 See Documentation/ABI/testing/ 195 See Documentation/ABI/testing/sysfs-driver-qat for details.
196 196
197 The following table reports ho 197 The following table reports how ring pairs are mapped to VFs
198 on the PF 0000:6b:0.0 configur 198 on the PF 0000:6b:0.0 configured for `sym;asym` or `asym;sym`:
199 199
200 =========== ============ ===== 200 =========== ============ =========== ============ ===========
201 PCI BDF/VF RP0/service RP1/s 201 PCI BDF/VF RP0/service RP1/service RP2/service RP3/service
202 =========== ============ ===== 202 =========== ============ =========== ============ ===========
203 0000:6b:0.1 RP 0 asym RP 1 203 0000:6b:0.1 RP 0 asym RP 1 sym RP 2 asym RP 3 sym
204 0000:6b:0.2 RP 4 asym RP 5 204 0000:6b:0.2 RP 4 asym RP 5 sym RP 6 asym RP 7 sym
205 0000:6b:0.3 RP 8 asym RP 9 205 0000:6b:0.3 RP 8 asym RP 9 sym RP10 asym RP11 sym
206 ... ... ... 206 ... ... ... ... ...
207 =========== ============ ===== 207 =========== ============ =========== ============ ===========
208 208
209 All VFs follow the same patter 209 All VFs follow the same pattern.
210 210
211 211
212 The following table reports ho 212 The following table reports how ring pairs are mapped to VFs on
213 the PF 0000:6b:0.0 configured 213 the PF 0000:6b:0.0 configured for `dc`:
214 214
215 =========== ============ ===== 215 =========== ============ =========== ============ ===========
216 PCI BDF/VF RP0/service RP1/s 216 PCI BDF/VF RP0/service RP1/service RP2/service RP3/service
217 =========== ============ ===== 217 =========== ============ =========== ============ ===========
218 0000:6b:0.1 RP 0 dc RP 1 218 0000:6b:0.1 RP 0 dc RP 1 dc RP 2 dc RP 3 dc
219 0000:6b:0.2 RP 4 dc RP 5 219 0000:6b:0.2 RP 4 dc RP 5 dc RP 6 dc RP 7 dc
220 0000:6b:0.3 RP 8 dc RP 9 220 0000:6b:0.3 RP 8 dc RP 9 dc RP10 dc RP11 dc
221 ... ... ... 221 ... ... ... ... ...
222 =========== ============ ===== 222 =========== ============ =========== ============ ===========
223 223
224 The mapping of a RP to a servi 224 The mapping of a RP to a service can be retrieved using
225 ``rp2srv`` from sysfs. 225 ``rp2srv`` from sysfs.
226 See Documentation/ABI/testing/ 226 See Documentation/ABI/testing/sysfs-driver-qat for details.
227 227
228 This attribute is only availab 228 This attribute is only available for qat_4xxx devices.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.