1 Using TopDown metrics !! 1 Using TopDown metrics in user space
2 --------------------- !! 2 -----------------------------------
3 3
4 TopDown metrics break apart performance bottle !! 4 Intel CPUs (since Sandy Bridge and Silvermont) support a TopDown
5 1 it is typical to get metrics on retiring, ba !! 5 methology to break down CPU pipeline execution into 4 bottlenecks:
6 bound, and backend bound. Higher levels provid !! 6 frontend bound, backend bound, bad speculation, retiring.
7 level 1 bottlenecks, such as at level 2: core <<
8 heavy operations, light operations, branch mis <<
9 clears, fetch latency and fetch bandwidth. For <<
10 7
11 perf stat --topdown implements this using avai !! 8 For more details on Topdown see [1][5]
12 per architecture. <<
13 9
14 % perf stat -a --topdown -I1000 !! 10 Traditionally this was implemented by events in generic counters
15 # time % tma_retiring % tma_b !! 11 and specific formulas to compute the bottlenecks.
16 1.001141351 11.5 !! 12
17 2.006141972 13.4 !! 13 perf stat --topdown implements this.
18 3.010162040 12.9 !! 14
19 4.014009311 12.5 !! 15 Full Top Down includes more levels that can break down the
20 5.017838554 11.8 !! 16 bottlenecks further. This is not directly implemented in perf,
21 5.704818971 14.0 !! 17 but available in other tools that can run on top of perf,
22 ... !! 18 such as toplev[2] or vtune[3]
23 19
24 New Topdown features in Intel Ice Lake !! 20 New Topdown features in Ice Lake
25 ====================================== !! 21 ===============================
26 22
27 With Ice Lake CPUs the TopDown metrics are dir 23 With Ice Lake CPUs the TopDown metrics are directly available as
28 fixed counters and do not require generic coun 24 fixed counters and do not require generic counters. This allows
29 to collect TopDown always in addition to other 25 to collect TopDown always in addition to other events.
30 26
31 Using TopDown through RDPMC in applications on !! 27 % perf stat -a --topdown -I1000
32 ============================================== !! 28 # time retiring bad speculation frontend bound backend bound
>> 29 1.001281330 23.0% 15.3% 29.6% 32.1%
>> 30 2.003009005 5.0% 6.8% 46.6% 41.6%
>> 31 3.004646182 6.7% 6.7% 46.0% 40.6%
>> 32 4.006326375 5.0% 6.4% 47.6% 41.0%
>> 33 5.007991804 5.1% 6.3% 46.3% 42.3%
>> 34 6.009626773 6.2% 7.1% 47.3% 39.3%
>> 35 7.011296356 4.7% 6.7% 46.2% 42.4%
>> 36 8.012951831 4.7% 6.7% 47.5% 41.1%
>> 37 ...
>> 38
>> 39 This also enables measuring TopDown per thread/process instead
>> 40 of only per core.
>> 41
>> 42 Using TopDown through RDPMC in applications on Ice Lake
>> 43 ======================================================
33 44
34 For more fine grained measurements it can be u 45 For more fine grained measurements it can be useful to
35 access the new directly from user space. This 46 access the new directly from user space. This is more complicated,
36 but drastically lowers overhead. 47 but drastically lowers overhead.
37 48
38 On Ice Lake, there is a new fixed counter 3: S 49 On Ice Lake, there is a new fixed counter 3: SLOTS, which reports
39 "pipeline SLOTS" (cycles multiplied by core is 50 "pipeline SLOTS" (cycles multiplied by core issue width) and a
40 metric register that reports slots ratios for 51 metric register that reports slots ratios for the different bottleneck
41 categories. 52 categories.
42 53
43 The metrics counter is CPU model specific and 54 The metrics counter is CPU model specific and is not available on older
44 CPUs. 55 CPUs.
45 56
46 Example code 57 Example code
47 ============ 58 ============
48 59
49 Library functions to do the functionality desc 60 Library functions to do the functionality described below
50 is also available in libjevents [4] 61 is also available in libjevents [4]
51 62
52 The application opens a group with fixed count 63 The application opens a group with fixed counter 3 (SLOTS) and any
53 metric event, and allow user programs to read 64 metric event, and allow user programs to read the performance counters.
54 65
55 Fixed counter 3 is mapped to a pseudo event ev 66 Fixed counter 3 is mapped to a pseudo event event=0x00, umask=04,
56 so the perf_event_attr structure should be ini 67 so the perf_event_attr structure should be initialized with
57 { .config = 0x0400, .type = PERF_TYPE_RAW } 68 { .config = 0x0400, .type = PERF_TYPE_RAW }
58 The metric events are mapped to the pseudo eve 69 The metric events are mapped to the pseudo event event=0x00, umask=0x8X.
59 For example, the perf_event_attr structure can 70 For example, the perf_event_attr structure can be initialized with
60 { .config = 0x8000, .type = PERF_TYPE_RAW } fo 71 { .config = 0x8000, .type = PERF_TYPE_RAW } for Retiring metric event
61 The Fixed counter 3 must be the leader of the 72 The Fixed counter 3 must be the leader of the group.
62 73
63 #include <linux/perf_event.h> 74 #include <linux/perf_event.h>
64 #include <sys/mman.h> <<
65 #include <sys/syscall.h> 75 #include <sys/syscall.h>
66 #include <unistd.h> 76 #include <unistd.h>
67 77
68 /* Provide own perf_event_open stub because gl 78 /* Provide own perf_event_open stub because glibc doesn't */
69 __attribute__((weak)) 79 __attribute__((weak))
70 int perf_event_open(struct perf_event_attr *at 80 int perf_event_open(struct perf_event_attr *attr, pid_t pid,
71 int cpu, int group_fd, uns 81 int cpu, int group_fd, unsigned long flags)
72 { 82 {
73 return syscall(__NR_perf_event_open, a 83 return syscall(__NR_perf_event_open, attr, pid, cpu, group_fd, flags);
74 } 84 }
75 85
76 /* Open slots counter file descriptor for curr 86 /* Open slots counter file descriptor for current task. */
77 struct perf_event_attr slots = { 87 struct perf_event_attr slots = {
78 .type = PERF_TYPE_RAW, 88 .type = PERF_TYPE_RAW,
79 .size = sizeof(struct perf_event_attr) 89 .size = sizeof(struct perf_event_attr),
80 .config = 0x400, 90 .config = 0x400,
81 .exclude_kernel = 1, 91 .exclude_kernel = 1,
82 }; 92 };
83 93
84 int slots_fd = perf_event_open(&slots, 0, -1, 94 int slots_fd = perf_event_open(&slots, 0, -1, -1, 0);
85 if (slots_fd < 0) 95 if (slots_fd < 0)
86 ... error ... 96 ... error ...
87 97
88 /* Memory mapping the fd permits _rdpmc calls <<
89 void *slots_p = mmap(0, getpagesize(), PROT_RE <<
90 if (!slot_p) <<
91 .... error ... <<
92 <<
93 /* 98 /*
94 * Open metrics event file descriptor for curr 99 * Open metrics event file descriptor for current task.
95 * Set slots event as the leader of the group. 100 * Set slots event as the leader of the group.
96 */ 101 */
97 struct perf_event_attr metrics = { 102 struct perf_event_attr metrics = {
98 .type = PERF_TYPE_RAW, 103 .type = PERF_TYPE_RAW,
99 .size = sizeof(struct perf_event_attr) 104 .size = sizeof(struct perf_event_attr),
100 .config = 0x8000, 105 .config = 0x8000,
101 .exclude_kernel = 1, 106 .exclude_kernel = 1,
102 }; 107 };
103 108
104 int metrics_fd = perf_event_open(&metrics, 0, 109 int metrics_fd = perf_event_open(&metrics, 0, -1, slots_fd, 0);
105 if (metrics_fd < 0) 110 if (metrics_fd < 0)
106 ... error ... 111 ... error ...
107 112
108 /* Memory mapping the fd permits _rdpmc calls <<
109 void *metrics_p = mmap(0, getpagesize(), PROT_ <<
110 if (!metrics_p) <<
111 ... error ... <<
112 <<
113 Note: the file descriptors returned by the per <<
114 mapped to permit calls to the _rdpmd instructi <<
115 by writing the /sys/devices/cpu/rdpmc sysfs no <<
116 113
117 The RDPMC instruction (or _rdpmc compiler intr 114 The RDPMC instruction (or _rdpmc compiler intrinsic) can now be used
118 to read slots and the topdown metrics at diffe 115 to read slots and the topdown metrics at different points of the program:
119 116
120 #include <stdint.h> 117 #include <stdint.h>
121 #include <x86intrin.h> 118 #include <x86intrin.h>
122 119
123 #define RDPMC_FIXED (1 << 30) /* ret 120 #define RDPMC_FIXED (1 << 30) /* return fixed counters */
124 #define RDPMC_METRIC (1 << 29) /* ret 121 #define RDPMC_METRIC (1 << 29) /* return metric counters */
125 122
126 #define FIXED_COUNTER_SLOTS 3 123 #define FIXED_COUNTER_SLOTS 3
127 #define METRIC_COUNTER_TOPDOWN_L1_L2 0 !! 124 #define METRIC_COUNTER_TOPDOWN_L1 0
128 125
129 static inline uint64_t read_slots(void) 126 static inline uint64_t read_slots(void)
130 { 127 {
131 return _rdpmc(RDPMC_FIXED | FIXED_COUN 128 return _rdpmc(RDPMC_FIXED | FIXED_COUNTER_SLOTS);
132 } 129 }
133 130
134 static inline uint64_t read_metrics(void) 131 static inline uint64_t read_metrics(void)
135 { 132 {
136 return _rdpmc(RDPMC_METRIC | METRIC_CO !! 133 return _rdpmc(RDPMC_METRIC | METRIC_COUNTER_TOPDOWN_L1);
137 } 134 }
138 135
139 Then the program can be instrumented to read t 136 Then the program can be instrumented to read these metrics at different
140 points. 137 points.
141 138
142 It's not a good idea to do this with too short 139 It's not a good idea to do this with too short code regions,
143 as the parallelism and overlap in the CPU prog 140 as the parallelism and overlap in the CPU program execution will
144 cause too much measurement inaccuracy. For exa 141 cause too much measurement inaccuracy. For example instrumenting
145 individual basic blocks is definitely too fine 142 individual basic blocks is definitely too fine grained.
146 143
147 _rdpmc calls should not be mixed with reading <<
148 through system calls, as the kernel will reset <<
149 call. <<
150 <<
151 Decoding metrics values 144 Decoding metrics values
152 ======================= 145 =======================
153 146
154 The value reported by read_metrics() contains 147 The value reported by read_metrics() contains four 8 bit fields
155 that represent a scaled ratio that represent t 148 that represent a scaled ratio that represent the Level 1 bottleneck.
156 All four fields add up to 0xff (= 100%) 149 All four fields add up to 0xff (= 100%)
157 150
158 The binary ratios in the metric value can be c 151 The binary ratios in the metric value can be converted to float ratios:
159 152
160 #define GET_METRIC(m, i) (((m) >> (i*8)) & 0xf 153 #define GET_METRIC(m, i) (((m) >> (i*8)) & 0xff)
161 154
162 /* L1 Topdown metric events */ <<
163 #define TOPDOWN_RETIRING(val) ((float)GET_ME 155 #define TOPDOWN_RETIRING(val) ((float)GET_METRIC(val, 0) / 0xff)
164 #define TOPDOWN_BAD_SPEC(val) ((float)GET_ME 156 #define TOPDOWN_BAD_SPEC(val) ((float)GET_METRIC(val, 1) / 0xff)
165 #define TOPDOWN_FE_BOUND(val) ((float)GET_ME 157 #define TOPDOWN_FE_BOUND(val) ((float)GET_METRIC(val, 2) / 0xff)
166 #define TOPDOWN_BE_BOUND(val) ((float)GET_ME 158 #define TOPDOWN_BE_BOUND(val) ((float)GET_METRIC(val, 3) / 0xff)
167 159
168 /* <<
169 * L2 Topdown metric events. <<
170 * Available on Sapphire Rapids and later plat <<
171 */ <<
172 #define TOPDOWN_HEAVY_OPS(val) ((floa <<
173 #define TOPDOWN_BR_MISPREDICT(val) ((floa <<
174 #define TOPDOWN_FETCH_LAT(val) ((floa <<
175 #define TOPDOWN_MEM_BOUND(val) ((floa <<
176 <<
177 and then converted to percent for printing. 160 and then converted to percent for printing.
178 161
179 The ratios in the metric accumulate for the ti 162 The ratios in the metric accumulate for the time when the counter
180 is enabled. For measuring programs it is often 163 is enabled. For measuring programs it is often useful to measure
181 specific sections. For this it is needed to de 164 specific sections. For this it is needed to deltas on metrics.
182 165
183 This can be done by scaling the metrics with t 166 This can be done by scaling the metrics with the slots counter
184 read at the same time. 167 read at the same time.
185 168
186 Then it's possible to take deltas of these slo 169 Then it's possible to take deltas of these slots counts
187 measured at different points, and determine th 170 measured at different points, and determine the metrics
188 for that time period. 171 for that time period.
189 172
190 slots_a = read_slots(); 173 slots_a = read_slots();
191 metric_a = read_metrics(); 174 metric_a = read_metrics();
192 175
193 ... larger code region ... 176 ... larger code region ...
194 177
195 slots_b = read_slots() 178 slots_b = read_slots()
196 metric_b = read_metrics() 179 metric_b = read_metrics()
197 180
198 # compute scaled metrics for measureme 181 # compute scaled metrics for measurement a
199 retiring_slots_a = GET_METRIC(metric_a 182 retiring_slots_a = GET_METRIC(metric_a, 0) * slots_a
200 bad_spec_slots_a = GET_METRIC(metric_a 183 bad_spec_slots_a = GET_METRIC(metric_a, 1) * slots_a
201 fe_bound_slots_a = GET_METRIC(metric_a 184 fe_bound_slots_a = GET_METRIC(metric_a, 2) * slots_a
202 be_bound_slots_a = GET_METRIC(metric_a 185 be_bound_slots_a = GET_METRIC(metric_a, 3) * slots_a
203 186
204 # compute delta scaled metrics between 187 # compute delta scaled metrics between b and a
205 retiring_slots = GET_METRIC(metric_b, 188 retiring_slots = GET_METRIC(metric_b, 0) * slots_b - retiring_slots_a
206 bad_spec_slots = GET_METRIC(metric_b, 189 bad_spec_slots = GET_METRIC(metric_b, 1) * slots_b - bad_spec_slots_a
207 fe_bound_slots = GET_METRIC(metric_b, 190 fe_bound_slots = GET_METRIC(metric_b, 2) * slots_b - fe_bound_slots_a
208 be_bound_slots = GET_METRIC(metric_b, 191 be_bound_slots = GET_METRIC(metric_b, 3) * slots_b - be_bound_slots_a
209 192
210 Later the individual ratios of L1 metric event !! 193 Later the individual ratios for the measurement period can be recreated
211 be recreated from these counts. !! 194 from these counts.
212 195
213 slots_delta = slots_b - slots_a 196 slots_delta = slots_b - slots_a
214 retiring_ratio = (float)retiring_slots 197 retiring_ratio = (float)retiring_slots / slots_delta
215 bad_spec_ratio = (float)bad_spec_slots 198 bad_spec_ratio = (float)bad_spec_slots / slots_delta
216 fe_bound_ratio = (float)fe_bound_slots 199 fe_bound_ratio = (float)fe_bound_slots / slots_delta
217 be_bound_ratio = (float)be_bound_slots 200 be_bound_ratio = (float)be_bound_slots / slota_delta
218 201
219 printf("Retiring %.2f%% Bad Speculatio 202 printf("Retiring %.2f%% Bad Speculation %.2f%% FE Bound %.2f%% BE Bound %.2f%%\n",
220 retiring_ratio * 100., 203 retiring_ratio * 100.,
221 bad_spec_ratio * 100., 204 bad_spec_ratio * 100.,
222 fe_bound_ratio * 100., 205 fe_bound_ratio * 100.,
223 be_bound_ratio * 100.); 206 be_bound_ratio * 100.);
224 207
225 The individual ratios of L2 metric events for <<
226 recreated from L1 and L2 metric counters. (Ava <<
227 later platforms) <<
228 <<
229 # compute scaled metrics for measureme <<
230 heavy_ops_slots_a = GET_METRIC(metric_ <<
231 br_mispredict_slots_a = GET_METRIC(met <<
232 fetch_lat_slots_a = GET_METRIC(metric_ <<
233 mem_bound_slots_a = GET_METRIC(metric_ <<
234 <<
235 # compute delta scaled metrics between <<
236 heavy_ops_slots = GET_METRIC(metric_b, <<
237 br_mispredict_slots = GET_METRIC(metri <<
238 fetch_lat_slots = GET_METRIC(metric_b, <<
239 mem_bound_slots = GET_METRIC(metric_b, <<
240 <<
241 slots_delta = slots_b - slots_a <<
242 heavy_ops_ratio = (float)heavy_ops_slo <<
243 light_ops_ratio = retiring_ratio - hea <<
244 <<
245 br_mispredict_ratio = (float)br_mispre <<
246 machine_clears_ratio = bad_spec_ratio <<
247 <<
248 fetch_lat_ratio = (float)fetch_lat_slo <<
249 fetch_bw_ratio = fe_bound_ratio - fetc <<
250 <<
251 mem_bound_ratio = (float)mem_bound_slo <<
252 core_bound_ratio = be_bound_ratio - me <<
253 <<
254 printf("Heavy Operations %.2f%% Light <<
255 "Branch Mispredict %.2f%% Machi <<
256 "Fetch Latency %.2f%% Fetch Ban <<
257 "Mem Bound %.2f%% Core Bound %. <<
258 heavy_ops_ratio * 100., <<
259 light_ops_ratio * 100., <<
260 br_mispredict_ratio * 100., <<
261 machine_clears_ratio * 100., <<
262 fetch_lat_ratio * 100., <<
263 fetch_bw_ratio * 100., <<
264 mem_bound_ratio * 100., <<
265 core_bound_ratio * 100.); <<
266 <<
267 Resetting metrics counters 208 Resetting metrics counters
268 ========================== 209 ==========================
269 210
270 Since the individual metrics are only 8bit the 211 Since the individual metrics are only 8bit they lose precision for
271 short regions over time because the number of 212 short regions over time because the number of cycles covered by each
272 fraction bit shrinks. So the counters need to 213 fraction bit shrinks. So the counters need to be reset regularly.
273 214
274 When using the kernel perf API the kernel rese 215 When using the kernel perf API the kernel resets on every read.
275 So as long as the reading is at reasonable int 216 So as long as the reading is at reasonable intervals (every few
276 seconds) the precision is good. 217 seconds) the precision is good.
277 218
278 When using perf stat it is recommended to alwa 219 When using perf stat it is recommended to always use the -I option,
279 with no longer interval than a few seconds 220 with no longer interval than a few seconds
280 221
281 perf stat -I 1000 --topdown ... 222 perf stat -I 1000 --topdown ...
282 223
283 For user programs using RDPMC directly the cou 224 For user programs using RDPMC directly the counter can
284 be reset explicitly using ioctl: 225 be reset explicitly using ioctl:
285 226
286 ioctl(perf_fd, PERF_EVENT_IOC_RESET, 0 227 ioctl(perf_fd, PERF_EVENT_IOC_RESET, 0);
287 228
288 This "opens" a new measurement period. 229 This "opens" a new measurement period.
289 230
290 A program using RDPMC for TopDown should sched 231 A program using RDPMC for TopDown should schedule such a reset
291 regularly, as in every few seconds. 232 regularly, as in every few seconds.
292 233
293 Limits on Intel Ice Lake !! 234 Limits on Ice Lake
294 ======================== !! 235 ==================
295 236
296 Four pseudo TopDown metric events are exposed 237 Four pseudo TopDown metric events are exposed for the end-users,
297 topdown-retiring, topdown-bad-spec, topdown-fe 238 topdown-retiring, topdown-bad-spec, topdown-fe-bound and topdown-be-bound.
298 They can be used to collect the TopDown value 239 They can be used to collect the TopDown value under the following
299 rules: 240 rules:
300 - All the TopDown metric events must be in a g 241 - All the TopDown metric events must be in a group with the SLOTS event.
301 - The SLOTS event must be the leader of the gr 242 - The SLOTS event must be the leader of the group.
302 - The PERF_FORMAT_GROUP flag must be applied f 243 - The PERF_FORMAT_GROUP flag must be applied for each TopDown metric
303 events 244 events
304 245
305 The SLOTS event and the TopDown metric events 246 The SLOTS event and the TopDown metric events can be counting members of
306 a sampling read group. Since the SLOTS event m 247 a sampling read group. Since the SLOTS event must be the leader of a TopDown
307 group, the second event of the group is the sa 248 group, the second event of the group is the sampling event.
308 For example, perf record -e '{slots, $sampling 249 For example, perf record -e '{slots, $sampling_event, topdown-retiring}:S'
309 250
310 Extension on Intel Sapphire Rapids Server <<
311 ========================================= <<
312 The metrics counter is extended to support TMA <<
313 The lower half of the register is the TMA leve <<
314 The upper half is also divided into four 8-bit <<
315 metrics. Four more TopDown metric events are e <<
316 topdown-heavy-ops, topdown-br-mispredict, topd <<
317 topdown-mem-bound. <<
318 <<
319 Each of the new level 2 metrics in the upper h <<
320 corresponding level 1 metric in the lower half <<
321 other four level 2 metrics by subtracting corr <<
322 <<
323 Light_Operations = Retiring - Heavy_Operat <<
324 Machine_Clears = Bad_Speculation - Branch_ <<
325 Fetch_Bandwidth = Frontend_Bound - Fetch_L <<
326 Core_Bound = Backend_Bound - Memory_Bound <<
327 <<
328 TPEBS in TopDown <<
329 ================ <<
330 <<
331 TPEBS (Timed PEBS) is one of the new Intel PMU <<
332 Rapids microarchitecture. The TPEBS feature ad <<
333 in the Basic Info group of the PEBS record. It <<
334 retirement of the previous instruction to the <<
335 Please refer to Section 8.4.1 of "Intel® Arch <<
336 Programming Reference" for more details about <<
337 extends PEBS record, sampling with weight opti <<
338 retire_latency value. <<
339 <<
340 perf record -e event_name -W ... <<
341 <<
342 In the most recent release of TMA, the metrics <<
343 values in some of the metrics’ formulas on p <<
344 For previous generations that do not support T <<
345 predefined per processor family by the hardwar <<
346 of workloads in execution environments, retire <<
347 time are more accurate. Therefore, new TMA met <<
348 more accurate performance analysis results. <<
349 <<
350 To support TPEBS in TMA metrics, a new modifie <<
351 capture retire_latency value of required event <<
352 with perf record. The retire_latency value wou <<
353 Currently, this feature is supported through p <<
354 <<
355 perf stat -M metric_name --record-tpeb <<
356 <<
357 <<
358 251
359 [1] https://software.intel.com/en-us/top-down- 252 [1] https://software.intel.com/en-us/top-down-microarchitecture-analysis-method-win
360 [2] https://sites.google.com/site/analysismeth !! 253 [2] https://github.com/andikleen/pmu-tools/wiki/toplev-manual
361 [3] https://perf.wiki.kernel.org/index.php/Top !! 254 [3] https://software.intel.com/en-us/intel-vtune-amplifier-xe
362 [4] https://github.com/andikleen/pmu-tools/tre 255 [4] https://github.com/andikleen/pmu-tools/tree/master/jevents
>> 256 [5] https://sites.google.com/site/analysismethods/yasin-pubs
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.