1 ============================================== 1 ================================================
2 HiSilicon PCIe Performance Monitoring Unit (PM 2 HiSilicon PCIe Performance Monitoring Unit (PMU)
3 ============================================== 3 ================================================
4 4
5 On Hip09, HiSilicon PCIe Performance Monitorin 5 On Hip09, HiSilicon PCIe Performance Monitoring Unit (PMU) could monitor
6 bandwidth, latency, bus utilization and buffer 6 bandwidth, latency, bus utilization and buffer occupancy data of PCIe.
7 7
8 Each PCIe Core has a PMU to monitor multi Root 8 Each PCIe Core has a PMU to monitor multi Root Ports of this PCIe Core and
9 all Endpoints downstream these Root Ports. 9 all Endpoints downstream these Root Ports.
10 10
11 11
12 HiSilicon PCIe PMU driver 12 HiSilicon PCIe PMU driver
13 ========================= 13 =========================
14 14
15 The PCIe PMU driver registers a perf PMU with 15 The PCIe PMU driver registers a perf PMU with the name of its sicl-id and PCIe
16 Core id.:: 16 Core id.::
17 17
18 /sys/bus/event_source/hisi_pcie<sicl>_core<c 18 /sys/bus/event_source/hisi_pcie<sicl>_core<core>
19 19
20 PMU driver provides description of available e 20 PMU driver provides description of available events and filter options in sysfs,
21 see /sys/bus/event_source/devices/hisi_pcie<si 21 see /sys/bus/event_source/devices/hisi_pcie<sicl>_core<core>.
22 22
23 The "format" directory describes all formats o 23 The "format" directory describes all formats of the config (events) and config1
24 (filter options) fields of the perf_event_attr 24 (filter options) fields of the perf_event_attr structure. The "events" directory
25 describes all documented events shown in perf 25 describes all documented events shown in perf list.
26 26
27 The "identifier" sysfs file allows users to id 27 The "identifier" sysfs file allows users to identify the version of the
28 PMU hardware device. 28 PMU hardware device.
29 29
30 The "bus" sysfs file allows users to get the b 30 The "bus" sysfs file allows users to get the bus number of Root Ports
31 monitored by PMU. 31 monitored by PMU.
32 32
33 Example usage of perf:: 33 Example usage of perf::
34 34
35 $# perf list 35 $# perf list
36 hisi_pcie0_core0/rx_mwr_latency/ [kernel PMU 36 hisi_pcie0_core0/rx_mwr_latency/ [kernel PMU event]
37 hisi_pcie0_core0/rx_mwr_cnt/ [kernel PMU eve 37 hisi_pcie0_core0/rx_mwr_cnt/ [kernel PMU event]
38 ------------------------------------------ 38 ------------------------------------------
39 39
40 $# perf stat -e hisi_pcie0_core0/rx_mwr_late 40 $# perf stat -e hisi_pcie0_core0/rx_mwr_latency,port=0xffff/
41 $# perf stat -e hisi_pcie0_core0/rx_mwr_cnt, 41 $# perf stat -e hisi_pcie0_core0/rx_mwr_cnt,port=0xffff/
42 42
43 The related events usually used to calculate t 43 The related events usually used to calculate the bandwidth, latency or others.
44 They need to start and end counting at the sam 44 They need to start and end counting at the same time, therefore related events
45 are best used in the same event group to get t 45 are best used in the same event group to get the expected value. There are two
46 ways to know if they are related events: 46 ways to know if they are related events:
47 47
48 a) By event name, such as the latency events " 48 a) By event name, such as the latency events "xxx_latency, xxx_cnt" or
49 bandwidth events "xxx_flux, xxx_time". 49 bandwidth events "xxx_flux, xxx_time".
50 b) By event type, such as "event=0xXXXX, event 50 b) By event type, such as "event=0xXXXX, event=0x1XXXX".
51 51
52 Example usage of perf group:: 52 Example usage of perf group::
53 53
54 $# perf stat -e "{hisi_pcie0_core0/rx_mwr_la 54 $# perf stat -e "{hisi_pcie0_core0/rx_mwr_latency,port=0xffff/,hisi_pcie0_core0/rx_mwr_cnt,port=0xffff/}"
55 55
56 The current driver does not support sampling. 56 The current driver does not support sampling. So "perf record" is unsupported.
57 Also attach to a task is unsupported for PCIe 57 Also attach to a task is unsupported for PCIe PMU.
58 58
59 Filter options 59 Filter options
60 -------------- 60 --------------
61 61
62 1. Target filter 62 1. Target filter
63 63
64 PMU could only monitor the performance of t 64 PMU could only monitor the performance of traffic downstream target Root
65 Ports or downstream target Endpoint. PCIe P 65 Ports or downstream target Endpoint. PCIe PMU driver support "port" and
66 "bdf" interfaces for users. 66 "bdf" interfaces for users.
67 Please notice that, one of these two interf 67 Please notice that, one of these two interfaces must be set, and these two
68 interfaces aren't supported at the same tim 68 interfaces aren't supported at the same time. If they are both set, only
69 "port" filter is valid. 69 "port" filter is valid.
70 If "port" filter not being set or is set ex 70 If "port" filter not being set or is set explicitly to zero (default), the
71 "bdf" filter will be in effect, because "bd 71 "bdf" filter will be in effect, because "bdf=0" meaning 0000:000:00.0.
72 72
73 - port 73 - port
74 74
75 "port" filter can be used in all PCIe PMU 75 "port" filter can be used in all PCIe PMU events, target Root Port can be
76 selected by configuring the 16-bits-bitma 76 selected by configuring the 16-bits-bitmap "port". Multi ports can be
77 selected for AP-layer-events, and only on 77 selected for AP-layer-events, and only one port can be selected for
78 TL/DL-layer-events. 78 TL/DL-layer-events.
79 79
80 For example, if target Root Port is 0000: 80 For example, if target Root Port is 0000:00:00.0 (x8 lanes), bit0 of
81 bitmap should be set, port=0x1; if target 81 bitmap should be set, port=0x1; if target Root Port is 0000:00:04.0 (x4
82 lanes), bit8 is set, port=0x100; if these 82 lanes), bit8 is set, port=0x100; if these two Root Ports are both
83 monitored, port=0x101. 83 monitored, port=0x101.
84 84
85 Example usage of perf:: 85 Example usage of perf::
86 86
87 $# perf stat -e hisi_pcie0_core0/rx_mwr 87 $# perf stat -e hisi_pcie0_core0/rx_mwr_latency,port=0x1/ sleep 5
88 88
89 - bdf 89 - bdf
90 90
91 "bdf" filter can only be used in bandwidt 91 "bdf" filter can only be used in bandwidth events, target Endpoint is
92 selected by configuring BDF to "bdf". Cou 92 selected by configuring BDF to "bdf". Counter only counts the bandwidth of
93 message requested by target Endpoint. 93 message requested by target Endpoint.
94 94
95 For example, "bdf=0x3900" means BDF of ta 95 For example, "bdf=0x3900" means BDF of target Endpoint is 0000:39:00.0.
96 96
97 Example usage of perf:: 97 Example usage of perf::
98 98
99 $# perf stat -e hisi_pcie0_core0/rx_mrd 99 $# perf stat -e hisi_pcie0_core0/rx_mrd_flux,bdf=0x3900/ sleep 5
100 100
101 2. Trigger filter 101 2. Trigger filter
102 102
103 Event statistics start when the first time 103 Event statistics start when the first time TLP length is greater/smaller
104 than trigger condition. You can set the tri 104 than trigger condition. You can set the trigger condition by writing
105 "trig_len", and set the trigger mode by wri 105 "trig_len", and set the trigger mode by writing "trig_mode". This filter can
106 only be used in bandwidth events. 106 only be used in bandwidth events.
107 107
108 For example, "trig_len=4" means trigger con 108 For example, "trig_len=4" means trigger condition is 2^4 DW, "trig_mode=0"
109 means statistics start when TLP length > tr 109 means statistics start when TLP length > trigger condition, "trig_mode=1"
110 means start when TLP length < condition. 110 means start when TLP length < condition.
111 111
112 Example usage of perf:: 112 Example usage of perf::
113 113
114 $# perf stat -e hisi_pcie0_core0/rx_mrd_f 114 $# perf stat -e hisi_pcie0_core0/rx_mrd_flux,port=0xffff,trig_len=0x4,trig_mode=1/ sleep 5
115 115
116 3. Threshold filter 116 3. Threshold filter
117 117
118 Counter counts when TLP length within the s 118 Counter counts when TLP length within the specified range. You can set the
119 threshold by writing "thr_len", and set the 119 threshold by writing "thr_len", and set the threshold mode by writing
120 "thr_mode". This filter can only be used in 120 "thr_mode". This filter can only be used in bandwidth events.
121 121
122 For example, "thr_len=4" means threshold is 122 For example, "thr_len=4" means threshold is 2^4 DW, "thr_mode=0" means
123 counter counts when TLP length >= threshold 123 counter counts when TLP length >= threshold, and "thr_mode=1" means counts
124 when TLP length < threshold. 124 when TLP length < threshold.
125 125
126 Example usage of perf:: 126 Example usage of perf::
127 127
128 $# perf stat -e hisi_pcie0_core0/rx_mrd_f 128 $# perf stat -e hisi_pcie0_core0/rx_mrd_flux,port=0xffff,thr_len=0x4,thr_mode=1/ sleep 5
129 129
130 4. TLP Length filter 130 4. TLP Length filter
131 131
132 When counting bandwidth, the data can be co 132 When counting bandwidth, the data can be composed of certain parts of TLP
133 packets. You can specify it through "len_mo 133 packets. You can specify it through "len_mode":
134 134
135 - 2'b00: Reserved (Do not use this since th 135 - 2'b00: Reserved (Do not use this since the behaviour is undefined)
136 - 2'b01: Bandwidth of TLP payloads 136 - 2'b01: Bandwidth of TLP payloads
137 - 2'b10: Bandwidth of TLP headers 137 - 2'b10: Bandwidth of TLP headers
138 - 2'b11: Bandwidth of both TLP payloads and 138 - 2'b11: Bandwidth of both TLP payloads and headers
139 139
140 For example, "len_mode=2" means only counti 140 For example, "len_mode=2" means only counting the bandwidth of TLP headers
141 and "len_mode=3" means the final bandwidth 141 and "len_mode=3" means the final bandwidth data is composed of both TLP
142 headers and payloads. Default value if not 142 headers and payloads. Default value if not specified is 2'b11.
143 143
144 Example usage of perf:: 144 Example usage of perf::
145 145
146 $# perf stat -e hisi_pcie0_core0/rx_mrd_f 146 $# perf stat -e hisi_pcie0_core0/rx_mrd_flux,port=0xffff,len_mode=0x1/ sleep 5
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.