1 .. SPDX-License-Identifier: GPL-2.0
2 .. include:: <isonum.txt>
3
4 ==========================
5 The MSI Driver Guide HOWTO
6 ==========================
7
8 :Authors: Tom L Nguyen; Martine Silbermann; Ma
9
10 :Copyright: 2003, 2008 Intel Corporation
11
12 About this guide
13 ================
14
15 This guide describes the basics of Message Sig
16 the advantages of using MSI over traditional i
17 to change your driver to use MSI or MSI-X and
18 try if a device doesn't support MSIs.
19
20
21 What are MSIs?
22 ==============
23
24 A Message Signaled Interrupt is a write from t
25 address which causes an interrupt to be receiv
26
27 The MSI capability was first specified in PCI
28 in PCI 3.0 to allow each interrupt to be maske
29 capability was also introduced with PCI 3.0.
30 per device than MSI and allows interrupts to b
31
32 Devices may support both MSI and MSI-X, but on
33 a time.
34
35
36 Why use MSIs?
37 =============
38
39 There are three reasons why using MSIs can giv
40 traditional pin-based interrupts.
41
42 Pin-based PCI interrupts are often shared amon
43 To support this, the kernel must call each int
44 with an interrupt, which leads to reduced perf
45 a whole. MSIs are never shared, so this probl
46
47 When a device writes data to memory, then rais
48 it is possible that the interrupt may arrive b
49 arrived in memory (this becomes more likely wi
50 bridges). In order to ensure that all the dat
51 the interrupt handler must read a register on
52 the interrupt. PCI transaction ordering rules
53 arrive in memory before the value may be retur
54 Using MSIs avoids this problem as the interrup
55 pass the data writes, so by the time the inter
56 knows that all the data has arrived in memory.
57
58 PCI devices can only support a single pin-base
59 Often drivers have to query the device to find
60 occurred, slowing down interrupt handling for
61 MSIs, a device can support more interrupts, al
62 to be specialised to a different purpose. One
63 infrequent conditions (such as errors) their o
64 the driver to handle the normal interrupt hand
65 Other possible designs include giving one inte
66 in a network card or each port in a storage co
67
68
69 How to use MSIs
70 ===============
71
72 PCI devices are initialised to use pin-based i
73 driver has to set up the device to use MSI or
74 support MSIs correctly, and for those machines
75 will simply fail and the device will continue
76
77 Include kernel support for MSIs
78 -------------------------------
79
80 To support MSI or MSI-X, the kernel must be bu
81 option enabled. This option is only available
82 and it may depend on some other options also b
83 on x86, you must also enable X86_UP_APIC or SM
84 CONFIG_PCI_MSI option.
85
86 Using MSI
87 ---------
88
89 Most of the hard work is done for the driver i
90 simply has to request that the PCI layer set u
91 device.
92
93 To automatically use MSI or MSI-X interrupt ve
94 function::
95
96 int pci_alloc_irq_vectors(struct pci_dev *de
97 unsigned int max_vecs, unsigne
98
99 which allocates up to max_vecs interrupt vecto
100 returns the number of vectors allocated or a n
101 has a requirements for a minimum number of vec
102 min_vecs argument set to this limit, and the P
103 if it can't meet the minimum number of vectors
104
105 The flags argument is used to specify which ty
106 by the device and the driver (PCI_IRQ_INTX, PC
107 A convenient short-hand (PCI_IRQ_ALL_TYPES) is
108 any possible kind of interrupt. If the PCI_IR
109 pci_alloc_irq_vectors() will spread the interr
110
111 To get the Linux IRQ numbers passed to request
112 vectors, use the following function::
113
114 int pci_irq_vector(struct pci_dev *dev, unsi
115
116 Any allocated resources should be freed before
117 the following function::
118
119 void pci_free_irq_vectors(struct pci_dev *de
120
121 If a device supports both MSI-X and MSI capabi
122 MSI-X facilities in preference to the MSI faci
123 number of interrupts between 1 and 2048. In c
124 a maximum of 32 interrupts (and must be a powe
125 MSI interrupt vectors must be allocated consec
126 not be able to allocate as many vectors for MS
127 some platforms, MSI interrupts must all be tar
128 whereas MSI-X interrupts can all be targeted a
129
130 If a device supports neither MSI-X or MSI it w
131 legacy IRQ vector.
132
133 The typical usage of MSI or MSI-X interrupts i
134 as possible, likely up to the limit supported
135 larger than the number supported by the device
136 capped to the supported limit, so there is no
137 vectors supported beforehand::
138
139 nvec = pci_alloc_irq_vectors(pdev, 1,
140 if (nvec < 0)
141 goto out_err;
142
143 If a driver is unable or unwilling to deal wit
144 interrupts it can request a particular number
145 number to pci_alloc_irq_vectors() function as
146 'max_vecs' parameters::
147
148 ret = pci_alloc_irq_vectors(pdev, nvec
149 if (ret < 0)
150 goto out_err;
151
152 The most notorious example of the request type
153 the single MSI mode for a device. It could be
154 'min_vecs' and 'max_vecs'::
155
156 ret = pci_alloc_irq_vectors(pdev, 1, 1
157 if (ret < 0)
158 goto out_err;
159
160 Some devices might not support using legacy li
161 the driver can specify that only MSI or MSI-X
162
163 nvec = pci_alloc_irq_vectors(pdev, 1,
164 if (nvec < 0)
165 goto out_err;
166
167 Legacy APIs
168 -----------
169
170 The following old APIs to enable and disable M
171 not be used in new code::
172
173 pci_enable_msi() /* deprecated
174 pci_disable_msi() /* deprecated
175 pci_enable_msix_range() /* deprecated
176 pci_enable_msix_exact() /* deprecated
177 pci_disable_msix() /* deprecated
178
179 Additionally there are APIs to provide the num
180 vectors: pci_msi_vec_count() and pci_msix_vec_
181 should be avoided in favor of letting pci_allo
182 number of vectors. If you have a legitimate s
183 of vectors we might have to revisit that decis
184 pci_nr_irq_vectors() helper that handles MSI a
185
186 Considerations when using MSIs
187 ------------------------------
188
189 Spinlocks
190 ~~~~~~~~~
191
192 Most device drivers have a per-device spinlock
193 interrupt handler. With pin-based interrupts
194 necessary to disable interrupts (Linux guarant
195 not be re-entered). If a device uses multiple
196 must disable interrupts while the lock is held
197 a different interrupt, the driver will deadloc
198 acquire the spinlock. Such deadlocks can be a
199 spin_lock_irqsave() or spin_lock_irq() which d
200 and acquire the lock (see Documentation/kernel
201
202 How to tell whether MSI/MSI-X is enabled on a
203 ----------------------------------------------
204
205 Using 'lspci -v' (as root) may show some devic
206 Signalled Interrupts" or "MSI-X" capabilities.
207 has an 'Enable' flag which is followed with ei
208 or "-" (disabled).
209
210
211 MSI quirks
212 ==========
213
214 Several PCI chipsets or devices are known not
215 The PCI stack provides three ways to disable M
216
217 1. globally
218 2. on all devices behind a specific bridge
219 3. on a single device
220
221 Disabling MSIs globally
222 -----------------------
223
224 Some host chipsets simply don't support MSIs p
225 lucky, the manufacturer knows this and has ind
226 FADT table. In this case, Linux automatically
227 Some boards don't include this information in
228 to detect them ourselves. The complete list o
229 quirk_disable_all_msi() function in drivers/pc
230
231 If you have a board which has problems with MS
232 on the kernel command line to disable MSIs on
233 in your best interests to report the problem t
234 including a full 'lspci -v' so we can add the
235
236 Disabling MSIs below a bridge
237 -----------------------------
238
239 Some PCI bridges are not able to route MSIs be
240 In this case, MSIs must be disabled on all dev
241
242 Some bridges allow you to enable MSIs by chang
243 PCI configuration space (especially the Hypert
244 as the nVidia nForce and Serverworks HT2000).
245 Linux mostly knows about them and automaticall
246 If you have a bridge unknown to Linux, you can
247 MSIs in configuration space using whatever met
248 enable MSIs on that bridge by doing::
249
250 echo 1 > /sys/bus/pci/devices/$bridge/m
251
252 where $bridge is the PCI address of the bridge
253 0000:00:0e.0).
254
255 To disable MSIs, echo 0 instead of 1. Changin
256 done with caution as it could break interrupt
257 below this bridge.
258
259 Again, please notify linux-pci@vger.kernel.org
260 special handling.
261
262 Disabling MSIs on a single device
263 ---------------------------------
264
265 Some devices are known to have faulty MSI impl
266 is handled in the individual device driver, bu
267 to handle this with a quirk. Some drivers hav
268 of MSI. While this is a convenient workaround
269 it is not good practice, and should not be emu
270
271 Finding why MSIs are disabled on a device
272 -----------------------------------------
273
274 From the above three sections, you can see tha
275 why MSIs may not be enabled for a given device
276 be to examine your dmesg carefully to determin
277 for your machine. You should also check your
278 have enabled CONFIG_PCI_MSI.
279
280 Then, 'lspci -t' gives the list of bridges abo
281 `/sys/bus/pci/devices/*/msi_bus` will tell you
282 or disabled (0). If 0 is found in any of the
283 to bridges between the PCI root and the device
284
285 It is also worth checking the device driver to
286 For example, it may contain calls to pci_alloc
287 PCI_IRQ_MSI or PCI_IRQ_MSIX flags.
288
289
290 List of device drivers MSI(-X) APIs
291 ===================================
292
293 The PCI/MSI subsystem has a dedicated C file f
294 APIs — `drivers/pci/msi/api.c`. The followin
295
296 .. kernel-doc:: drivers/pci/msi/api.c
297 :export:
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.