1 .. SPDX-License-Identifier: GPL-2.0
2 .. include:: <isonum.txt>
3
4 ====================================
5 PCI Express I/O Virtualization Howto
6 ====================================
7
8 :Copyright: |copy| 2009 Intel Corporation
9 :Authors: - Yu Zhao <yu.zhao@intel.com>
10 - Donald Dutile <ddutile@redhat.com>
11
12 Overview
13 ========
14
15 What is SR-IOV
16 --------------
17
18 Single Root I/O Virtualization (SR-IOV) is a PCI Express Extended
19 capability which makes one physical device appear as multiple virtual
20 devices. The physical device is referred to as Physical Function (PF)
21 while the virtual devices are referred to as Virtual Functions (VF).
22 Allocation of the VF can be dynamically controlled by the PF via
23 registers encapsulated in the capability. By default, this feature is
24 not enabled and the PF behaves as traditional PCIe device. Once it's
25 turned on, each VF's PCI configuration space can be accessed by its own
26 Bus, Device and Function Number (Routing ID). And each VF also has PCI
27 Memory Space, which is used to map its register set. VF device driver
28 operates on the register set so it can be functional and appear as a
29 real existing PCI device.
30
31 User Guide
32 ==========
33
34 How can I enable SR-IOV capability
35 ----------------------------------
36
37 Multiple methods are available for SR-IOV enablement.
38 In the first method, the device driver (PF driver) will control the
39 enabling and disabling of the capability via API provided by SR-IOV core.
40 If the hardware has SR-IOV capability, loading its PF driver would
41 enable it and all VFs associated with the PF. Some PF drivers require
42 a module parameter to be set to determine the number of VFs to enable.
43 In the second method, a write to the sysfs file sriov_numvfs will
44 enable and disable the VFs associated with a PCIe PF. This method
45 enables per-PF, VF enable/disable values versus the first method,
46 which applies to all PFs of the same device. Additionally, the
47 PCI SRIOV core support ensures that enable/disable operations are
48 valid to reduce duplication in multiple drivers for the same
49 checks, e.g., check numvfs == 0 if enabling VFs, ensure
50 numvfs <= totalvfs.
51 The second method is the recommended method for new/future VF devices.
52
53 How can I use the Virtual Functions
54 -----------------------------------
55
56 The VF is treated as hot-plugged PCI devices in the kernel, so they
57 should be able to work in the same way as real PCI devices. The VF
58 requires device driver that is same as a normal PCI device's.
59
60 Developer Guide
61 ===============
62
63 SR-IOV API
64 ----------
65
66 To enable SR-IOV capability:
67
68 (a) For the first method, in the driver::
69
70 int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn);
71
72 'nr_virtfn' is number of VFs to be enabled.
73
74 (b) For the second method, from sysfs::
75
76 echo 'nr_virtfn' > \
77 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs
78
79 To disable SR-IOV capability:
80
81 (a) For the first method, in the driver::
82
83 void pci_disable_sriov(struct pci_dev *dev);
84
85 (b) For the second method, from sysfs::
86
87 echo 0 > \
88 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs
89
90 To enable auto probing VFs by a compatible driver on the host, run
91 command below before enabling SR-IOV capabilities. This is the
92 default behavior.
93 ::
94
95 echo 1 > \
96 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_drivers_autoprobe
97
98 To disable auto probing VFs by a compatible driver on the host, run
99 command below before enabling SR-IOV capabilities. Updating this
100 entry will not affect VFs which are already probed.
101 ::
102
103 echo 0 > \
104 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_drivers_autoprobe
105
106 Usage example
107 -------------
108
109 Following piece of code illustrates the usage of the SR-IOV API.
110 ::
111
112 static int dev_probe(struct pci_dev *dev, const struct pci_device_id *id)
113 {
114 pci_enable_sriov(dev, NR_VIRTFN);
115
116 ...
117
118 return 0;
119 }
120
121 static void dev_remove(struct pci_dev *dev)
122 {
123 pci_disable_sriov(dev);
124
125 ...
126 }
127
128 static int dev_suspend(struct device *dev)
129 {
130 ...
131
132 return 0;
133 }
134
135 static int dev_resume(struct device *dev)
136 {
137 ...
138
139 return 0;
140 }
141
142 static void dev_shutdown(struct pci_dev *dev)
143 {
144 ...
145 }
146
147 static int dev_sriov_configure(struct pci_dev *dev, int numvfs)
148 {
149 if (numvfs > 0) {
150 ...
151 pci_enable_sriov(dev, numvfs);
152 ...
153 return numvfs;
154 }
155 if (numvfs == 0) {
156 ....
157 pci_disable_sriov(dev);
158 ...
159 return 0;
160 }
161 }
162
163 static struct pci_driver dev_driver = {
164 .name = "SR-IOV Physical Function driver",
165 .id_table = dev_id_table,
166 .probe = dev_probe,
167 .remove = dev_remove,
168 .driver.pm = &dev_pm_ops,
169 .shutdown = dev_shutdown,
170 .sriov_configure = dev_sriov_configure,
171 };
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.