1 .. SPDX-License-Identifier: GPL-2.0 1 .. SPDX-License-Identifier: GPL-2.0
2 .. include:: <isonum.txt> 2 .. include:: <isonum.txt>
3 3
4 ==================================== 4 ====================================
5 PCI Express I/O Virtualization Howto 5 PCI Express I/O Virtualization Howto
6 ==================================== 6 ====================================
7 7
8 :Copyright: |copy| 2009 Intel Corporation 8 :Copyright: |copy| 2009 Intel Corporation
9 :Authors: - Yu Zhao <yu.zhao@intel.com> 9 :Authors: - Yu Zhao <yu.zhao@intel.com>
10 - Donald Dutile <ddutile@redhat.com> 10 - Donald Dutile <ddutile@redhat.com>
11 11
12 Overview 12 Overview
13 ======== 13 ========
14 14
15 What is SR-IOV 15 What is SR-IOV
16 -------------- 16 --------------
17 17
18 Single Root I/O Virtualization (SR-IOV) is a P 18 Single Root I/O Virtualization (SR-IOV) is a PCI Express Extended
19 capability which makes one physical device app 19 capability which makes one physical device appear as multiple virtual
20 devices. The physical device is referred to as 20 devices. The physical device is referred to as Physical Function (PF)
21 while the virtual devices are referred to as V 21 while the virtual devices are referred to as Virtual Functions (VF).
22 Allocation of the VF can be dynamically contro 22 Allocation of the VF can be dynamically controlled by the PF via
23 registers encapsulated in the capability. By d 23 registers encapsulated in the capability. By default, this feature is
24 not enabled and the PF behaves as traditional 24 not enabled and the PF behaves as traditional PCIe device. Once it's
25 turned on, each VF's PCI configuration space c 25 turned on, each VF's PCI configuration space can be accessed by its own
26 Bus, Device and Function Number (Routing ID). 26 Bus, Device and Function Number (Routing ID). And each VF also has PCI
27 Memory Space, which is used to map its registe 27 Memory Space, which is used to map its register set. VF device driver
28 operates on the register set so it can be func 28 operates on the register set so it can be functional and appear as a
29 real existing PCI device. 29 real existing PCI device.
30 30
31 User Guide 31 User Guide
32 ========== 32 ==========
33 33
34 How can I enable SR-IOV capability 34 How can I enable SR-IOV capability
35 ---------------------------------- 35 ----------------------------------
36 36
37 Multiple methods are available for SR-IOV enab 37 Multiple methods are available for SR-IOV enablement.
38 In the first method, the device driver (PF dri 38 In the first method, the device driver (PF driver) will control the
39 enabling and disabling of the capability via A 39 enabling and disabling of the capability via API provided by SR-IOV core.
40 If the hardware has SR-IOV capability, loading 40 If the hardware has SR-IOV capability, loading its PF driver would
41 enable it and all VFs associated with the PF. 41 enable it and all VFs associated with the PF. Some PF drivers require
42 a module parameter to be set to determine the 42 a module parameter to be set to determine the number of VFs to enable.
43 In the second method, a write to the sysfs fil 43 In the second method, a write to the sysfs file sriov_numvfs will
44 enable and disable the VFs associated with a P 44 enable and disable the VFs associated with a PCIe PF. This method
45 enables per-PF, VF enable/disable values versu 45 enables per-PF, VF enable/disable values versus the first method,
46 which applies to all PFs of the same device. 46 which applies to all PFs of the same device. Additionally, the
47 PCI SRIOV core support ensures that enable/dis 47 PCI SRIOV core support ensures that enable/disable operations are
48 valid to reduce duplication in multiple driver 48 valid to reduce duplication in multiple drivers for the same
49 checks, e.g., check numvfs == 0 if enabling VF 49 checks, e.g., check numvfs == 0 if enabling VFs, ensure
50 numvfs <= totalvfs. 50 numvfs <= totalvfs.
51 The second method is the recommended method fo 51 The second method is the recommended method for new/future VF devices.
52 52
53 How can I use the Virtual Functions 53 How can I use the Virtual Functions
54 ----------------------------------- 54 -----------------------------------
55 55
56 The VF is treated as hot-plugged PCI devices i 56 The VF is treated as hot-plugged PCI devices in the kernel, so they
57 should be able to work in the same way as real 57 should be able to work in the same way as real PCI devices. The VF
58 requires device driver that is same as a norma 58 requires device driver that is same as a normal PCI device's.
59 59
60 Developer Guide 60 Developer Guide
61 =============== 61 ===============
62 62
63 SR-IOV API 63 SR-IOV API
64 ---------- 64 ----------
65 65
66 To enable SR-IOV capability: 66 To enable SR-IOV capability:
67 67
68 (a) For the first method, in the driver:: 68 (a) For the first method, in the driver::
69 69
70 int pci_enable_sriov(struct pci_dev *d 70 int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn);
71 71
72 'nr_virtfn' is number of VFs to be enabled. 72 'nr_virtfn' is number of VFs to be enabled.
73 73
74 (b) For the second method, from sysfs:: 74 (b) For the second method, from sysfs::
75 75
76 echo 'nr_virtfn' > \ 76 echo 'nr_virtfn' > \
77 /sys/bus/pci/devices/<DOMAIN:BUS:DEVIC 77 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs
78 78
79 To disable SR-IOV capability: 79 To disable SR-IOV capability:
80 80
81 (a) For the first method, in the driver:: 81 (a) For the first method, in the driver::
82 82
83 void pci_disable_sriov(struct pci_dev 83 void pci_disable_sriov(struct pci_dev *dev);
84 84
85 (b) For the second method, from sysfs:: 85 (b) For the second method, from sysfs::
86 86
87 echo 0 > \ 87 echo 0 > \
88 /sys/bus/pci/devices/<DOMAIN:BUS:DEVIC 88 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs
89 89
90 To enable auto probing VFs by a compatible dri 90 To enable auto probing VFs by a compatible driver on the host, run
91 command below before enabling SR-IOV capabilit 91 command below before enabling SR-IOV capabilities. This is the
92 default behavior. 92 default behavior.
93 :: 93 ::
94 94
95 echo 1 > \ 95 echo 1 > \
96 /sys/bus/pci/devices/<DOMAIN:BUS:DEVIC 96 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_drivers_autoprobe
97 97
98 To disable auto probing VFs by a compatible dr 98 To disable auto probing VFs by a compatible driver on the host, run
99 command below before enabling SR-IOV capabilit 99 command below before enabling SR-IOV capabilities. Updating this
100 entry will not affect VFs which are already pr 100 entry will not affect VFs which are already probed.
101 :: 101 ::
102 102
103 echo 0 > \ 103 echo 0 > \
104 /sys/bus/pci/devices/<DOMAIN:BUS:DEVIC 104 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_drivers_autoprobe
105 105
106 Usage example 106 Usage example
107 ------------- 107 -------------
108 108
109 Following piece of code illustrates the usage 109 Following piece of code illustrates the usage of the SR-IOV API.
110 :: 110 ::
111 111
112 static int dev_probe(struct pci_dev *d 112 static int dev_probe(struct pci_dev *dev, const struct pci_device_id *id)
113 { 113 {
114 pci_enable_sriov(dev, NR_VIRTF 114 pci_enable_sriov(dev, NR_VIRTFN);
115 115
116 ... 116 ...
117 117
118 return 0; 118 return 0;
119 } 119 }
120 120
121 static void dev_remove(struct pci_dev 121 static void dev_remove(struct pci_dev *dev)
122 { 122 {
123 pci_disable_sriov(dev); 123 pci_disable_sriov(dev);
124 124
125 ... 125 ...
126 } 126 }
127 127
128 static int dev_suspend(struct device * !! 128 static int dev_suspend(struct pci_dev *dev, pm_message_t state)
129 { 129 {
130 ... 130 ...
131 131
132 return 0; 132 return 0;
133 } 133 }
134 134
135 static int dev_resume(struct device *d !! 135 static int dev_resume(struct pci_dev *dev)
136 { 136 {
137 ... 137 ...
138 138
139 return 0; 139 return 0;
140 } 140 }
141 141
142 static void dev_shutdown(struct pci_de 142 static void dev_shutdown(struct pci_dev *dev)
143 { 143 {
144 ... 144 ...
145 } 145 }
146 146
147 static int dev_sriov_configure(struct 147 static int dev_sriov_configure(struct pci_dev *dev, int numvfs)
148 { 148 {
149 if (numvfs > 0) { 149 if (numvfs > 0) {
150 ... 150 ...
151 pci_enable_sriov(dev, 151 pci_enable_sriov(dev, numvfs);
152 ... 152 ...
153 return numvfs; 153 return numvfs;
154 } 154 }
155 if (numvfs == 0) { 155 if (numvfs == 0) {
156 .... 156 ....
157 pci_disable_sriov(dev) 157 pci_disable_sriov(dev);
158 ... 158 ...
159 return 0; 159 return 0;
160 } 160 }
161 } 161 }
162 162
163 static struct pci_driver dev_driver = 163 static struct pci_driver dev_driver = {
164 .name = "SR-IOV Physic 164 .name = "SR-IOV Physical Function driver",
165 .id_table = dev_id_table, 165 .id_table = dev_id_table,
166 .probe = dev_probe, 166 .probe = dev_probe,
167 .remove = dev_remove, 167 .remove = dev_remove,
168 .driver.pm = &dev_pm_ops, !! 168 .suspend = dev_suspend,
>> 169 .resume = dev_resume,
169 .shutdown = dev_shutdown, 170 .shutdown = dev_shutdown,
170 .sriov_configure = dev_sriov_c 171 .sriov_configure = dev_sriov_configure,
171 }; 172 };
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.