1 ==============================================
2 Intel Omni-Path (OPA) Virtual Network Interfac
3 ==============================================
4
5 Intel Omni-Path (OPA) Virtual Network Interfac
6 supports Ethernet functionality over Omni-Path
7 the Ethernet packets between HFI nodes.
8
9 Architecture
10 =============
11 The patterns of exchanges of Omni-Path encapsu
12 involves one or more virtual Ethernet switches
13 fabric topology. A subset of HFI nodes on the
14 permitted to exchange encapsulated Ethernet pa
15 virtual Ethernet switch. The virtual Ethernet
16 abstractions achieved by configuring the HFI n
17 header generation and processing. In the simpl
18 nodes across the fabric exchange encapsulated
19 single virtual Ethernet switch. A virtual Ethe
20 an independent Ethernet network. The configura
21 Ethernet Manager (EM) which is part of the tru
22 application. HFI nodes can have multiple VNICs
23 different virtual Ethernet switch. The below d
24 of two virtual Ethernet switches with two HFI
25
26 +--------------
27 | Subnet/
28 | Ethernet
29 | Manager
30 +--------------
31 / /
32 / /
33 / /
34 / /
35 +-----------------------------+ +----------
36 | Virtual Ethernet Switch | | Virtual
37 | +---------+ +---------+ | | +--------
38 | | VPORT | | VPORT | | | | VPORT
39 +--+---------+----+---------+-+ +-+--------
40 | \ /
41 | \ /
42 | \/
43 | / \
44 | / \
45 +-----------+------------+ +----------
46 | VNIC | VNIC | | VNIC
47 +-----------+------------+ +----------
48 | HFI | |
49 +------------------------+ +----------
50
51
52 The Omni-Path encapsulated Ethernet packet for
53
54 ==================== =========================
55 Bits Field
56 ==================== =========================
57 Quad Word 0:
58 0-19 SLID (lower 20 bits)
59 20-30 Length (in Quad Words)
60 31 BECN bit
61 32-51 DLID (lower 20 bits)
62 52-56 SC (Service Class)
63 57-59 RC (Routing Control)
64 60 FECN bit
65 61-62 L2 (=10, 16B format)
66 63 LT (=1, Link Transfer Hea
67
68 Quad Word 1:
69 0-7 L4 type (=0x78 ETHERNET)
70 8-11 SLID[23:20]
71 12-15 DLID[23:20]
72 16-31 PKEY
73 32-47 Entropy
74 48-63 Reserved
75
76 Quad Word 2:
77 0-15 Reserved
78 16-31 L4 header
79 32-63 Ethernet Packet
80
81 Quad Words 3 to N-1:
82 0-63 Ethernet packet (pad exte
83
84 Quad Word N (last):
85 0-23 Ethernet packet (pad exte
86 24-55 ICRC
87 56-61 Tail
88 62-63 LT (=01, Link Transfer Ta
89 ==================== =========================
90
91 Ethernet packet is padded on the transmit side
92 packet is quad word aligned. The 'Tail' field
93 padded. On the receive side the 'Tail' field i
94 removed (along with ICRC, Tail and OPA header)
95 the network stack.
96
97 The L4 header field contains the virtual Ether
98 belongs to. On the receive side, this field is
99 received VNIC packets to different VNIC ports.
100
101 Driver Design
102 ==============
103 Intel OPA VNIC software design is presented in
104 OPA VNIC functionality has a HW dependent comp
105 independent component.
106
107 The support has been added for IB device to al
108 netdev devices. The RDMA netdev supports inter
109 stack thus creating standard network interface
110 netdev device type.
111
112 The HW dependent VNIC functionality is part of
113 implements the verbs to allocate and free the
114 It involves HW resource allocation/management
115 It interfaces with the network stack and imple
116 net_device_ops functions. It expects Omni-Path
117 packets in the transmit path and provides HW a
118 the Omni-Path header from the received packets
119 the network stack. It also implements the RDMA
120
121 The OPA VNIC module implements the HW independ
122 It consists of two parts. The VNIC Ethernet Ma
123 registers itself with IB core as an IB client
124 IB MAD stack. It exchanges the management info
125 Manager (EM) and the VNIC netdev. The VNIC net
126 the OPA_VNIC RDMA netdev devices. It overrides
127 set by HW dependent VNIC driver where required
128 operation. It also handles the encapsulation o
129 Omni-Path header in the transmit path. For eac
130 information required for encapsulation is conf
131 interface. It also passes any control informat
132 by invoking the RDMA netdev control operations
133
134 +-------------------+ +---------------
135 | | | Linux
136 | IB MAD | | Network
137 | | | Stack
138 +-------------------+ +---------------
139 | | |
140 | | |
141 +----------------------------+ |
142 | | |
143 | OPA VNIC Module | |
144 | (OPA VNIC RDMA Netdev | |
145 | & EMA functions) | |
146 | | |
147 +----------------------------+ |
148 | |
149 | |
150 +------------------+ |
151 | IB core | |
152 +------------------+ |
153 | |
154 | |
155 +-------------------------------------
156 |
157 | HFI1 Driver with VNIC support
158 |
159 +-------------------------------------
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.