1 =========================== 1 ===========================
2 drm/i915 Intel GFX Driver 2 drm/i915 Intel GFX Driver
3 =========================== 3 ===========================
4 4
5 The drm/i915 driver supports all (with the exc 5 The drm/i915 driver supports all (with the exception of some very early
6 models) integrated GFX chipsets with both Inte 6 models) integrated GFX chipsets with both Intel display and rendering
7 blocks. This excludes a set of SoC platforms w 7 blocks. This excludes a set of SoC platforms with an SGX rendering unit,
8 those have basic support through the gma500 dr 8 those have basic support through the gma500 drm driver.
9 9
10 Core Driver Infrastructure 10 Core Driver Infrastructure
11 ========================== 11 ==========================
12 12
13 This section covers core driver infrastructure 13 This section covers core driver infrastructure used by both the display
14 and the GEM parts of the driver. 14 and the GEM parts of the driver.
15 15
16 Runtime Power Management 16 Runtime Power Management
17 ------------------------ 17 ------------------------
18 18
19 .. kernel-doc:: drivers/gpu/drm/i915/intel_run 19 .. kernel-doc:: drivers/gpu/drm/i915/intel_runtime_pm.c
20 :doc: runtime pm 20 :doc: runtime pm
21 21
22 .. kernel-doc:: drivers/gpu/drm/i915/intel_run 22 .. kernel-doc:: drivers/gpu/drm/i915/intel_runtime_pm.c
23 :internal: 23 :internal:
24 24
25 .. kernel-doc:: drivers/gpu/drm/i915/intel_unc 25 .. kernel-doc:: drivers/gpu/drm/i915/intel_uncore.c
26 :internal: 26 :internal:
27 27
28 Interrupt Handling 28 Interrupt Handling
29 ------------------ 29 ------------------
30 30
31 .. kernel-doc:: drivers/gpu/drm/i915/i915_irq. 31 .. kernel-doc:: drivers/gpu/drm/i915/i915_irq.c
32 :doc: interrupt handling 32 :doc: interrupt handling
33 33
34 .. kernel-doc:: drivers/gpu/drm/i915/i915_irq. 34 .. kernel-doc:: drivers/gpu/drm/i915/i915_irq.c
35 :functions: intel_irq_init intel_irq_init_h 35 :functions: intel_irq_init intel_irq_init_hw intel_hpd_init
36 36
37 .. kernel-doc:: drivers/gpu/drm/i915/i915_irq. 37 .. kernel-doc:: drivers/gpu/drm/i915/i915_irq.c
38 :functions: intel_runtime_pm_disable_interr 38 :functions: intel_runtime_pm_disable_interrupts
39 39
40 .. kernel-doc:: drivers/gpu/drm/i915/i915_irq. 40 .. kernel-doc:: drivers/gpu/drm/i915/i915_irq.c
41 :functions: intel_runtime_pm_enable_interru 41 :functions: intel_runtime_pm_enable_interrupts
42 42
43 Intel GVT-g Guest Support(vGPU) 43 Intel GVT-g Guest Support(vGPU)
44 ------------------------------- 44 -------------------------------
45 45
46 .. kernel-doc:: drivers/gpu/drm/i915/i915_vgpu 46 .. kernel-doc:: drivers/gpu/drm/i915/i915_vgpu.c
47 :doc: Intel GVT-g guest support 47 :doc: Intel GVT-g guest support
48 48
49 .. kernel-doc:: drivers/gpu/drm/i915/i915_vgpu 49 .. kernel-doc:: drivers/gpu/drm/i915/i915_vgpu.c
50 :internal: 50 :internal:
51 51
52 Intel GVT-g Host Support(vGPU device model) 52 Intel GVT-g Host Support(vGPU device model)
53 ------------------------------------------- 53 -------------------------------------------
54 54
55 .. kernel-doc:: drivers/gpu/drm/i915/intel_gvt 55 .. kernel-doc:: drivers/gpu/drm/i915/intel_gvt.c
56 :doc: Intel GVT-g host support 56 :doc: Intel GVT-g host support
57 57
58 .. kernel-doc:: drivers/gpu/drm/i915/intel_gvt 58 .. kernel-doc:: drivers/gpu/drm/i915/intel_gvt.c
59 :internal: 59 :internal:
60 60
61 Workarounds 61 Workarounds
62 ----------- 62 -----------
63 63
64 .. kernel-doc:: drivers/gpu/drm/i915/gt/intel_ 64 .. kernel-doc:: drivers/gpu/drm/i915/gt/intel_workarounds.c
65 :doc: Hardware workarounds 65 :doc: Hardware workarounds
66 66
67 Display Hardware Handling 67 Display Hardware Handling
68 ========================= 68 =========================
69 69
70 This section covers everything related to the 70 This section covers everything related to the display hardware including
71 the mode setting infrastructure, plane, sprite 71 the mode setting infrastructure, plane, sprite and cursor handling and
72 display, output probing and related topics. 72 display, output probing and related topics.
73 73
74 Mode Setting Infrastructure 74 Mode Setting Infrastructure
75 --------------------------- 75 ---------------------------
76 76
77 The i915 driver is thus far the only DRM drive 77 The i915 driver is thus far the only DRM driver which doesn't use the
78 common DRM helper code to implement mode setti 78 common DRM helper code to implement mode setting sequences. Thus it has
79 its own tailor-made infrastructure for executi 79 its own tailor-made infrastructure for executing a display configuration
80 change. 80 change.
81 81
82 Frontbuffer Tracking 82 Frontbuffer Tracking
83 -------------------- 83 --------------------
84 84
85 .. kernel-doc:: drivers/gpu/drm/i915/display/i 85 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_frontbuffer.c
86 :doc: frontbuffer tracking 86 :doc: frontbuffer tracking
87 87
88 .. kernel-doc:: drivers/gpu/drm/i915/display/i 88 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_frontbuffer.h
89 :internal: 89 :internal:
90 90
91 .. kernel-doc:: drivers/gpu/drm/i915/display/i 91 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_frontbuffer.c
92 :internal: 92 :internal:
93 93
94 Display FIFO Underrun Reporting 94 Display FIFO Underrun Reporting
95 ------------------------------- 95 -------------------------------
96 96
97 .. kernel-doc:: drivers/gpu/drm/i915/display/i 97 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_fifo_underrun.c
98 :doc: fifo underrun handling 98 :doc: fifo underrun handling
99 99
100 .. kernel-doc:: drivers/gpu/drm/i915/display/i 100 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_fifo_underrun.c
101 :internal: 101 :internal:
102 102
103 Plane Configuration 103 Plane Configuration
104 ------------------- 104 -------------------
105 105
106 This section covers plane configuration and co 106 This section covers plane configuration and composition with the primary
107 plane, sprites, cursors and overlays. This inc 107 plane, sprites, cursors and overlays. This includes the infrastructure
108 to do atomic vsync'ed updates of all this stat 108 to do atomic vsync'ed updates of all this state and also tightly coupled
109 topics like watermark setup and computation, f 109 topics like watermark setup and computation, framebuffer compression and
110 panel self refresh. 110 panel self refresh.
111 111
112 Atomic Plane Helpers 112 Atomic Plane Helpers
113 -------------------- 113 --------------------
114 114
115 .. kernel-doc:: drivers/gpu/drm/i915/display/i 115 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_atomic_plane.c
116 :doc: atomic plane helpers 116 :doc: atomic plane helpers
117 117
118 .. kernel-doc:: drivers/gpu/drm/i915/display/i 118 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_atomic_plane.c
119 :internal: 119 :internal:
120 120
121 Asynchronous Page Flip <<
122 ---------------------- <<
123 <<
124 .. kernel-doc:: drivers/gpu/drm/i915/display/i <<
125 :doc: asynchronous flip implementation <<
126 <<
127 Output Probing 121 Output Probing
128 -------------- 122 --------------
129 123
130 This section covers output probing and related 124 This section covers output probing and related infrastructure like the
131 hotplug interrupt storm detection and mitigati 125 hotplug interrupt storm detection and mitigation code. Note that the
132 i915 driver still uses most of the common DRM 126 i915 driver still uses most of the common DRM helper code for output
133 probing, so those sections fully apply. 127 probing, so those sections fully apply.
134 128
135 Hotplug 129 Hotplug
136 ------- 130 -------
137 131
138 .. kernel-doc:: drivers/gpu/drm/i915/display/i 132 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_hotplug.c
139 :doc: Hotplug 133 :doc: Hotplug
140 134
141 .. kernel-doc:: drivers/gpu/drm/i915/display/i 135 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_hotplug.c
142 :internal: 136 :internal:
143 137
144 High Definition Audio 138 High Definition Audio
145 --------------------- 139 ---------------------
146 140
147 .. kernel-doc:: drivers/gpu/drm/i915/display/i 141 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_audio.c
148 :doc: High Definition Audio over HDMI and D 142 :doc: High Definition Audio over HDMI and Display Port
149 143
150 .. kernel-doc:: drivers/gpu/drm/i915/display/i 144 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_audio.c
151 :internal: 145 :internal:
152 146
153 .. kernel-doc:: include/drm/intel/i915_compone !! 147 .. kernel-doc:: include/drm/i915_component.h
154 :internal: 148 :internal:
155 149
156 Intel HDMI LPE Audio Support 150 Intel HDMI LPE Audio Support
157 ---------------------------- 151 ----------------------------
158 152
159 .. kernel-doc:: drivers/gpu/drm/i915/display/i 153 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_lpe_audio.c
160 :doc: LPE Audio integration for HDMI or DP 154 :doc: LPE Audio integration for HDMI or DP playback
161 155
162 .. kernel-doc:: drivers/gpu/drm/i915/display/i 156 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_lpe_audio.c
163 :internal: 157 :internal:
164 158
165 Panel Self Refresh PSR (PSR/SRD) 159 Panel Self Refresh PSR (PSR/SRD)
166 -------------------------------- 160 --------------------------------
167 161
168 .. kernel-doc:: drivers/gpu/drm/i915/display/i 162 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_psr.c
169 :doc: Panel Self Refresh (PSR/SRD) 163 :doc: Panel Self Refresh (PSR/SRD)
170 164
171 .. kernel-doc:: drivers/gpu/drm/i915/display/i 165 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_psr.c
172 :internal: 166 :internal:
173 167
174 Frame Buffer Compression (FBC) 168 Frame Buffer Compression (FBC)
175 ------------------------------ 169 ------------------------------
176 170
177 .. kernel-doc:: drivers/gpu/drm/i915/display/i 171 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_fbc.c
178 :doc: Frame Buffer Compression (FBC) 172 :doc: Frame Buffer Compression (FBC)
179 173
180 .. kernel-doc:: drivers/gpu/drm/i915/display/i 174 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_fbc.c
181 :internal: 175 :internal:
182 176
183 Display Refresh Rate Switching (DRRS) 177 Display Refresh Rate Switching (DRRS)
184 ------------------------------------- 178 -------------------------------------
185 179
186 .. kernel-doc:: drivers/gpu/drm/i915/display/i !! 180 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_dp.c
187 :doc: Display Refresh Rate Switching (DRRS) 181 :doc: Display Refresh Rate Switching (DRRS)
188 182
189 .. kernel-doc:: drivers/gpu/drm/i915/display/i !! 183 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_dp.c
190 :internal: !! 184 :functions: intel_dp_set_drrs_state
>> 185
>> 186 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_dp.c
>> 187 :functions: intel_edp_drrs_enable
>> 188
>> 189 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_dp.c
>> 190 :functions: intel_edp_drrs_disable
>> 191
>> 192 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_dp.c
>> 193 :functions: intel_edp_drrs_invalidate
>> 194
>> 195 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_dp.c
>> 196 :functions: intel_edp_drrs_flush
>> 197
>> 198 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_dp.c
>> 199 :functions: intel_dp_drrs_init
191 200
192 DPIO 201 DPIO
193 ---- 202 ----
194 203
195 .. kernel-doc:: drivers/gpu/drm/i915/display/i 204 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_dpio_phy.c
196 :doc: DPIO 205 :doc: DPIO
197 206
198 DMC Firmware Support !! 207 CSR firmware support for DMC
199 -------------------- !! 208 ----------------------------
200 209
201 .. kernel-doc:: drivers/gpu/drm/i915/display/i !! 210 .. kernel-doc:: drivers/gpu/drm/i915/intel_csr.c
202 :doc: DMC Firmware Support !! 211 :doc: csr support for dmc
203 212
204 .. kernel-doc:: drivers/gpu/drm/i915/display/i !! 213 .. kernel-doc:: drivers/gpu/drm/i915/intel_csr.c
205 :internal: 214 :internal:
206 215
207 DMC wakelock support <<
208 -------------------- <<
209 <<
210 .. kernel-doc:: drivers/gpu/drm/i915/display/i <<
211 :doc: DMC wakelock support <<
212 <<
213 Video BIOS Table (VBT) 216 Video BIOS Table (VBT)
214 ---------------------- 217 ----------------------
215 218
216 .. kernel-doc:: drivers/gpu/drm/i915/display/i 219 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_bios.c
217 :doc: Video BIOS Table (VBT) 220 :doc: Video BIOS Table (VBT)
218 221
219 .. kernel-doc:: drivers/gpu/drm/i915/display/i 222 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_bios.c
220 :internal: 223 :internal:
221 224
222 .. kernel-doc:: drivers/gpu/drm/i915/display/i 225 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_vbt_defs.h
223 :internal: 226 :internal:
224 227
225 Display clocks 228 Display clocks
226 -------------- 229 --------------
227 230
228 .. kernel-doc:: drivers/gpu/drm/i915/display/i 231 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_cdclk.c
229 :doc: CDCLK / RAWCLK 232 :doc: CDCLK / RAWCLK
230 233
231 .. kernel-doc:: drivers/gpu/drm/i915/display/i 234 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_cdclk.c
232 :internal: 235 :internal:
233 236
234 Display PLLs 237 Display PLLs
235 ------------ 238 ------------
236 239
237 .. kernel-doc:: drivers/gpu/drm/i915/display/i 240 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_dpll_mgr.c
238 :doc: Display PLLs 241 :doc: Display PLLs
239 242
240 .. kernel-doc:: drivers/gpu/drm/i915/display/i 243 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_dpll_mgr.c
241 :internal: 244 :internal:
242 245
243 .. kernel-doc:: drivers/gpu/drm/i915/display/i 246 .. kernel-doc:: drivers/gpu/drm/i915/display/intel_dpll_mgr.h
244 :internal: 247 :internal:
245 248
246 Display State Buffer <<
247 -------------------- <<
248 <<
249 .. kernel-doc:: drivers/gpu/drm/i915/display/i <<
250 :doc: DSB <<
251 <<
252 .. kernel-doc:: drivers/gpu/drm/i915/display/i <<
253 :internal: <<
254 <<
255 GT Programming <<
256 ============== <<
257 <<
258 Multicast/Replicated (MCR) Registers <<
259 ------------------------------------ <<
260 <<
261 .. kernel-doc:: drivers/gpu/drm/i915/gt/intel_ <<
262 :doc: GT Multicast/Replicated (MCR) Registe <<
263 <<
264 .. kernel-doc:: drivers/gpu/drm/i915/gt/intel_ <<
265 :internal: <<
266 <<
267 Memory Management and Command Submission 249 Memory Management and Command Submission
268 ======================================== 250 ========================================
269 251
270 This sections covers all things related to the 252 This sections covers all things related to the GEM implementation in the
271 i915 driver. 253 i915 driver.
272 254
273 Intel GPU Basics 255 Intel GPU Basics
274 ---------------- 256 ----------------
275 257
276 An Intel GPU has multiple engines. There are s !! 258 An Intel GPU has multiple engines. There are several engine types.
277 259
278 - Render Command Streamer (RCS). An engine for !! 260 - RCS engine is for rendering 3D and performing compute, this is named
279 performing compute. !! 261 `I915_EXEC_RENDER` in user space.
280 - Blitting Command Streamer (BCS). An engine f !! 262 - BCS is a blitting (copy) engine, this is named `I915_EXEC_BLT` in user
281 copying operations. !! 263 space.
282 - Video Command Streamer. An engine used for v !! 264 - VCS is a video encode and decode engine, this is named `I915_EXEC_BSD`
283 sometimes called 'BSD' in hardware documenta !! 265 in user space
284 - Video Enhancement Command Streamer (VECS). A !! 266 - VECS is video enhancement engine, this is named `I915_EXEC_VEBOX` in user
285 Also sometimes called 'VEBOX' in hardware do !! 267 space.
286 - Compute Command Streamer (CCS). An engine th !! 268 - The enumeration `I915_EXEC_DEFAULT` does not refer to specific engine;
287 GPGPU pipelines, but not the 3D pipeline. !! 269 instead it is to be used by user space to specify a default rendering
288 - Graphics Security Controller (GSCCS). A dedi !! 270 engine (for 3D) that may or may not be the same as RCS.
289 communication with GSC controller on securit <<
290 High-bandwidth Digital Content Protection (H <<
291 and HuC firmware authentication. <<
292 271
293 The Intel GPU family is a family of integrated 272 The Intel GPU family is a family of integrated GPU's using Unified
294 Memory Access. For having the GPU "do work", u 273 Memory Access. For having the GPU "do work", user space will feed the
295 GPU batch buffers via one of the ioctls `DRM_I 274 GPU batch buffers via one of the ioctls `DRM_IOCTL_I915_GEM_EXECBUFFER2`
296 or `DRM_IOCTL_I915_GEM_EXECBUFFER2_WR`. Most s 275 or `DRM_IOCTL_I915_GEM_EXECBUFFER2_WR`. Most such batchbuffers will
297 instruct the GPU to perform work (for example 276 instruct the GPU to perform work (for example rendering) and that work
298 needs memory from which to read and memory to 277 needs memory from which to read and memory to which to write. All memory
299 is encapsulated within GEM buffer objects (usu 278 is encapsulated within GEM buffer objects (usually created with the ioctl
300 `DRM_IOCTL_I915_GEM_CREATE`). An ioctl providi 279 `DRM_IOCTL_I915_GEM_CREATE`). An ioctl providing a batchbuffer for the GPU
301 to create will also list all GEM buffer object 280 to create will also list all GEM buffer objects that the batchbuffer reads
302 and/or writes. For implementation details of m 281 and/or writes. For implementation details of memory management see
303 `GEM BO Management Implementation Details`_. 282 `GEM BO Management Implementation Details`_.
304 283
305 The i915 driver allows user space to create a 284 The i915 driver allows user space to create a context via the ioctl
306 `DRM_IOCTL_I915_GEM_CONTEXT_CREATE` which is i 285 `DRM_IOCTL_I915_GEM_CONTEXT_CREATE` which is identified by a 32-bit
307 integer. Such a context should be viewed by us 286 integer. Such a context should be viewed by user-space as -loosely-
308 analogous to the idea of a CPU process of an o 287 analogous to the idea of a CPU process of an operating system. The i915
309 driver guarantees that commands issued to a fi 288 driver guarantees that commands issued to a fixed context are to be
310 executed so that writes of a previously issued 289 executed so that writes of a previously issued command are seen by
311 reads of following commands. Actions issued be 290 reads of following commands. Actions issued between different contexts
312 (even if from the same file descriptor) are NO 291 (even if from the same file descriptor) are NOT given that guarantee
313 and the only way to synchronize across context 292 and the only way to synchronize across contexts (even from the same
314 file descriptor) is through the use of fences. 293 file descriptor) is through the use of fences. At least as far back as
315 Gen4, also have that a context carries with it 294 Gen4, also have that a context carries with it a GPU HW context;
316 the HW context is essentially (most of at leas !! 295 the HW context is essentially (most of atleast) the state of a GPU.
317 In addition to the ordering guarantees, the ke 296 In addition to the ordering guarantees, the kernel will restore GPU
318 state via HW context when commands are issued 297 state via HW context when commands are issued to a context, this saves
319 user space the need to restore (most of at lea !! 298 user space the need to restore (most of atleast) the GPU state at the
320 start of each batchbuffer. The non-deprecated 299 start of each batchbuffer. The non-deprecated ioctls to submit batchbuffer
321 work can pass that ID (in the lower bits of dr 300 work can pass that ID (in the lower bits of drm_i915_gem_execbuffer2::rsvd1)
322 to identify what context to use with the comma 301 to identify what context to use with the command.
323 302
324 The GPU has its own memory management and addr 303 The GPU has its own memory management and address space. The kernel
325 driver maintains the memory translation table 304 driver maintains the memory translation table for the GPU. For older
326 GPUs (i.e. those before Gen8), there is a sing 305 GPUs (i.e. those before Gen8), there is a single global such translation
327 table, a global Graphics Translation Table (GT 306 table, a global Graphics Translation Table (GTT). For newer generation
328 GPUs each context has its own translation tabl 307 GPUs each context has its own translation table, called Per-Process
329 Graphics Translation Table (PPGTT). Of importa 308 Graphics Translation Table (PPGTT). Of important note, is that although
330 PPGTT is named per-process it is actually per 309 PPGTT is named per-process it is actually per context. When user space
331 submits a batchbuffer, the kernel walks the li 310 submits a batchbuffer, the kernel walks the list of GEM buffer objects
332 used by the batchbuffer and guarantees that no 311 used by the batchbuffer and guarantees that not only is the memory of
333 each such GEM buffer object resident but it is 312 each such GEM buffer object resident but it is also present in the
334 (PP)GTT. If the GEM buffer object is not yet p 313 (PP)GTT. If the GEM buffer object is not yet placed in the (PP)GTT,
335 then it is given an address. Two consequences 314 then it is given an address. Two consequences of this are: the kernel
336 needs to edit the batchbuffer submitted to wri 315 needs to edit the batchbuffer submitted to write the correct value of
337 the GPU address when a GEM BO is assigned a GP 316 the GPU address when a GEM BO is assigned a GPU address and the kernel
338 might evict a different GEM BO from the (PP)GT 317 might evict a different GEM BO from the (PP)GTT to make address room
339 for another GEM BO. Consequently, the ioctls s 318 for another GEM BO. Consequently, the ioctls submitting a batchbuffer
340 for execution also include a list of all locat 319 for execution also include a list of all locations within buffers that
341 refer to GPU-addresses so that the kernel can 320 refer to GPU-addresses so that the kernel can edit the buffer correctly.
342 This process is dubbed relocation. 321 This process is dubbed relocation.
343 322
344 Locking Guidelines <<
345 ------------------ <<
346 <<
347 .. note:: <<
348 This is a description of how the locking sh <<
349 refactoring is done. Does not necessarily r <<
350 looks like while WIP. <<
351 <<
352 #. All locking rules and interface contracts w <<
353 (dma-buf, dma_fence) need to be followed. <<
354 <<
355 #. No struct_mutex anywhere in the code <<
356 <<
357 #. dma_resv will be the outermost lock (when n <<
358 is to be hoisted at highest level and passe <<
359 in the call chain <<
360 <<
361 #. While holding lru/memory manager (buddy, dr <<
362 system memory allocations are not allowed <<
363 <<
364 * Enforce this by priming lockdep (wit <<
365 allocate memory while holding these <<
366 of the shrinker vs. struct_mutex sag <<
367 real bad. <<
368 <<
369 #. Do not nest different lru/memory manager lo <<
370 Take them in turn to update memory allocati <<
371 dma_resv ww_mutex to serialize against othe <<
372 <<
373 #. The suggestion for lru/memory managers lock <<
374 enough to be spinlocks. <<
375 <<
376 #. All features need to come with exhaustive k <<
377 IGT tests when appropriate <<
378 <<
379 #. All LMEM uAPI paths need to be fully restar <<
380 for all locks/waits/sleeps) <<
381 <<
382 * Error handling validation through si <<
383 Still the best strategy we have for <<
384 corner cases. <<
385 Must be excessively used in the IGT, <<
386 that we really have full path covera <<
387 <<
388 * -EDEADLK handling with ww_mutex <<
389 <<
390 GEM BO Management Implementation Details 323 GEM BO Management Implementation Details
391 ---------------------------------------- 324 ----------------------------------------
392 325
393 .. kernel-doc:: drivers/gpu/drm/i915/i915_vma_ !! 326 .. kernel-doc:: drivers/gpu/drm/i915/i915_vma.h
394 :doc: Virtual Memory Address 327 :doc: Virtual Memory Address
395 328
396 Buffer Object Eviction 329 Buffer Object Eviction
397 ---------------------- 330 ----------------------
398 331
399 This section documents the interface functions 332 This section documents the interface functions for evicting buffer
400 objects to make space available in the virtual 333 objects to make space available in the virtual gpu address spaces. Note
401 that this is mostly orthogonal to shrinking bu 334 that this is mostly orthogonal to shrinking buffer objects caches, which
402 has the goal to make main memory (shared with 335 has the goal to make main memory (shared with the gpu through the
403 unified memory architecture) available. 336 unified memory architecture) available.
404 337
405 .. kernel-doc:: drivers/gpu/drm/i915/i915_gem_ 338 .. kernel-doc:: drivers/gpu/drm/i915/i915_gem_evict.c
406 :internal: 339 :internal:
407 340
408 Buffer Object Memory Shrinking 341 Buffer Object Memory Shrinking
409 ------------------------------ 342 ------------------------------
410 343
411 This section documents the interface function 344 This section documents the interface function for shrinking memory usage
412 of buffer object caches. Shrinking is used to 345 of buffer object caches. Shrinking is used to make main memory
413 available. Note that this is mostly orthogonal 346 available. Note that this is mostly orthogonal to evicting buffer
414 objects, which has the goal to make space in g 347 objects, which has the goal to make space in gpu virtual address spaces.
415 348
416 .. kernel-doc:: drivers/gpu/drm/i915/gem/i915_ 349 .. kernel-doc:: drivers/gpu/drm/i915/gem/i915_gem_shrinker.c
417 :internal: 350 :internal:
418 351
419 Batchbuffer Parsing 352 Batchbuffer Parsing
420 ------------------- 353 -------------------
421 354
422 .. kernel-doc:: drivers/gpu/drm/i915/i915_cmd_ 355 .. kernel-doc:: drivers/gpu/drm/i915/i915_cmd_parser.c
423 :doc: batch buffer command parser 356 :doc: batch buffer command parser
424 357
425 .. kernel-doc:: drivers/gpu/drm/i915/i915_cmd_ 358 .. kernel-doc:: drivers/gpu/drm/i915/i915_cmd_parser.c
426 :internal: 359 :internal:
427 360
>> 361 Batchbuffer Pools
>> 362 -----------------
>> 363
>> 364 .. kernel-doc:: drivers/gpu/drm/i915/i915_gem_batch_pool.c
>> 365 :doc: batch pool
>> 366
>> 367 .. kernel-doc:: drivers/gpu/drm/i915/i915_gem_batch_pool.c
>> 368 :internal:
>> 369
428 User Batchbuffer Execution 370 User Batchbuffer Execution
429 -------------------------- 371 --------------------------
430 372
431 .. kernel-doc:: drivers/gpu/drm/i915/gem/i915_ <<
432 <<
433 .. kernel-doc:: drivers/gpu/drm/i915/gem/i915_ 373 .. kernel-doc:: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
434 :doc: User command execution 374 :doc: User command execution
435 375
436 Scheduling <<
437 ---------- <<
438 .. kernel-doc:: drivers/gpu/drm/i915/i915_sche <<
439 :functions: i915_sched_engine <<
440 <<
441 Logical Rings, Logical Ring Contexts and Execl 376 Logical Rings, Logical Ring Contexts and Execlists
442 ---------------------------------------------- 377 --------------------------------------------------
443 378
444 .. kernel-doc:: drivers/gpu/drm/i915/gt/intel_ !! 379 .. kernel-doc:: drivers/gpu/drm/i915/gt/intel_lrc.c
445 :doc: Logical Rings, Logical Ring Contexts 380 :doc: Logical Rings, Logical Ring Contexts and Execlists
446 381
447 Global GTT views 382 Global GTT views
448 ---------------- 383 ----------------
449 384
450 .. kernel-doc:: drivers/gpu/drm/i915/i915_vma_ !! 385 .. kernel-doc:: drivers/gpu/drm/i915/i915_gem_gtt.c
451 :doc: Global GTT views 386 :doc: Global GTT views
452 387
453 .. kernel-doc:: drivers/gpu/drm/i915/i915_gem_ 388 .. kernel-doc:: drivers/gpu/drm/i915/i915_gem_gtt.c
454 :internal: 389 :internal:
455 390
456 GTT Fences and Swizzling 391 GTT Fences and Swizzling
457 ------------------------ 392 ------------------------
458 393
459 .. kernel-doc:: drivers/gpu/drm/i915/gt/intel_ !! 394 .. kernel-doc:: drivers/gpu/drm/i915/i915_gem_fence_reg.c
460 :internal: 395 :internal:
461 396
462 Global GTT Fence Handling 397 Global GTT Fence Handling
463 ~~~~~~~~~~~~~~~~~~~~~~~~~ 398 ~~~~~~~~~~~~~~~~~~~~~~~~~
464 399
465 .. kernel-doc:: drivers/gpu/drm/i915/gt/intel_ !! 400 .. kernel-doc:: drivers/gpu/drm/i915/i915_gem_fence_reg.c
466 :doc: fence register handling 401 :doc: fence register handling
467 402
468 Hardware Tiling and Swizzling Details 403 Hardware Tiling and Swizzling Details
469 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 404 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
470 405
471 .. kernel-doc:: drivers/gpu/drm/i915/gt/intel_ !! 406 .. kernel-doc:: drivers/gpu/drm/i915/i915_gem_fence_reg.c
472 :doc: tiling swizzling details 407 :doc: tiling swizzling details
473 408
474 Object Tiling IOCTLs 409 Object Tiling IOCTLs
475 -------------------- 410 --------------------
476 411
477 .. kernel-doc:: drivers/gpu/drm/i915/gem/i915_ 412 .. kernel-doc:: drivers/gpu/drm/i915/gem/i915_gem_tiling.c
478 :internal: 413 :internal:
479 414
480 .. kernel-doc:: drivers/gpu/drm/i915/gem/i915_ 415 .. kernel-doc:: drivers/gpu/drm/i915/gem/i915_gem_tiling.c
481 :doc: buffer object tiling 416 :doc: buffer object tiling
482 417
483 Protected Objects <<
484 ----------------- <<
485 <<
486 .. kernel-doc:: drivers/gpu/drm/i915/pxp/intel <<
487 :doc: PXP <<
488 <<
489 .. kernel-doc:: drivers/gpu/drm/i915/pxp/intel <<
490 <<
491 Microcontrollers <<
492 ================ <<
493 <<
494 Starting from gen9, three microcontrollers are <<
495 graphics microcontroller (GuC), the HEVC/H.265 <<
496 display microcontroller (DMC). The driver is r <<
497 firmwares on the microcontrollers; the GuC and <<
498 to WOPCM using the DMA engine, while the DMC f <<
499 <<
500 WOPCM 418 WOPCM
501 ----- !! 419 =====
502 420
503 WOPCM Layout 421 WOPCM Layout
504 ~~~~~~~~~~~~ !! 422 ------------
505 423
506 .. kernel-doc:: drivers/gpu/drm/i915/gt/intel_ !! 424 .. kernel-doc:: drivers/gpu/drm/i915/intel_wopcm.c
507 :doc: WOPCM Layout 425 :doc: WOPCM Layout
508 426
509 GuC 427 GuC
510 --- !! 428 ===
511 429
512 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/int !! 430 Firmware Layout
513 :doc: GuC !! 431 -------------------
514 <<
515 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/int <<
516 <<
517 GuC Firmware Layout <<
518 ~~~~~~~~~~~~~~~~~~~ <<
519 432
520 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/int 433 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_uc_fw_abi.h
521 :doc: Firmware Layout 434 :doc: Firmware Layout
522 435
523 GuC Memory Management <<
524 ~~~~~~~~~~~~~~~~~~~~~ <<
525 <<
526 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/int <<
527 :doc: GuC Memory Management <<
528 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/int <<
529 :functions: intel_guc_allocate_vma <<
530 <<
531 <<
532 GuC-specific firmware loader 436 GuC-specific firmware loader
533 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ !! 437 ----------------------------
534 438
535 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/int 439 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_guc_fw.c
536 :internal: 440 :internal:
537 441
538 GuC-based command submission 442 GuC-based command submission
539 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ !! 443 ----------------------------
540 444
541 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/int 445 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
542 :doc: GuC-based command submission 446 :doc: GuC-based command submission
543 447
544 GuC ABI !! 448 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
545 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ !! 449 :internal:
>> 450
>> 451 GuC Address Space
>> 452 -----------------
546 453
547 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/abi !! 454 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_guc.c
548 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/abi !! 455 :doc: GuC Address Space
549 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/abi <<
550 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/abi <<
551 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/abi <<
552 <<
553 HuC <<
554 --- <<
555 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/int <<
556 :doc: HuC <<
557 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/int <<
558 :functions: intel_huc_auth <<
559 <<
560 HuC Memory Management <<
561 ~~~~~~~~~~~~~~~~~~~~~ <<
562 <<
563 .. kernel-doc:: drivers/gpu/drm/i915/gt/uc/int <<
564 :doc: HuC Memory Management <<
565 <<
566 HuC Firmware Layout <<
567 ~~~~~~~~~~~~~~~~~~~ <<
568 The HuC FW layout is the same as the GuC one, <<
569 <<
570 DMC <<
571 --- <<
572 See `DMC Firmware Support`_ <<
573 456
574 Tracing 457 Tracing
575 ======= 458 =======
576 459
577 This sections covers all things related to the 460 This sections covers all things related to the tracepoints implemented
578 in the i915 driver. 461 in the i915 driver.
579 462
580 i915_ppgtt_create and i915_ppgtt_release 463 i915_ppgtt_create and i915_ppgtt_release
581 ---------------------------------------- 464 ----------------------------------------
582 465
583 .. kernel-doc:: drivers/gpu/drm/i915/i915_trac 466 .. kernel-doc:: drivers/gpu/drm/i915/i915_trace.h
584 :doc: i915_ppgtt_create and i915_ppgtt_rele 467 :doc: i915_ppgtt_create and i915_ppgtt_release tracepoints
585 468
586 i915_context_create and i915_context_free 469 i915_context_create and i915_context_free
587 ----------------------------------------- 470 -----------------------------------------
588 471
589 .. kernel-doc:: drivers/gpu/drm/i915/i915_trac 472 .. kernel-doc:: drivers/gpu/drm/i915/i915_trace.h
590 :doc: i915_context_create and i915_context_ 473 :doc: i915_context_create and i915_context_free tracepoints
591 474
592 Perf 475 Perf
593 ==== 476 ====
594 477
595 Overview 478 Overview
596 -------- 479 --------
597 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 480 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
598 :doc: i915 Perf Overview 481 :doc: i915 Perf Overview
599 482
600 Comparison with Core Perf 483 Comparison with Core Perf
601 ------------------------- 484 -------------------------
602 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 485 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
603 :doc: i915 Perf History and Comparison with 486 :doc: i915 Perf History and Comparison with Core Perf
604 487
605 i915 Driver Entry Points 488 i915 Driver Entry Points
606 ------------------------ 489 ------------------------
607 490
608 This section covers the entrypoints exported o 491 This section covers the entrypoints exported outside of i915_perf.c to
609 integrate with drm/i915 and to handle the `DRM 492 integrate with drm/i915 and to handle the `DRM_I915_PERF_OPEN` ioctl.
610 493
611 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 494 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
612 :functions: i915_perf_init 495 :functions: i915_perf_init
613 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 496 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
614 :functions: i915_perf_fini 497 :functions: i915_perf_fini
615 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 498 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
616 :functions: i915_perf_register 499 :functions: i915_perf_register
617 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 500 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
618 :functions: i915_perf_unregister 501 :functions: i915_perf_unregister
619 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 502 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
620 :functions: i915_perf_open_ioctl 503 :functions: i915_perf_open_ioctl
621 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 504 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
622 :functions: i915_perf_release 505 :functions: i915_perf_release
623 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 506 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
624 :functions: i915_perf_add_config_ioctl 507 :functions: i915_perf_add_config_ioctl
625 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 508 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
626 :functions: i915_perf_remove_config_ioctl 509 :functions: i915_perf_remove_config_ioctl
627 510
628 i915 Perf Stream 511 i915 Perf Stream
629 ---------------- 512 ----------------
630 513
631 This section covers the stream-semantics-agnos 514 This section covers the stream-semantics-agnostic structures and functions
632 for representing an i915 perf stream FD and as 515 for representing an i915 perf stream FD and associated file operations.
633 516
634 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf !! 517 .. kernel-doc:: drivers/gpu/drm/i915/i915_drv.h
635 :functions: i915_perf_stream 518 :functions: i915_perf_stream
636 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf !! 519 .. kernel-doc:: drivers/gpu/drm/i915/i915_drv.h
637 :functions: i915_perf_stream_ops 520 :functions: i915_perf_stream_ops
638 521
639 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 522 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
640 :functions: read_properties_unlocked 523 :functions: read_properties_unlocked
641 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 524 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
642 :functions: i915_perf_open_ioctl_locked 525 :functions: i915_perf_open_ioctl_locked
643 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 526 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
644 :functions: i915_perf_destroy_locked 527 :functions: i915_perf_destroy_locked
645 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 528 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
646 :functions: i915_perf_read 529 :functions: i915_perf_read
647 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 530 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
648 :functions: i915_perf_ioctl 531 :functions: i915_perf_ioctl
649 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 532 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
650 :functions: i915_perf_enable_locked 533 :functions: i915_perf_enable_locked
651 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 534 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
652 :functions: i915_perf_disable_locked 535 :functions: i915_perf_disable_locked
653 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 536 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
654 :functions: i915_perf_poll 537 :functions: i915_perf_poll
655 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 538 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
656 :functions: i915_perf_poll_locked 539 :functions: i915_perf_poll_locked
657 540
658 i915 Perf Observation Architecture Stream 541 i915 Perf Observation Architecture Stream
659 ----------------------------------------- 542 -----------------------------------------
660 543
661 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf !! 544 .. kernel-doc:: drivers/gpu/drm/i915/i915_drv.h
662 :functions: i915_oa_ops 545 :functions: i915_oa_ops
663 546
664 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 547 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
665 :functions: i915_oa_stream_init 548 :functions: i915_oa_stream_init
666 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 549 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
667 :functions: i915_oa_read 550 :functions: i915_oa_read
668 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 551 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
669 :functions: i915_oa_stream_enable 552 :functions: i915_oa_stream_enable
670 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 553 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
671 :functions: i915_oa_stream_disable 554 :functions: i915_oa_stream_disable
672 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 555 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
673 :functions: i915_oa_wait_unlocked 556 :functions: i915_oa_wait_unlocked
674 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 557 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
675 :functions: i915_oa_poll_wait 558 :functions: i915_oa_poll_wait
676 559
677 Other i915 Perf Internals !! 560 All i915 Perf Internals
678 ------------------------- !! 561 -----------------------
679 562
680 This section simply includes all other current !! 563 This section simply includes all currently documented i915 perf internals, in
681 in no particular order, but may include some m !! 564 no particular order, but may include some more minor utilities or platform
682 specific details than found in the more high-l 565 specific details than found in the more high-level sections.
683 566
684 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf 567 .. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
685 :internal: 568 :internal:
686 :no-identifiers: <<
687 i915_perf_init <<
688 i915_perf_fini <<
689 i915_perf_register <<
690 i915_perf_unregister <<
691 i915_perf_open_ioctl <<
692 i915_perf_release <<
693 i915_perf_add_config_ioctl <<
694 i915_perf_remove_config_ioctl <<
695 read_properties_unlocked <<
696 i915_perf_open_ioctl_locked <<
697 i915_perf_destroy_locked <<
698 i915_perf_read i915_perf_ioctl <<
699 i915_perf_enable_locked <<
700 i915_perf_disable_locked <<
701 i915_perf_poll i915_perf_poll_locked <<
702 i915_oa_stream_init i915_oa_read <<
703 i915_oa_stream_enable <<
704 i915_oa_stream_disable <<
705 i915_oa_wait_unlocked <<
706 i915_oa_poll_wait <<
707 569
708 Style 570 Style
709 ===== 571 =====
710 572
711 The drm/i915 driver codebase has some style ru 573 The drm/i915 driver codebase has some style rules in addition to (and, in some
712 cases, deviating from) the kernel coding style 574 cases, deviating from) the kernel coding style.
713 575
714 Register macro definition style 576 Register macro definition style
715 ------------------------------- 577 -------------------------------
716 578
717 The style guide for ``i915_reg.h``. 579 The style guide for ``i915_reg.h``.
718 580
719 .. kernel-doc:: drivers/gpu/drm/i915/i915_reg. 581 .. kernel-doc:: drivers/gpu/drm/i915/i915_reg.h
720 :doc: The i915 register macro definition st 582 :doc: The i915 register macro definition style guide
721 <<
722 .. _i915-usage-stats: <<
723 <<
724 i915 DRM client usage stats implementation <<
725 ========================================== <<
726 <<
727 The drm/i915 driver implements the DRM client <<
728 documented in :ref:`drm-client-usage-stats`. <<
729 <<
730 Example of the output showing the implemented <<
731 the currently possible format options: <<
732 <<
733 :: <<
734 <<
735 pos: 0 <<
736 flags: 0100002 <<
737 mnt_id: 21 <<
738 drm-driver: i915 <<
739 drm-pdev: 0000:00:02.0 <<
740 drm-client-id: 7 <<
741 drm-engine-render: 9288864723 ns <<
742 drm-engine-copy: 2035071108 ns <<
743 drm-engine-video: 0 ns <<
744 drm-engine-capacity-video: 2 <<
745 drm-engine-video-enhance: 0 ns <<
746 <<
747 Possible `drm-engine-` key names are: `render` <<
748 `video-enhance`. <<
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.