1 .. SPDX-License-Identifier: GPL-2.0 1 .. SPDX-License-Identifier: GPL-2.0
2 2
3 i.MX Video Capture Driver 3 i.MX Video Capture Driver
4 ========================= 4 =========================
5 5
6 Introduction 6 Introduction
7 ------------ 7 ------------
8 8
9 The Freescale i.MX5/6 contains an Image Proces 9 The Freescale i.MX5/6 contains an Image Processing Unit (IPU), which
10 handles the flow of image frames to and from c 10 handles the flow of image frames to and from capture devices and
11 display devices. 11 display devices.
12 12
13 For image capture, the IPU contains the follow 13 For image capture, the IPU contains the following internal subunits:
14 14
15 - Image DMA Controller (IDMAC) 15 - Image DMA Controller (IDMAC)
16 - Camera Serial Interface (CSI) 16 - Camera Serial Interface (CSI)
17 - Image Converter (IC) 17 - Image Converter (IC)
18 - Sensor Multi-FIFO Controller (SMFC) 18 - Sensor Multi-FIFO Controller (SMFC)
19 - Image Rotator (IRT) 19 - Image Rotator (IRT)
20 - Video De-Interlacing or Combining Block (VDI 20 - Video De-Interlacing or Combining Block (VDIC)
21 21
22 The IDMAC is the DMA controller for transfer o 22 The IDMAC is the DMA controller for transfer of image frames to and from
23 memory. Various dedicated DMA channels exist f 23 memory. Various dedicated DMA channels exist for both video capture and
24 display paths. During transfer, the IDMAC is a 24 display paths. During transfer, the IDMAC is also capable of vertical
25 image flip, 8x8 block transfer (see IRT descri 25 image flip, 8x8 block transfer (see IRT description), pixel component
26 re-ordering (for example UYVY to YUYV) within 26 re-ordering (for example UYVY to YUYV) within the same colorspace, and
27 packed <--> planar conversion. The IDMAC can a 27 packed <--> planar conversion. The IDMAC can also perform a simple
28 de-interlacing by interweaving even and odd li 28 de-interlacing by interweaving even and odd lines during transfer
29 (without motion compensation which requires th 29 (without motion compensation which requires the VDIC).
30 30
31 The CSI is the backend capture unit that inter 31 The CSI is the backend capture unit that interfaces directly with
32 camera sensors over Parallel, BT.656/1120, and 32 camera sensors over Parallel, BT.656/1120, and MIPI CSI-2 buses.
33 33
34 The IC handles color-space conversion, resizin 34 The IC handles color-space conversion, resizing (downscaling and
35 upscaling), horizontal flip, and 90/270 degree 35 upscaling), horizontal flip, and 90/270 degree rotation operations.
36 36
37 There are three independent "tasks" within the 37 There are three independent "tasks" within the IC that can carry out
38 conversions concurrently: pre-process encoding 38 conversions concurrently: pre-process encoding, pre-process viewfinder,
39 and post-processing. Within each task, convers 39 and post-processing. Within each task, conversions are split into three
40 sections: downsizing section, main section (up 40 sections: downsizing section, main section (upsizing, flip, colorspace
41 conversion, and graphics plane combining), and 41 conversion, and graphics plane combining), and rotation section.
42 42
43 The IPU time-shares the IC task operations. Th 43 The IPU time-shares the IC task operations. The time-slice granularity
44 is one burst of eight pixels in the downsizing 44 is one burst of eight pixels in the downsizing section, one image line
45 in the main processing section, one image fram 45 in the main processing section, one image frame in the rotation section.
46 46
47 The SMFC is composed of four independent FIFOs 47 The SMFC is composed of four independent FIFOs that each can transfer
48 captured frames from sensors directly to memor 48 captured frames from sensors directly to memory concurrently via four
49 IDMAC channels. 49 IDMAC channels.
50 50
51 The IRT carries out 90 and 270 degree image ro 51 The IRT carries out 90 and 270 degree image rotation operations. The
52 rotation operation is carried out on 8x8 pixel 52 rotation operation is carried out on 8x8 pixel blocks at a time. This
53 operation is supported by the IDMAC which hand 53 operation is supported by the IDMAC which handles the 8x8 block transfer
54 along with block reordering, in coordination w 54 along with block reordering, in coordination with vertical flip.
55 55
56 The VDIC handles the conversion of interlaced 56 The VDIC handles the conversion of interlaced video to progressive, with
57 support for different motion compensation mode 57 support for different motion compensation modes (low, medium, and high
58 motion). The deinterlaced output frames from t 58 motion). The deinterlaced output frames from the VDIC can be sent to the
59 IC pre-process viewfinder task for further con 59 IC pre-process viewfinder task for further conversions. The VDIC also
60 contains a Combiner that combines two image pl 60 contains a Combiner that combines two image planes, with alpha blending
61 and color keying. 61 and color keying.
62 62
63 In addition to the IPU internal subunits, ther 63 In addition to the IPU internal subunits, there are also two units
64 outside the IPU that are also involved in vide 64 outside the IPU that are also involved in video capture on i.MX:
65 65
66 - MIPI CSI-2 Receiver for camera sensors with 66 - MIPI CSI-2 Receiver for camera sensors with the MIPI CSI-2 bus
67 interface. This is a Synopsys DesignWare cor 67 interface. This is a Synopsys DesignWare core.
68 - Two video multiplexers for selecting among m 68 - Two video multiplexers for selecting among multiple sensor inputs
69 to send to a CSI. 69 to send to a CSI.
70 70
71 For more info, refer to the latest versions of 71 For more info, refer to the latest versions of the i.MX5/6 reference
72 manuals [#f1]_ and [#f2]_. 72 manuals [#f1]_ and [#f2]_.
73 73
74 74
75 Features 75 Features
76 -------- 76 --------
77 77
78 Some of the features of this driver include: 78 Some of the features of this driver include:
79 79
80 - Many different pipelines can be configured v 80 - Many different pipelines can be configured via media controller API,
81 that correspond to the hardware video captur 81 that correspond to the hardware video capture pipelines supported in
82 the i.MX. 82 the i.MX.
83 83
84 - Supports parallel, BT.565, and MIPI CSI-2 in 84 - Supports parallel, BT.565, and MIPI CSI-2 interfaces.
85 85
86 - Concurrent independent streams, by configuri 86 - Concurrent independent streams, by configuring pipelines to multiple
87 video capture interfaces using independent e 87 video capture interfaces using independent entities.
88 88
89 - Scaling, color-space conversion, horizontal 89 - Scaling, color-space conversion, horizontal and vertical flip, and
90 image rotation via IC task subdevs. 90 image rotation via IC task subdevs.
91 91
92 - Many pixel formats supported (RGB, packed an 92 - Many pixel formats supported (RGB, packed and planar YUV, partial
93 planar YUV). 93 planar YUV).
94 94
95 - The VDIC subdev supports motion compensated 95 - The VDIC subdev supports motion compensated de-interlacing, with three
96 motion compensation modes: low, medium, and 96 motion compensation modes: low, medium, and high motion. Pipelines are
97 defined that allow sending frames to the VDI 97 defined that allow sending frames to the VDIC subdev directly from the
98 CSI. There is also support in the future for 98 CSI. There is also support in the future for sending frames to the
99 VDIC from memory buffers via a output/mem2me 99 VDIC from memory buffers via a output/mem2mem devices.
100 100
101 - Includes a Frame Interval Monitor (FIM) that 101 - Includes a Frame Interval Monitor (FIM) that can correct vertical sync
102 problems with the ADV718x video decoders. 102 problems with the ADV718x video decoders.
103 103
104 104
105 Topology 105 Topology
106 -------- 106 --------
107 107
108 The following shows the media topologies for t 108 The following shows the media topologies for the i.MX6Q SabreSD and
109 i.MX6Q SabreAuto. Refer to these diagrams in t 109 i.MX6Q SabreAuto. Refer to these diagrams in the entity descriptions
110 in the next section. 110 in the next section.
111 111
112 The i.MX5/6 topologies can differ upstream fro 112 The i.MX5/6 topologies can differ upstream from the IPUv3 CSI video
113 multiplexers, but the internal IPUv3 topology 113 multiplexers, but the internal IPUv3 topology downstream from there
114 is common to all i.MX5/6 platforms. For exampl 114 is common to all i.MX5/6 platforms. For example, the SabreSD, with the
115 MIPI CSI-2 OV5640 sensor, requires the i.MX6 M 115 MIPI CSI-2 OV5640 sensor, requires the i.MX6 MIPI CSI-2 receiver. But
116 the SabreAuto has only the ADV7180 decoder on 116 the SabreAuto has only the ADV7180 decoder on a parallel bt.656 bus, and
117 therefore does not require the MIPI CSI-2 rece 117 therefore does not require the MIPI CSI-2 receiver, so it is missing in
118 its graph. 118 its graph.
119 119
120 .. _imx6q_topology_graph: 120 .. _imx6q_topology_graph:
121 121
122 .. kernel-figure:: imx6q-sabresd.dot 122 .. kernel-figure:: imx6q-sabresd.dot
123 :alt: Diagram of the i.MX6Q SabreSD medi 123 :alt: Diagram of the i.MX6Q SabreSD media pipeline topology
124 :align: center 124 :align: center
125 125
126 Media pipeline graph on i.MX6Q SabreSD 126 Media pipeline graph on i.MX6Q SabreSD
127 127
128 .. kernel-figure:: imx6q-sabreauto.dot 128 .. kernel-figure:: imx6q-sabreauto.dot
129 :alt: Diagram of the i.MX6Q SabreAuto me 129 :alt: Diagram of the i.MX6Q SabreAuto media pipeline topology
130 :align: center 130 :align: center
131 131
132 Media pipeline graph on i.MX6Q SabreAuto 132 Media pipeline graph on i.MX6Q SabreAuto
133 133
134 Entities 134 Entities
135 -------- 135 --------
136 136
137 imx6-mipi-csi2 137 imx6-mipi-csi2
138 -------------- 138 --------------
139 139
140 This is the MIPI CSI-2 receiver entity. It has 140 This is the MIPI CSI-2 receiver entity. It has one sink pad to receive
141 the MIPI CSI-2 stream (usually from a MIPI CSI 141 the MIPI CSI-2 stream (usually from a MIPI CSI-2 camera sensor). It has
142 four source pads, corresponding to the four MI 142 four source pads, corresponding to the four MIPI CSI-2 demuxed virtual
143 channel outputs. Multiple source pads can be e 143 channel outputs. Multiple source pads can be enabled to independently
144 stream from multiple virtual channels. 144 stream from multiple virtual channels.
145 145
146 This entity actually consists of two sub-block 146 This entity actually consists of two sub-blocks. One is the MIPI CSI-2
147 core. This is a Synopsys Designware MIPI CSI-2 147 core. This is a Synopsys Designware MIPI CSI-2 core. The other sub-block
148 is a "CSI-2 to IPU gasket". The gasket acts as 148 is a "CSI-2 to IPU gasket". The gasket acts as a demultiplexer of the
149 four virtual channels streams, providing four 149 four virtual channels streams, providing four separate parallel buses
150 containing each virtual channel that are route 150 containing each virtual channel that are routed to CSIs or video
151 multiplexers as described below. 151 multiplexers as described below.
152 152
153 On i.MX6 solo/dual-lite, all four virtual chan 153 On i.MX6 solo/dual-lite, all four virtual channel buses are routed to
154 two video multiplexers. Both CSI0 and CSI1 can 154 two video multiplexers. Both CSI0 and CSI1 can receive any virtual
155 channel, as selected by the video multiplexers 155 channel, as selected by the video multiplexers.
156 156
157 On i.MX6 Quad, virtual channel 0 is routed to 157 On i.MX6 Quad, virtual channel 0 is routed to IPU1-CSI0 (after selected
158 by a video mux), virtual channels 1 and 2 are 158 by a video mux), virtual channels 1 and 2 are hard-wired to IPU1-CSI1
159 and IPU2-CSI0, respectively, and virtual chann 159 and IPU2-CSI0, respectively, and virtual channel 3 is routed to
160 IPU2-CSI1 (again selected by a video mux). 160 IPU2-CSI1 (again selected by a video mux).
161 161
162 ipuX_csiY_mux 162 ipuX_csiY_mux
163 ------------- 163 -------------
164 164
165 These are the video multiplexers. They have tw 165 These are the video multiplexers. They have two or more sink pads to
166 select from either camera sensors with a paral 166 select from either camera sensors with a parallel interface, or from
167 MIPI CSI-2 virtual channels from imx6-mipi-csi 167 MIPI CSI-2 virtual channels from imx6-mipi-csi2 entity. They have a
168 single source pad that routes to a CSI (ipuX_c 168 single source pad that routes to a CSI (ipuX_csiY entities).
169 169
170 On i.MX6 solo/dual-lite, there are two video m 170 On i.MX6 solo/dual-lite, there are two video mux entities. One sits
171 in front of IPU1-CSI0 to select between a para 171 in front of IPU1-CSI0 to select between a parallel sensor and any of
172 the four MIPI CSI-2 virtual channels (a total 172 the four MIPI CSI-2 virtual channels (a total of five sink pads). The
173 other mux sits in front of IPU1-CSI1, and agai 173 other mux sits in front of IPU1-CSI1, and again has five sink pads to
174 select between a parallel sensor and any of th 174 select between a parallel sensor and any of the four MIPI CSI-2 virtual
175 channels. 175 channels.
176 176
177 On i.MX6 Quad, there are two video mux entitie 177 On i.MX6 Quad, there are two video mux entities. One sits in front of
178 IPU1-CSI0 to select between a parallel sensor 178 IPU1-CSI0 to select between a parallel sensor and MIPI CSI-2 virtual
179 channel 0 (two sink pads). The other mux sits 179 channel 0 (two sink pads). The other mux sits in front of IPU2-CSI1 to
180 select between a parallel sensor and MIPI CSI- 180 select between a parallel sensor and MIPI CSI-2 virtual channel 3 (two
181 sink pads). 181 sink pads).
182 182
183 ipuX_csiY 183 ipuX_csiY
184 --------- 184 ---------
185 185
186 These are the CSI entities. They have a single 186 These are the CSI entities. They have a single sink pad receiving from
187 either a video mux or from a MIPI CSI-2 virtua 187 either a video mux or from a MIPI CSI-2 virtual channel as described
188 above. 188 above.
189 189
190 This entity has two source pads. The first sou 190 This entity has two source pads. The first source pad can link directly
191 to the ipuX_vdic entity or the ipuX_ic_prp ent 191 to the ipuX_vdic entity or the ipuX_ic_prp entity, using hardware links
192 that require no IDMAC memory buffer transfer. 192 that require no IDMAC memory buffer transfer.
193 193
194 When the direct source pad is routed to the ip 194 When the direct source pad is routed to the ipuX_ic_prp entity, frames
195 from the CSI can be processed by one or both o 195 from the CSI can be processed by one or both of the IC pre-processing
196 tasks. 196 tasks.
197 197
198 When the direct source pad is routed to the ip 198 When the direct source pad is routed to the ipuX_vdic entity, the VDIC
199 will carry out motion-compensated de-interlace 199 will carry out motion-compensated de-interlace using "high motion" mode
200 (see description of ipuX_vdic entity). 200 (see description of ipuX_vdic entity).
201 201
202 The second source pad sends video frames direc 202 The second source pad sends video frames directly to memory buffers
203 via the SMFC and an IDMAC channel, bypassing I 203 via the SMFC and an IDMAC channel, bypassing IC pre-processing. This
204 source pad is routed to a capture device node, 204 source pad is routed to a capture device node, with a node name of the
205 format "ipuX_csiY capture". 205 format "ipuX_csiY capture".
206 206
207 Note that since the IDMAC source pad makes use 207 Note that since the IDMAC source pad makes use of an IDMAC channel,
208 pixel reordering within the same colorspace ca 208 pixel reordering within the same colorspace can be carried out by the
209 IDMAC channel. For example, if the CSI sink pa 209 IDMAC channel. For example, if the CSI sink pad is receiving in UYVY
210 order, the capture device linked to the IDMAC 210 order, the capture device linked to the IDMAC source pad can capture
211 in YUYV order. Also, if the CSI sink pad is re 211 in YUYV order. Also, if the CSI sink pad is receiving a packed YUV
212 format, the capture device can capture a plana 212 format, the capture device can capture a planar YUV format such as
213 YUV420. 213 YUV420.
214 214
215 The IDMAC channel at the IDMAC source pad also 215 The IDMAC channel at the IDMAC source pad also supports simple
216 interweave without motion compensation, which 216 interweave without motion compensation, which is activated if the source
217 pad's field type is sequential top-bottom or b 217 pad's field type is sequential top-bottom or bottom-top, and the
218 requested capture interface field type is set 218 requested capture interface field type is set to interlaced (t-b, b-t,
219 or unqualified interlaced). The capture interf 219 or unqualified interlaced). The capture interface will enforce the same
220 field order as the source pad field order (int 220 field order as the source pad field order (interlaced-bt if source pad
221 is seq-bt, interlaced-tb if source pad is seq- 221 is seq-bt, interlaced-tb if source pad is seq-tb).
222 222
223 For events produced by ipuX_csiY, see ref:`imx 223 For events produced by ipuX_csiY, see ref:`imx_api_ipuX_csiY`.
224 224
225 Cropping in ipuX_csiY 225 Cropping in ipuX_csiY
226 --------------------- 226 ---------------------
227 227
228 The CSI supports cropping the incoming raw sen 228 The CSI supports cropping the incoming raw sensor frames. This is
229 implemented in the ipuX_csiY entities at the s 229 implemented in the ipuX_csiY entities at the sink pad, using the
230 crop selection subdev API. 230 crop selection subdev API.
231 231
232 The CSI also supports fixed divide-by-two down 232 The CSI also supports fixed divide-by-two downscaling independently in
233 width and height. This is implemented in the i 233 width and height. This is implemented in the ipuX_csiY entities at
234 the sink pad, using the compose selection subd 234 the sink pad, using the compose selection subdev API.
235 235
236 The output rectangle at the ipuX_csiY source p 236 The output rectangle at the ipuX_csiY source pad is the same as
237 the compose rectangle at the sink pad. So the 237 the compose rectangle at the sink pad. So the source pad rectangle
238 cannot be negotiated, it must be set using the 238 cannot be negotiated, it must be set using the compose selection
239 API at sink pad (if /2 downscale is desired, o 239 API at sink pad (if /2 downscale is desired, otherwise source pad
240 rectangle is equal to incoming rectangle). 240 rectangle is equal to incoming rectangle).
241 241
242 To give an example of crop and /2 downscale, t 242 To give an example of crop and /2 downscale, this will crop a
243 1280x960 input frame to 640x480, and then /2 d 243 1280x960 input frame to 640x480, and then /2 downscale in both
244 dimensions to 320x240 (assumes ipu1_csi0 is li 244 dimensions to 320x240 (assumes ipu1_csi0 is linked to ipu1_csi0_mux):
245 245
246 .. code-block:: none 246 .. code-block:: none
247 247
248 media-ctl -V "'ipu1_csi0_mux':2[fmt:UYVY2X8 248 media-ctl -V "'ipu1_csi0_mux':2[fmt:UYVY2X8/1280x960]"
249 media-ctl -V "'ipu1_csi0':0[crop:(0,0)/640x 249 media-ctl -V "'ipu1_csi0':0[crop:(0,0)/640x480]"
250 media-ctl -V "'ipu1_csi0':0[compose:(0,0)/3 250 media-ctl -V "'ipu1_csi0':0[compose:(0,0)/320x240]"
251 251
252 Frame Skipping in ipuX_csiY 252 Frame Skipping in ipuX_csiY
253 --------------------------- 253 ---------------------------
254 254
255 The CSI supports frame rate decimation, via fr 255 The CSI supports frame rate decimation, via frame skipping. Frame
256 rate decimation is specified by setting the fr 256 rate decimation is specified by setting the frame intervals at
257 sink and source pads. The ipuX_csiY entity the 257 sink and source pads. The ipuX_csiY entity then applies the best
258 frame skip setting to the CSI to achieve the d 258 frame skip setting to the CSI to achieve the desired frame rate
259 at the source pad. 259 at the source pad.
260 260
261 The following example reduces an assumed incom 261 The following example reduces an assumed incoming 60 Hz frame
262 rate by half at the IDMAC output source pad: 262 rate by half at the IDMAC output source pad:
263 263
264 .. code-block:: none 264 .. code-block:: none
265 265
266 media-ctl -V "'ipu1_csi0':0[fmt:UYVY2X8/640 266 media-ctl -V "'ipu1_csi0':0[fmt:UYVY2X8/640x480@1/60]"
267 media-ctl -V "'ipu1_csi0':2[fmt:UYVY2X8/640 267 media-ctl -V "'ipu1_csi0':2[fmt:UYVY2X8/640x480@1/30]"
268 268
269 Frame Interval Monitor in ipuX_csiY 269 Frame Interval Monitor in ipuX_csiY
270 ----------------------------------- 270 -----------------------------------
271 271
272 See ref:`imx_api_FIM`. 272 See ref:`imx_api_FIM`.
273 273
274 ipuX_vdic 274 ipuX_vdic
275 --------- 275 ---------
276 276
277 The VDIC carries out motion compensated de-int 277 The VDIC carries out motion compensated de-interlacing, with three
278 motion compensation modes: low, medium, and hi 278 motion compensation modes: low, medium, and high motion. The mode is
279 specified with the menu control V4L2_CID_DEINT 279 specified with the menu control V4L2_CID_DEINTERLACING_MODE. The VDIC
280 has two sink pads and a single source pad. 280 has two sink pads and a single source pad.
281 281
282 The direct sink pad receives from an ipuX_csiY 282 The direct sink pad receives from an ipuX_csiY direct pad. With this
283 link the VDIC can only operate in high motion 283 link the VDIC can only operate in high motion mode.
284 284
285 When the IDMAC sink pad is activated, it recei 285 When the IDMAC sink pad is activated, it receives from an output
286 or mem2mem device node. With this pipeline, th 286 or mem2mem device node. With this pipeline, the VDIC can also operate
287 in low and medium modes, because these modes r 287 in low and medium modes, because these modes require receiving
288 frames from memory buffers. Note that an outpu 288 frames from memory buffers. Note that an output or mem2mem device
289 is not implemented yet, so this sink pad curre 289 is not implemented yet, so this sink pad currently has no links.
290 290
291 The source pad routes to the IC pre-processing 291 The source pad routes to the IC pre-processing entity ipuX_ic_prp.
292 292
293 ipuX_ic_prp 293 ipuX_ic_prp
294 ----------- 294 -----------
295 295
296 This is the IC pre-processing entity. It acts 296 This is the IC pre-processing entity. It acts as a router, routing
297 data from its sink pad to one or both of its s 297 data from its sink pad to one or both of its source pads.
298 298
299 This entity has a single sink pad. The sink pa 299 This entity has a single sink pad. The sink pad can receive from the
300 ipuX_csiY direct pad, or from ipuX_vdic. 300 ipuX_csiY direct pad, or from ipuX_vdic.
301 301
302 This entity has two source pads. One source pa 302 This entity has two source pads. One source pad routes to the
303 pre-process encode task entity (ipuX_ic_prpenc 303 pre-process encode task entity (ipuX_ic_prpenc), the other to the
304 pre-process viewfinder task entity (ipuX_ic_pr 304 pre-process viewfinder task entity (ipuX_ic_prpvf). Both source pads
305 can be activated at the same time if the sink 305 can be activated at the same time if the sink pad is receiving from
306 ipuX_csiY. Only the source pad to the pre-proc 306 ipuX_csiY. Only the source pad to the pre-process viewfinder task entity
307 can be activated if the sink pad is receiving 307 can be activated if the sink pad is receiving from ipuX_vdic (frames
308 from the VDIC can only be processed by the pre 308 from the VDIC can only be processed by the pre-process viewfinder task).
309 309
310 ipuX_ic_prpenc 310 ipuX_ic_prpenc
311 -------------- 311 --------------
312 312
313 This is the IC pre-processing encode entity. I 313 This is the IC pre-processing encode entity. It has a single sink
314 pad from ipuX_ic_prp, and a single source pad. 314 pad from ipuX_ic_prp, and a single source pad. The source pad is
315 routed to a capture device node, with a node n 315 routed to a capture device node, with a node name of the format
316 "ipuX_ic_prpenc capture". 316 "ipuX_ic_prpenc capture".
317 317
318 This entity performs the IC pre-process encode 318 This entity performs the IC pre-process encode task operations:
319 color-space conversion, resizing (downscaling 319 color-space conversion, resizing (downscaling and upscaling),
320 horizontal and vertical flip, and 90/270 degre 320 horizontal and vertical flip, and 90/270 degree rotation. Flip
321 and rotation are provided via standard V4L2 co 321 and rotation are provided via standard V4L2 controls.
322 322
323 Like the ipuX_csiY IDMAC source, this entity a 323 Like the ipuX_csiY IDMAC source, this entity also supports simple
324 de-interlace without motion compensation, and 324 de-interlace without motion compensation, and pixel reordering.
325 325
326 ipuX_ic_prpvf 326 ipuX_ic_prpvf
327 ------------- 327 -------------
328 328
329 This is the IC pre-processing viewfinder entit 329 This is the IC pre-processing viewfinder entity. It has a single sink
330 pad from ipuX_ic_prp, and a single source pad. 330 pad from ipuX_ic_prp, and a single source pad. The source pad is routed
331 to a capture device node, with a node name of 331 to a capture device node, with a node name of the format
332 "ipuX_ic_prpvf capture". 332 "ipuX_ic_prpvf capture".
333 333
334 This entity is identical in operation to ipuX_ 334 This entity is identical in operation to ipuX_ic_prpenc, with the same
335 resizing and CSC operations and flip/rotation 335 resizing and CSC operations and flip/rotation controls. It will receive
336 and process de-interlaced frames from the ipuX 336 and process de-interlaced frames from the ipuX_vdic if ipuX_ic_prp is
337 receiving from ipuX_vdic. 337 receiving from ipuX_vdic.
338 338
339 Like the ipuX_csiY IDMAC source, this entity s 339 Like the ipuX_csiY IDMAC source, this entity supports simple
340 interweaving without motion compensation. Howe 340 interweaving without motion compensation. However, note that if the
341 ipuX_vdic is included in the pipeline (ipuX_ic 341 ipuX_vdic is included in the pipeline (ipuX_ic_prp is receiving from
342 ipuX_vdic), it's not possible to use interweav 342 ipuX_vdic), it's not possible to use interweave in ipuX_ic_prpvf,
343 since the ipuX_vdic has already carried out de 343 since the ipuX_vdic has already carried out de-interlacing (with
344 motion compensation) and therefore the field t 344 motion compensation) and therefore the field type output from
345 ipuX_vdic can only be none (progressive). 345 ipuX_vdic can only be none (progressive).
346 346
347 Capture Pipelines 347 Capture Pipelines
348 ----------------- 348 -----------------
349 349
350 The following describe the various use-cases s 350 The following describe the various use-cases supported by the pipelines.
351 351
352 The links shown do not include the backend sen 352 The links shown do not include the backend sensor, video mux, or mipi
353 csi-2 receiver links. This depends on the type 353 csi-2 receiver links. This depends on the type of sensor interface
354 (parallel or mipi csi-2). So these pipelines b 354 (parallel or mipi csi-2). So these pipelines begin with:
355 355
356 sensor -> ipuX_csiY_mux -> ... 356 sensor -> ipuX_csiY_mux -> ...
357 357
358 for parallel sensors, or: 358 for parallel sensors, or:
359 359
360 sensor -> imx6-mipi-csi2 -> (ipuX_csiY_mux) -> 360 sensor -> imx6-mipi-csi2 -> (ipuX_csiY_mux) -> ...
361 361
362 for mipi csi-2 sensors. The imx6-mipi-csi2 rec 362 for mipi csi-2 sensors. The imx6-mipi-csi2 receiver may need to route
363 to the video mux (ipuX_csiY_mux) before sendin 363 to the video mux (ipuX_csiY_mux) before sending to the CSI, depending
364 on the mipi csi-2 virtual channel, hence ipuX_ 364 on the mipi csi-2 virtual channel, hence ipuX_csiY_mux is shown in
365 parenthesis. 365 parenthesis.
366 366
367 Unprocessed Video Capture: 367 Unprocessed Video Capture:
368 -------------------------- 368 --------------------------
369 369
370 Send frames directly from sensor to camera dev 370 Send frames directly from sensor to camera device interface node, with
371 no conversions, via ipuX_csiY IDMAC source pad 371 no conversions, via ipuX_csiY IDMAC source pad:
372 372
373 -> ipuX_csiY:2 -> ipuX_csiY capture 373 -> ipuX_csiY:2 -> ipuX_csiY capture
374 374
375 IC Direct Conversions: 375 IC Direct Conversions:
376 ---------------------- 376 ----------------------
377 377
378 This pipeline uses the preprocess encode entit 378 This pipeline uses the preprocess encode entity to route frames directly
379 from the CSI to the IC, to carry out scaling u 379 from the CSI to the IC, to carry out scaling up to 1024x1024 resolution,
380 CSC, flipping, and image rotation: 380 CSC, flipping, and image rotation:
381 381
382 -> ipuX_csiY:1 -> 0:ipuX_ic_prp:1 -> 0:ipuX_ic 382 -> ipuX_csiY:1 -> 0:ipuX_ic_prp:1 -> 0:ipuX_ic_prpenc:1 -> ipuX_ic_prpenc capture
383 383
384 Motion Compensated De-interlace: 384 Motion Compensated De-interlace:
385 -------------------------------- 385 --------------------------------
386 386
387 This pipeline routes frames from the CSI direc 387 This pipeline routes frames from the CSI direct pad to the VDIC entity to
388 support motion-compensated de-interlacing (hig 388 support motion-compensated de-interlacing (high motion mode only),
389 scaling up to 1024x1024, CSC, flip, and rotati 389 scaling up to 1024x1024, CSC, flip, and rotation:
390 390
391 -> ipuX_csiY:1 -> 0:ipuX_vdic:2 -> 0:ipuX_ic_p 391 -> ipuX_csiY:1 -> 0:ipuX_vdic:2 -> 0:ipuX_ic_prp:2 -> 0:ipuX_ic_prpvf:1 -> ipuX_ic_prpvf capture
392 392
393 393
394 Usage Notes 394 Usage Notes
395 ----------- 395 -----------
396 396
397 To aid in configuration and for backward compa 397 To aid in configuration and for backward compatibility with V4L2
398 applications that access controls only from vi 398 applications that access controls only from video device nodes, the
399 capture device interfaces inherit controls fro 399 capture device interfaces inherit controls from the active entities
400 in the current pipeline, so controls can be ac 400 in the current pipeline, so controls can be accessed either directly
401 from the subdev or from the active capture dev 401 from the subdev or from the active capture device interface. For
402 example, the FIM controls are available either 402 example, the FIM controls are available either from the ipuX_csiY
403 subdevs or from the active capture device. 403 subdevs or from the active capture device.
404 404
405 The following are specific usage notes for the 405 The following are specific usage notes for the Sabre* reference
406 boards: 406 boards:
407 407
408 408
409 i.MX6Q SabreLite with OV5642 and OV5640 409 i.MX6Q SabreLite with OV5642 and OV5640
410 --------------------------------------- 410 ---------------------------------------
411 411
412 This platform requires the OmniVision OV5642 m 412 This platform requires the OmniVision OV5642 module with a parallel
413 camera interface, and the OV5640 module with a 413 camera interface, and the OV5640 module with a MIPI CSI-2
414 interface. Both modules are available from Bou 414 interface. Both modules are available from Boundary Devices:
415 415
416 - https://boundarydevices.com/product/nit6x_5m 416 - https://boundarydevices.com/product/nit6x_5mp
417 - https://boundarydevices.com/product/nit6x_5m 417 - https://boundarydevices.com/product/nit6x_5mp_mipi
418 418
419 Note that if only one camera module is availab 419 Note that if only one camera module is available, the other sensor
420 node can be disabled in the device tree. 420 node can be disabled in the device tree.
421 421
422 The OV5642 module is connected to the parallel 422 The OV5642 module is connected to the parallel bus input on the i.MX
423 internal video mux to IPU1 CSI0. It's i2c bus 423 internal video mux to IPU1 CSI0. It's i2c bus connects to i2c bus 2.
424 424
425 The MIPI CSI-2 OV5640 module is connected to t 425 The MIPI CSI-2 OV5640 module is connected to the i.MX internal MIPI CSI-2
426 receiver, and the four virtual channel outputs 426 receiver, and the four virtual channel outputs from the receiver are
427 routed as follows: vc0 to the IPU1 CSI0 mux, v 427 routed as follows: vc0 to the IPU1 CSI0 mux, vc1 directly to IPU1 CSI1,
428 vc2 directly to IPU2 CSI0, and vc3 to the IPU2 428 vc2 directly to IPU2 CSI0, and vc3 to the IPU2 CSI1 mux. The OV5640 is
429 also connected to i2c bus 2 on the SabreLite, 429 also connected to i2c bus 2 on the SabreLite, therefore the OV5642 and
430 OV5640 must not share the same i2c slave addre 430 OV5640 must not share the same i2c slave address.
431 431
432 The following basic example configures unproce 432 The following basic example configures unprocessed video capture
433 pipelines for both sensors. The OV5642 is rout 433 pipelines for both sensors. The OV5642 is routed to ipu1_csi0, and
434 the OV5640, transmitting on MIPI CSI-2 virtual 434 the OV5640, transmitting on MIPI CSI-2 virtual channel 1 (which is
435 imx6-mipi-csi2 pad 2), is routed to ipu1_csi1. 435 imx6-mipi-csi2 pad 2), is routed to ipu1_csi1. Both sensors are
436 configured to output 640x480, and the OV5642 o 436 configured to output 640x480, and the OV5642 outputs YUYV2X8, the
437 OV5640 UYVY2X8: 437 OV5640 UYVY2X8:
438 438
439 .. code-block:: none 439 .. code-block:: none
440 440
441 # Setup links for OV5642 441 # Setup links for OV5642
442 media-ctl -l "'ov5642 1-0042':0 -> 'ipu1_cs 442 media-ctl -l "'ov5642 1-0042':0 -> 'ipu1_csi0_mux':1[1]"
443 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_cs 443 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
444 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 c 444 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
445 # Setup links for OV5640 445 # Setup links for OV5640
446 media-ctl -l "'ov5640 1-0040':0 -> 'imx6-mi 446 media-ctl -l "'ov5640 1-0040':0 -> 'imx6-mipi-csi2':0[1]"
447 media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_c 447 media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
448 media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 c 448 media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
449 # Configure pads for OV5642 pipeline 449 # Configure pads for OV5642 pipeline
450 media-ctl -V "'ov5642 1-0042':0 [fmt:YUYV2X 450 media-ctl -V "'ov5642 1-0042':0 [fmt:YUYV2X8/640x480 field:none]"
451 media-ctl -V "'ipu1_csi0_mux':2 [fmt:YUYV2X 451 media-ctl -V "'ipu1_csi0_mux':2 [fmt:YUYV2X8/640x480 field:none]"
452 media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/640 452 media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/640x480 field:none]"
453 # Configure pads for OV5640 pipeline 453 # Configure pads for OV5640 pipeline
454 media-ctl -V "'ov5640 1-0040':0 [fmt:UYVY2X 454 media-ctl -V "'ov5640 1-0040':0 [fmt:UYVY2X8/640x480 field:none]"
455 media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2 455 media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/640x480 field:none]"
456 media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/640 456 media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/640x480 field:none]"
457 457
458 Streaming can then begin independently on the 458 Streaming can then begin independently on the capture device nodes
459 "ipu1_csi0 capture" and "ipu1_csi1 capture". T 459 "ipu1_csi0 capture" and "ipu1_csi1 capture". The v4l2-ctl tool can
460 be used to select any supported YUV pixelforma 460 be used to select any supported YUV pixelformat on the capture device
461 nodes, including planar. 461 nodes, including planar.
462 462
463 i.MX6Q SabreAuto with ADV7180 decoder 463 i.MX6Q SabreAuto with ADV7180 decoder
464 ------------------------------------- 464 -------------------------------------
465 465
466 On the i.MX6Q SabreAuto, an on-board ADV7180 S 466 On the i.MX6Q SabreAuto, an on-board ADV7180 SD decoder is connected to the
467 parallel bus input on the internal video mux t 467 parallel bus input on the internal video mux to IPU1 CSI0.
468 468
469 The following example configures a pipeline to 469 The following example configures a pipeline to capture from the ADV7180
470 video decoder, assuming NTSC 720x480 input sig 470 video decoder, assuming NTSC 720x480 input signals, using simple
471 interweave (unconverted and without motion com 471 interweave (unconverted and without motion compensation). The adv7180
472 must output sequential or alternating fields ( 472 must output sequential or alternating fields (field type 'seq-bt' for
473 NTSC, or 'alternate'): 473 NTSC, or 'alternate'):
474 474
475 .. code-block:: none 475 .. code-block:: none
476 476
477 # Setup links 477 # Setup links
478 media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_c 478 media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]"
479 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_cs 479 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
480 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 c 480 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
481 # Configure pads 481 # Configure pads
482 media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2 482 media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480 field:seq-bt]"
483 media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X 483 media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480]"
484 media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720 484 media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]"
485 # Configure "ipu1_csi0 capture" interface ( 485 # Configure "ipu1_csi0 capture" interface (assumed at /dev/video4)
486 v4l2-ctl -d4 --set-fmt-video=field=interlac 486 v4l2-ctl -d4 --set-fmt-video=field=interlaced_bt
487 487
488 Streaming can then begin on /dev/video4. The v 488 Streaming can then begin on /dev/video4. The v4l2-ctl tool can also be
489 used to select any supported YUV pixelformat o 489 used to select any supported YUV pixelformat on /dev/video4.
490 490
491 This example configures a pipeline to capture 491 This example configures a pipeline to capture from the ADV7180
492 video decoder, assuming PAL 720x576 input sign 492 video decoder, assuming PAL 720x576 input signals, with Motion
493 Compensated de-interlacing. The adv7180 must o 493 Compensated de-interlacing. The adv7180 must output sequential or
494 alternating fields (field type 'seq-tb' for PA 494 alternating fields (field type 'seq-tb' for PAL, or 'alternate').
495 495
496 .. code-block:: none 496 .. code-block:: none
497 497
498 # Setup links 498 # Setup links
499 media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_c 499 media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]"
500 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_cs 500 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
501 media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic': 501 media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]"
502 media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp 502 media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]"
503 media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_p 503 media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]"
504 media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic 504 media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]"
505 # Configure pads 505 # Configure pads
506 media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2 506 media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x576 field:seq-tb]"
507 media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X 507 media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x576]"
508 media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720 508 media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x576]"
509 media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720 509 media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x576 field:none]"
510 media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/7 510 media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]"
511 media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32 511 media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]"
512 # Configure "ipu1_ic_prpvf capture" interfa 512 # Configure "ipu1_ic_prpvf capture" interface (assumed at /dev/video2)
513 v4l2-ctl -d2 --set-fmt-video=field=none 513 v4l2-ctl -d2 --set-fmt-video=field=none
514 514
515 Streaming can then begin on /dev/video2. The v 515 Streaming can then begin on /dev/video2. The v4l2-ctl tool can also be
516 used to select any supported YUV pixelformat o 516 used to select any supported YUV pixelformat on /dev/video2.
517 517
518 This platform accepts Composite Video analog i 518 This platform accepts Composite Video analog inputs to the ADV7180 on
519 Ain1 (connector J42). 519 Ain1 (connector J42).
520 520
521 i.MX6DL SabreAuto with ADV7180 decoder 521 i.MX6DL SabreAuto with ADV7180 decoder
522 -------------------------------------- 522 --------------------------------------
523 523
524 On the i.MX6DL SabreAuto, an on-board ADV7180 524 On the i.MX6DL SabreAuto, an on-board ADV7180 SD decoder is connected to the
525 parallel bus input on the internal video mux t 525 parallel bus input on the internal video mux to IPU1 CSI0.
526 526
527 The following example configures a pipeline to 527 The following example configures a pipeline to capture from the ADV7180
528 video decoder, assuming NTSC 720x480 input sig 528 video decoder, assuming NTSC 720x480 input signals, using simple
529 interweave (unconverted and without motion com 529 interweave (unconverted and without motion compensation). The adv7180
530 must output sequential or alternating fields ( 530 must output sequential or alternating fields (field type 'seq-bt' for
531 NTSC, or 'alternate'): 531 NTSC, or 'alternate'):
532 532
533 .. code-block:: none 533 .. code-block:: none
534 534
535 # Setup links 535 # Setup links
536 media-ctl -l "'adv7180 4-0021':0 -> 'ipu1_c 536 media-ctl -l "'adv7180 4-0021':0 -> 'ipu1_csi0_mux':4[1]"
537 media-ctl -l "'ipu1_csi0_mux':5 -> 'ipu1_cs 537 media-ctl -l "'ipu1_csi0_mux':5 -> 'ipu1_csi0':0[1]"
538 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 c 538 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
539 # Configure pads 539 # Configure pads
540 media-ctl -V "'adv7180 4-0021':0 [fmt:UYVY2 540 media-ctl -V "'adv7180 4-0021':0 [fmt:UYVY2X8/720x480 field:seq-bt]"
541 media-ctl -V "'ipu1_csi0_mux':5 [fmt:UYVY2X 541 media-ctl -V "'ipu1_csi0_mux':5 [fmt:UYVY2X8/720x480]"
542 media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720 542 media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]"
543 # Configure "ipu1_csi0 capture" interface ( 543 # Configure "ipu1_csi0 capture" interface (assumed at /dev/video0)
544 v4l2-ctl -d0 --set-fmt-video=field=interlac 544 v4l2-ctl -d0 --set-fmt-video=field=interlaced_bt
545 545
546 Streaming can then begin on /dev/video0. The v 546 Streaming can then begin on /dev/video0. The v4l2-ctl tool can also be
547 used to select any supported YUV pixelformat o 547 used to select any supported YUV pixelformat on /dev/video0.
548 548
549 This example configures a pipeline to capture 549 This example configures a pipeline to capture from the ADV7180
550 video decoder, assuming PAL 720x576 input sign 550 video decoder, assuming PAL 720x576 input signals, with Motion
551 Compensated de-interlacing. The adv7180 must o 551 Compensated de-interlacing. The adv7180 must output sequential or
552 alternating fields (field type 'seq-tb' for PA 552 alternating fields (field type 'seq-tb' for PAL, or 'alternate').
553 553
554 .. code-block:: none 554 .. code-block:: none
555 555
556 # Setup links 556 # Setup links
557 media-ctl -l "'adv7180 4-0021':0 -> 'ipu1_c 557 media-ctl -l "'adv7180 4-0021':0 -> 'ipu1_csi0_mux':4[1]"
558 media-ctl -l "'ipu1_csi0_mux':5 -> 'ipu1_cs 558 media-ctl -l "'ipu1_csi0_mux':5 -> 'ipu1_csi0':0[1]"
559 media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic': 559 media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]"
560 media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp 560 media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]"
561 media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_p 561 media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]"
562 media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic 562 media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]"
563 # Configure pads 563 # Configure pads
564 media-ctl -V "'adv7180 4-0021':0 [fmt:UYVY2 564 media-ctl -V "'adv7180 4-0021':0 [fmt:UYVY2X8/720x576 field:seq-tb]"
565 media-ctl -V "'ipu1_csi0_mux':5 [fmt:UYVY2X 565 media-ctl -V "'ipu1_csi0_mux':5 [fmt:UYVY2X8/720x576]"
566 media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720 566 media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x576]"
567 media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720 567 media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x576 field:none]"
568 media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/7 568 media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]"
569 media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32 569 media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]"
570 # Configure "ipu1_ic_prpvf capture" interfa 570 # Configure "ipu1_ic_prpvf capture" interface (assumed at /dev/video2)
571 v4l2-ctl -d2 --set-fmt-video=field=none 571 v4l2-ctl -d2 --set-fmt-video=field=none
572 572
573 Streaming can then begin on /dev/video2. The v 573 Streaming can then begin on /dev/video2. The v4l2-ctl tool can also be
574 used to select any supported YUV pixelformat o 574 used to select any supported YUV pixelformat on /dev/video2.
575 575
576 This platform accepts Composite Video analog i 576 This platform accepts Composite Video analog inputs to the ADV7180 on
577 Ain1 (connector J42). 577 Ain1 (connector J42).
578 578
579 i.MX6Q SabreSD with MIPI CSI-2 OV5640 579 i.MX6Q SabreSD with MIPI CSI-2 OV5640
580 ------------------------------------- 580 -------------------------------------
581 581
582 Similarly to i.MX6Q SabreLite, the i.MX6Q Sabr 582 Similarly to i.MX6Q SabreLite, the i.MX6Q SabreSD supports a parallel
583 interface OV5642 module on IPU1 CSI0, and a MI 583 interface OV5642 module on IPU1 CSI0, and a MIPI CSI-2 OV5640
584 module. The OV5642 connects to i2c bus 1 and t 584 module. The OV5642 connects to i2c bus 1 and the OV5640 to i2c bus 2.
585 585
586 The device tree for SabreSD includes OF graphs 586 The device tree for SabreSD includes OF graphs for both the parallel
587 OV5642 and the MIPI CSI-2 OV5640, but as of th 587 OV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI
588 CSI-2 OV5640 has been tested, so the OV5642 no 588 CSI-2 OV5640 has been tested, so the OV5642 node is currently disabled.
589 The OV5640 module connects to MIPI connector J 589 The OV5640 module connects to MIPI connector J5. The NXP part number
590 for the OV5640 module that connects to the Sab 590 for the OV5640 module that connects to the SabreSD board is H120729.
591 591
592 The following example configures unprocessed v 592 The following example configures unprocessed video capture pipeline to
593 capture from the OV5640, transmitting on MIPI 593 capture from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
594 594
595 .. code-block:: none 595 .. code-block:: none
596 596
597 # Setup links 597 # Setup links
598 media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mi 598 media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
599 media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_c 599 media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
600 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_cs 600 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
601 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 c 601 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
602 # Configure pads 602 # Configure pads
603 media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X 603 media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
604 media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2 604 media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
605 media-ctl -V "'ipu1_csi0_mux':0 [fmt:UYVY2X 605 media-ctl -V "'ipu1_csi0_mux':0 [fmt:UYVY2X8/640x480]"
606 media-ctl -V "'ipu1_csi0':0 [fmt:AYUV32/640 606 media-ctl -V "'ipu1_csi0':0 [fmt:AYUV32/640x480]"
607 607
608 Streaming can then begin on "ipu1_csi0 capture 608 Streaming can then begin on "ipu1_csi0 capture" node. The v4l2-ctl
609 tool can be used to select any supported pixel 609 tool can be used to select any supported pixelformat on the capture
610 device node. 610 device node.
611 611
612 To determine what is the /dev/video node corre 612 To determine what is the /dev/video node correspondent to
613 "ipu1_csi0 capture": 613 "ipu1_csi0 capture":
614 614
615 .. code-block:: none 615 .. code-block:: none
616 616
617 media-ctl -e "ipu1_csi0 capture" 617 media-ctl -e "ipu1_csi0 capture"
618 /dev/video0 618 /dev/video0
619 619
620 /dev/video0 is the streaming element in this c 620 /dev/video0 is the streaming element in this case.
621 621
622 Starting the streaming via v4l2-ctl: 622 Starting the streaming via v4l2-ctl:
623 623
624 .. code-block:: none 624 .. code-block:: none
625 625
626 v4l2-ctl --stream-mmap -d /dev/video0 626 v4l2-ctl --stream-mmap -d /dev/video0
627 627
628 Starting the streaming via Gstreamer and sendi 628 Starting the streaming via Gstreamer and sending the content to the display:
629 629
630 .. code-block:: none 630 .. code-block:: none
631 631
632 gst-launch-1.0 v4l2src device=/dev/video0 ! 632 gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink
633 633
634 The following example configures a direct conv 634 The following example configures a direct conversion pipeline to capture
635 from the OV5640, transmitting on MIPI CSI-2 vi 635 from the OV5640, transmitting on MIPI CSI-2 virtual channel 0. It also
636 shows colorspace conversion and scaling at IC 636 shows colorspace conversion and scaling at IC output.
637 637
638 .. code-block:: none 638 .. code-block:: none
639 639
640 # Setup links 640 # Setup links
641 media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mi 641 media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
642 media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_c 642 media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
643 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_cs 643 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
644 media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp 644 media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
645 media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_p 645 media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
646 media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_i 646 media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
647 # Configure pads 647 # Configure pads
648 media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X 648 media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
649 media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2 649 media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
650 media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X 650 media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]"
651 media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640 651 media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]"
652 media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/6 652 media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
653 media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8 653 media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"
654 # Set a format at the capture interface 654 # Set a format at the capture interface
655 v4l2-ctl -d /dev/video1 --set-fmt-video=pix 655 v4l2-ctl -d /dev/video1 --set-fmt-video=pixelformat=RGB3
656 656
657 Streaming can then begin on "ipu1_ic_prpenc ca 657 Streaming can then begin on "ipu1_ic_prpenc capture" node.
658 658
659 To determine what is the /dev/video node corre 659 To determine what is the /dev/video node correspondent to
660 "ipu1_ic_prpenc capture": 660 "ipu1_ic_prpenc capture":
661 661
662 .. code-block:: none 662 .. code-block:: none
663 663
664 media-ctl -e "ipu1_ic_prpenc capture" 664 media-ctl -e "ipu1_ic_prpenc capture"
665 /dev/video1 665 /dev/video1
666 666
667 667
668 /dev/video1 is the streaming element in this c 668 /dev/video1 is the streaming element in this case.
669 669
670 Starting the streaming via v4l2-ctl: 670 Starting the streaming via v4l2-ctl:
671 671
672 .. code-block:: none 672 .. code-block:: none
673 673
674 v4l2-ctl --stream-mmap -d /dev/video1 674 v4l2-ctl --stream-mmap -d /dev/video1
675 675
676 Starting the streaming via Gstreamer and sendi 676 Starting the streaming via Gstreamer and sending the content to the display:
677 677
678 .. code-block:: none 678 .. code-block:: none
679 679
680 gst-launch-1.0 v4l2src device=/dev/video1 ! 680 gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink
681 681
682 Known Issues 682 Known Issues
683 ------------ 683 ------------
684 684
685 1. When using 90 or 270 degree rotation contro 685 1. When using 90 or 270 degree rotation control at capture resolutions
686 near the IC resizer limit of 1024x1024, and 686 near the IC resizer limit of 1024x1024, and combined with planar
687 pixel formats (YUV420, YUV422p), frame capt 687 pixel formats (YUV420, YUV422p), frame capture will often fail with
688 no end-of-frame interrupts from the IDMAC c 688 no end-of-frame interrupts from the IDMAC channel. To work around
689 this, use lower resolution and/or packed fo 689 this, use lower resolution and/or packed formats (YUYV, RGB3, etc.)
690 when 90 or 270 rotations are needed. 690 when 90 or 270 rotations are needed.
691 691
692 692
693 File list 693 File list
694 --------- 694 ---------
695 695
696 drivers/staging/media/imx/ 696 drivers/staging/media/imx/
697 include/media/imx.h 697 include/media/imx.h
698 include/linux/imx-media.h 698 include/linux/imx-media.h
699 699
700 References 700 References
701 ---------- 701 ----------
702 702
703 .. [#f1] http://www.nxp.com/assets/documents/d 703 .. [#f1] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6DQRM.pdf
704 .. [#f2] http://www.nxp.com/assets/documents/d 704 .. [#f2] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6SDLRM.pdf
705 705
706 706
707 Authors 707 Authors
708 ------- 708 -------
709 709
710 - Steve Longerbeam <steve_longerbeam@mentor.com 710 - Steve Longerbeam <steve_longerbeam@mentor.com>
711 - Philipp Zabel <kernel@pengutronix.de> 711 - Philipp Zabel <kernel@pengutronix.de>
712 - Russell King <linux@armlinux.org.uk> 712 - Russell King <linux@armlinux.org.uk>
713 713
714 Copyright (C) 2012-2017 Mentor Graphics Inc. 714 Copyright (C) 2012-2017 Mentor Graphics Inc.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.