1 .. SPDX-License-Identifier: GPL-2.0
2
3 .. include:: <isonum.txt>
4
5 ========================================================
6 Intel Image Processing Unit 6 (IPU6) Input System driver
7 ========================================================
8
9 Copyright |copy| 2023--2024 Intel Corporation
10
11 Introduction
12 ============
13
14 This file documents the Intel IPU6 (6th generation Image Processing Unit)
15 Input System (MIPI CSI2 receiver) drivers located under
16 drivers/media/pci/intel/ipu6.
17
18 The Intel IPU6 can be found in certain Intel SoCs but not in all SKUs:
19
20 * Tiger Lake
21 * Jasper Lake
22 * Alder Lake
23 * Raptor Lake
24 * Meteor Lake
25
26 Intel IPU6 is made up of two components - Input System (ISYS) and Processing
27 System (PSYS).
28
29 The Input System mainly works as MIPI CSI-2 receiver which receives and
30 processes the image data from the sensors and outputs the frames to memory.
31
32 There are 2 driver modules - intel-ipu6 and intel-ipu6-isys. intel-ipu6 is an
33 IPU6 common driver which does PCI configuration, firmware loading and parsing,
34 firmware authentication, DMA mapping and IPU-MMU (internal Memory mapping Unit)
35 configuration. intel_ipu6_isys implements V4L2, Media Controller and V4L2
36 sub-device interfaces. The IPU6 ISYS driver supports camera sensors connected
37 to the IPU6 ISYS through V4L2 sub-device sensor drivers.
38
39 .. Note:: See Documentation/driver-api/media/drivers/ipu6.rst for more
40 information about the IPU6 hardware.
41
42 Input system driver
43 ===================
44
45 The Input System driver mainly configures CSI-2 D-PHY, constructs the firmware
46 stream configuration, sends commands to firmware, gets response from hardware
47 and firmware and then returns buffers to user. The ISYS is represented as
48 several V4L2 sub-devices as well as video nodes.
49
50 .. kernel-figure:: ipu6_isys_graph.svg
51 :alt: ipu6 isys media graph with multiple streams support
52
53 IPU6 ISYS media graph with multiple streams support
54
55 The graph has been produced using the following command:
56
57 .. code-block:: none
58
59 fdp -Gsplines=true -Tsvg < dot > dot.svg
60
61 Capturing frames with IPU6 ISYS
62 -------------------------------
63
64 IPU6 ISYS is used to capture frames from the camera sensors connected to the
65 CSI2 ports. The supported input formats of ISYS are listed in table below:
66
67 .. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}|
68
69 .. flat-table::
70 :header-rows: 1
71
72 * - IPU6 ISYS supported input formats
73
74 * - RGB565, RGB888
75
76 * - UYVY8, YUYV8
77
78 * - RAW8, RAW10, RAW12
79
80 .. _ipu6_isys_capture_examples:
81
82 Examples
83 ~~~~~~~~
84
85 Here is an example of IPU6 ISYS raw capture on Dell XPS 9315 laptop. On this
86 machine, ov01a10 sensor is connected to IPU ISYS CSI-2 port 2, which can
87 generate images at sBGGR10 with resolution 1280x800.
88
89 Using the media controller APIs, we can configure ov01a10 sensor by
90 media-ctl [#f1]_ and yavta [#f2]_ to transmit frames to IPU6 ISYS.
91
92 .. code-block:: none
93
94 # Example 1 capture frame from ov01a10 camera sensor
95 # This example assumes /dev/media0 as the IPU ISYS media device
96 export MDEV=/dev/media0
97
98 # Establish the link for the media devices using media-ctl
99 media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]"
100
101 # Set the format for the media devices
102 media-ctl -d $MDEV -V "ov01a10:0 [fmt:SBGGR10/1280x800]"
103 media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:0 [fmt:SBGGR10/1280x800]"
104 media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:1 [fmt:SBGGR10/1280x800]"
105
106 Once the media pipeline is configured, desired sensor specific settings
107 (such as exposure and gain settings) can be set, using the yavta tool.
108
109 e.g
110
111 .. code-block:: none
112
113 # and that ov01a10 sensor is connected to i2c bus 3 with address 0x36
114 export SDEV=$(media-ctl -d $MDEV -e "ov01a10 3-0036")
115
116 yavta -w 0x009e0903 400 $SDEV
117 yavta -w 0x009e0913 1000 $SDEV
118 yavta -w 0x009e0911 2000 $SDEV
119
120 Once the desired sensor settings are set, frame captures can be done as below.
121
122 e.g
123
124 .. code-block:: none
125
126 yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --file=/tmp/frame-#.bin \
127 -f SBGGR10 $(media-ctl -d $MDEV -e "Intel IPU6 ISYS Capture 0")
128
129 With the above command, 10 frames are captured at 1280x800 resolution with
130 sBGGR10 format. The captured frames are available as /tmp/frame-#.bin files.
131
132 Here is another example of IPU6 ISYS RAW and metadata capture from camera
133 sensor ov2740 on Lenovo X1 Yoga laptop.
134
135 .. code-block:: none
136
137 media-ctl -l "\"ov2740 14-0036\":0 -> \"Intel IPU6 CSI2 1\":0[1]"
138 media-ctl -l "\"Intel IPU6 CSI2 1\":1 -> \"Intel IPU6 ISYS Capture 0\":0[1]"
139 media-ctl -l "\"Intel IPU6 CSI2 1\":2 -> \"Intel IPU6 ISYS Capture 1\":0[1]"
140
141 # set routing
142 media-ctl -R "\"Intel IPU6 CSI2 1\" [0/0->1/0[1],0/1->2/1[1]]"
143
144 media-ctl -V "\"Intel IPU6 CSI2 1\":0/0 [fmt:SGRBG10/1932x1092]"
145 media-ctl -V "\"Intel IPU6 CSI2 1\":0/1 [fmt:GENERIC_8/97x1]"
146 media-ctl -V "\"Intel IPU6 CSI2 1\":1/0 [fmt:SGRBG10/1932x1092]"
147 media-ctl -V "\"Intel IPU6 CSI2 1\":2/1 [fmt:GENERIC_8/97x1]"
148
149 CAPTURE_DEV=$(media-ctl -e "Intel IPU6 ISYS Capture 0")
150 ./yavta --data-prefix -c100 -n5 -I -s1932x1092 --file=/tmp/frame-#.bin \
151 -f SGRBG10 ${CAPTURE_DEV}
152
153 CAPTURE_META=$(media-ctl -e "Intel IPU6 ISYS Capture 1")
154 ./yavta --data-prefix -c100 -n5 -I -s97x1 -B meta-capture \
155 --file=/tmp/meta-#.bin -f GENERIC_8 ${CAPTURE_META}
156
157 References
158 ==========
159
160 .. [#f1] https://git.ideasonboard.org/media-ctl.git
161 .. [#f2] https://git.ideasonboard.org/yavta.git
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.