1 .. SPDX-License-Identifier: GPL-2.0 1 .. SPDX-License-Identifier: GPL-2.0
2 2
3 .. include:: <isonum.txt> 3 .. include:: <isonum.txt>
4 4
5 ============================================== 5 ========================================================
6 Intel Image Processing Unit 6 (IPU6) Input Sys 6 Intel Image Processing Unit 6 (IPU6) Input System driver
7 ============================================== 7 ========================================================
8 8
9 Copyright |copy| 2023--2024 Intel Corporation 9 Copyright |copy| 2023--2024 Intel Corporation
10 10
11 Introduction 11 Introduction
12 ============ 12 ============
13 13
14 This file documents the Intel IPU6 (6th genera 14 This file documents the Intel IPU6 (6th generation Image Processing Unit)
15 Input System (MIPI CSI2 receiver) drivers loca 15 Input System (MIPI CSI2 receiver) drivers located under
16 drivers/media/pci/intel/ipu6. 16 drivers/media/pci/intel/ipu6.
17 17
18 The Intel IPU6 can be found in certain Intel S 18 The Intel IPU6 can be found in certain Intel SoCs but not in all SKUs:
19 19
20 * Tiger Lake 20 * Tiger Lake
21 * Jasper Lake 21 * Jasper Lake
22 * Alder Lake 22 * Alder Lake
23 * Raptor Lake 23 * Raptor Lake
24 * Meteor Lake 24 * Meteor Lake
25 25
26 Intel IPU6 is made up of two components - Inpu 26 Intel IPU6 is made up of two components - Input System (ISYS) and Processing
27 System (PSYS). 27 System (PSYS).
28 28
29 The Input System mainly works as MIPI CSI-2 re 29 The Input System mainly works as MIPI CSI-2 receiver which receives and
30 processes the image data from the sensors and 30 processes the image data from the sensors and outputs the frames to memory.
31 31
32 There are 2 driver modules - intel-ipu6 and in 32 There are 2 driver modules - intel-ipu6 and intel-ipu6-isys. intel-ipu6 is an
33 IPU6 common driver which does PCI configuratio 33 IPU6 common driver which does PCI configuration, firmware loading and parsing,
34 firmware authentication, DMA mapping and IPU-M 34 firmware authentication, DMA mapping and IPU-MMU (internal Memory mapping Unit)
35 configuration. intel_ipu6_isys implements V4L2 35 configuration. intel_ipu6_isys implements V4L2, Media Controller and V4L2
36 sub-device interfaces. The IPU6 ISYS driver su 36 sub-device interfaces. The IPU6 ISYS driver supports camera sensors connected
37 to the IPU6 ISYS through V4L2 sub-device senso 37 to the IPU6 ISYS through V4L2 sub-device sensor drivers.
38 38
39 .. Note:: See Documentation/driver-api/media/d 39 .. Note:: See Documentation/driver-api/media/drivers/ipu6.rst for more
40 information about the IPU6 hardware. 40 information about the IPU6 hardware.
41 41
42 Input system driver 42 Input system driver
43 =================== 43 ===================
44 44
45 The Input System driver mainly configures CSI- 45 The Input System driver mainly configures CSI-2 D-PHY, constructs the firmware
46 stream configuration, sends commands to firmwa 46 stream configuration, sends commands to firmware, gets response from hardware
47 and firmware and then returns buffers to user. 47 and firmware and then returns buffers to user. The ISYS is represented as
48 several V4L2 sub-devices as well as video node 48 several V4L2 sub-devices as well as video nodes.
49 49
50 .. kernel-figure:: ipu6_isys_graph.svg 50 .. kernel-figure:: ipu6_isys_graph.svg
51 :alt: ipu6 isys media graph with multiple s 51 :alt: ipu6 isys media graph with multiple streams support
52 52
53 IPU6 ISYS media graph with multiple streams 53 IPU6 ISYS media graph with multiple streams support
54 54
55 The graph has been produced using the followin 55 The graph has been produced using the following command:
56 56
57 .. code-block:: none 57 .. code-block:: none
58 58
59 fdp -Gsplines=true -Tsvg < dot > dot.svg 59 fdp -Gsplines=true -Tsvg < dot > dot.svg
60 60
61 Capturing frames with IPU6 ISYS 61 Capturing frames with IPU6 ISYS
62 ------------------------------- 62 -------------------------------
63 63
64 IPU6 ISYS is used to capture frames from the c 64 IPU6 ISYS is used to capture frames from the camera sensors connected to the
65 CSI2 ports. The supported input formats of ISY 65 CSI2 ports. The supported input formats of ISYS are listed in table below:
66 66
67 .. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm 67 .. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}|
68 68
69 .. flat-table:: 69 .. flat-table::
70 :header-rows: 1 70 :header-rows: 1
71 71
72 * - IPU6 ISYS supported input formats 72 * - IPU6 ISYS supported input formats
73 73
74 * - RGB565, RGB888 74 * - RGB565, RGB888
75 75
76 * - UYVY8, YUYV8 76 * - UYVY8, YUYV8
77 77
78 * - RAW8, RAW10, RAW12 78 * - RAW8, RAW10, RAW12
79 79
80 .. _ipu6_isys_capture_examples: 80 .. _ipu6_isys_capture_examples:
81 81
82 Examples 82 Examples
83 ~~~~~~~~ 83 ~~~~~~~~
84 84
85 Here is an example of IPU6 ISYS raw capture on 85 Here is an example of IPU6 ISYS raw capture on Dell XPS 9315 laptop. On this
86 machine, ov01a10 sensor is connected to IPU IS 86 machine, ov01a10 sensor is connected to IPU ISYS CSI-2 port 2, which can
87 generate images at sBGGR10 with resolution 128 87 generate images at sBGGR10 with resolution 1280x800.
88 88
89 Using the media controller APIs, we can config 89 Using the media controller APIs, we can configure ov01a10 sensor by
90 media-ctl [#f1]_ and yavta [#f2]_ to transmit 90 media-ctl [#f1]_ and yavta [#f2]_ to transmit frames to IPU6 ISYS.
91 91
92 .. code-block:: none 92 .. code-block:: none
93 93
94 # Example 1 capture frame from ov01a10 cam 94 # Example 1 capture frame from ov01a10 camera sensor
95 # This example assumes /dev/media0 as the 95 # This example assumes /dev/media0 as the IPU ISYS media device
96 export MDEV=/dev/media0 96 export MDEV=/dev/media0
97 97
98 # Establish the link for the media devices 98 # Establish the link for the media devices using media-ctl
99 media-ctl -d $MDEV -l "\"ov01a10 3-0036\": 99 media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]"
100 100
101 # Set the format for the media devices 101 # Set the format for the media devices
102 media-ctl -d $MDEV -V "ov01a10:0 [fmt:SBGG 102 media-ctl -d $MDEV -V "ov01a10:0 [fmt:SBGGR10/1280x800]"
103 media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:0 103 media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:0 [fmt:SBGGR10/1280x800]"
104 media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:1 104 media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:1 [fmt:SBGGR10/1280x800]"
105 105
106 Once the media pipeline is configured, desired 106 Once the media pipeline is configured, desired sensor specific settings
107 (such as exposure and gain settings) can be se 107 (such as exposure and gain settings) can be set, using the yavta tool.
108 108
109 e.g 109 e.g
110 110
111 .. code-block:: none 111 .. code-block:: none
112 112
113 # and that ov01a10 sensor is connected to 113 # and that ov01a10 sensor is connected to i2c bus 3 with address 0x36
114 export SDEV=$(media-ctl -d $MDEV -e "ov01a 114 export SDEV=$(media-ctl -d $MDEV -e "ov01a10 3-0036")
115 115
116 yavta -w 0x009e0903 400 $SDEV 116 yavta -w 0x009e0903 400 $SDEV
117 yavta -w 0x009e0913 1000 $SDEV 117 yavta -w 0x009e0913 1000 $SDEV
118 yavta -w 0x009e0911 2000 $SDEV 118 yavta -w 0x009e0911 2000 $SDEV
119 119
120 Once the desired sensor settings are set, fram 120 Once the desired sensor settings are set, frame captures can be done as below.
121 121
122 e.g 122 e.g
123 123
124 .. code-block:: none 124 .. code-block:: none
125 125
126 yavta --data-prefix -u -c10 -n5 -I -s 1280 126 yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --file=/tmp/frame-#.bin \
127 -f SBGGR10 $(media-ctl -d $MDEV -e 127 -f SBGGR10 $(media-ctl -d $MDEV -e "Intel IPU6 ISYS Capture 0")
128 128
129 With the above command, 10 frames are captured 129 With the above command, 10 frames are captured at 1280x800 resolution with
130 sBGGR10 format. The captured frames are availa 130 sBGGR10 format. The captured frames are available as /tmp/frame-#.bin files.
131 131
132 Here is another example of IPU6 ISYS RAW and m 132 Here is another example of IPU6 ISYS RAW and metadata capture from camera
133 sensor ov2740 on Lenovo X1 Yoga laptop. 133 sensor ov2740 on Lenovo X1 Yoga laptop.
134 134
135 .. code-block:: none 135 .. code-block:: none
136 136
137 media-ctl -l "\"ov2740 14-0036\":0 -> \"In 137 media-ctl -l "\"ov2740 14-0036\":0 -> \"Intel IPU6 CSI2 1\":0[1]"
138 media-ctl -l "\"Intel IPU6 CSI2 1\":1 -> \ 138 media-ctl -l "\"Intel IPU6 CSI2 1\":1 -> \"Intel IPU6 ISYS Capture 0\":0[1]"
139 media-ctl -l "\"Intel IPU6 CSI2 1\":2 -> \ 139 media-ctl -l "\"Intel IPU6 CSI2 1\":2 -> \"Intel IPU6 ISYS Capture 1\":0[1]"
140 140
141 # set routing 141 # set routing
142 media-ctl -R "\"Intel IPU6 CSI2 1\" [0/0-> 142 media-ctl -R "\"Intel IPU6 CSI2 1\" [0/0->1/0[1],0/1->2/1[1]]"
143 143
144 media-ctl -V "\"Intel IPU6 CSI2 1\":0/0 [f 144 media-ctl -V "\"Intel IPU6 CSI2 1\":0/0 [fmt:SGRBG10/1932x1092]"
145 media-ctl -V "\"Intel IPU6 CSI2 1\":0/1 [f 145 media-ctl -V "\"Intel IPU6 CSI2 1\":0/1 [fmt:GENERIC_8/97x1]"
146 media-ctl -V "\"Intel IPU6 CSI2 1\":1/0 [f 146 media-ctl -V "\"Intel IPU6 CSI2 1\":1/0 [fmt:SGRBG10/1932x1092]"
147 media-ctl -V "\"Intel IPU6 CSI2 1\":2/1 [f 147 media-ctl -V "\"Intel IPU6 CSI2 1\":2/1 [fmt:GENERIC_8/97x1]"
148 148
149 CAPTURE_DEV=$(media-ctl -e "Intel IPU6 ISY 149 CAPTURE_DEV=$(media-ctl -e "Intel IPU6 ISYS Capture 0")
150 ./yavta --data-prefix -c100 -n5 -I -s1932x 150 ./yavta --data-prefix -c100 -n5 -I -s1932x1092 --file=/tmp/frame-#.bin \
151 -f SGRBG10 ${CAPTURE_DEV} 151 -f SGRBG10 ${CAPTURE_DEV}
152 152
153 CAPTURE_META=$(media-ctl -e "Intel IPU6 IS 153 CAPTURE_META=$(media-ctl -e "Intel IPU6 ISYS Capture 1")
154 ./yavta --data-prefix -c100 -n5 -I -s97x1 154 ./yavta --data-prefix -c100 -n5 -I -s97x1 -B meta-capture \
155 --file=/tmp/meta-#.bin -f GENERIC_8 ${ 155 --file=/tmp/meta-#.bin -f GENERIC_8 ${CAPTURE_META}
156 156
157 References 157 References
158 ========== 158 ==========
159 159
160 .. [#f1] https://git.ideasonboard.org/media-ct 160 .. [#f1] https://git.ideasonboard.org/media-ctl.git
161 .. [#f2] https://git.ideasonboard.org/yavta.gi 161 .. [#f2] https://git.ideasonboard.org/yavta.git
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.