1 .. SPDX-License-Identifier: GPL-2.0 2 3 .. include:: <isonum.txt> 4 5 ========================================= 6 Rockchip Image Signal Processor (rkisp1) 7 ========================================= 8 9 Introduction 10 ============ 11 12 This file documents the driver for the Rockchip ISP1 that is part of RK3288 13 and RK3399 SoCs. The driver is located under drivers/media/platform/rockchip/ 14 rkisp1 and uses the Media-Controller API. 15 16 Revisions 17 ========= 18 19 There exist multiple smaller revisions to this ISP that got introduced in 20 later SoCs. Revisions can be found in the enum :c:type:`rkisp1_cif_isp_version` 21 in the UAPI and the revision of the ISP inside the running SoC can be read 22 in the field hw_revision of struct media_device_info as returned by 23 ioctl MEDIA_IOC_DEVICE_INFO. 24 25 Versions in use are: 26 27 - RKISP1_V10: used at least in rk3288 and rk3399 28 - RKISP1_V11: declared in the original vendor code, but not used 29 - RKISP1_V12: used at least in rk3326 and px30 30 - RKISP1_V13: used at least in rk1808 31 32 Topology 33 ======== 34 .. _rkisp1_topology_graph: 35 36 .. kernel-figure:: rkisp1.dot 37 :alt: Diagram of the default media pipeline topology 38 :align: center 39 40 41 The driver has 4 video devices: 42 43 - rkisp1_mainpath: capture device for retrieving images, usually in higher 44 resolution. 45 - rkisp1_selfpath: capture device for retrieving images. 46 - rkisp1_stats: a metadata capture device that sends statistics. 47 - rkisp1_params: a metadata output device that receives parameters 48 configurations from userspace. 49 50 The driver has 3 subdevices: 51 52 - rkisp1_resizer_mainpath: used to resize and downsample frames for the 53 mainpath capture device. 54 - rkisp1_resizer_selfpath: used to resize and downsample frames for the 55 selfpath capture device. 56 - rkisp1_isp: is connected to the sensor and is responsible for all the isp 57 operations. 58 59 60 rkisp1_mainpath, rkisp1_selfpath - Frames Capture Video Nodes 61 ------------------------------------------------------------- 62 Those are the `mainpath` and `selfpath` capture devices to capture frames. 63 Those entities are the DMA engines that write the frames to memory. 64 The selfpath video device can capture YUV/RGB formats. Its input is YUV encoded 65 stream and it is able to convert it to RGB. The selfpath is not able to 66 capture bayer formats. 67 The mainpath can capture both bayer and YUV formats but it is not able to 68 capture RGB formats. 69 Both capture videos support 70 the ``V4L2_CAP_IO_MC`` :ref:`capability <device-capabilities>`. 71 72 73 rkisp1_resizer_mainpath, rkisp1_resizer_selfpath - Resizers Subdevices Nodes 74 ---------------------------------------------------------------------------- 75 Those are resizer entities for the mainpath and the selfpath. Those entities 76 can scale the frames up and down and also change the YUV sampling (for example 77 YUV4:2:2 -> YUV4:2:0). They also have cropping capability on the sink pad. 78 The resizers entities can only operate on YUV:4:2:2 format 79 (MEDIA_BUS_FMT_YUYV8_2X8). 80 The mainpath capture device supports capturing video in bayer formats. In that 81 case the resizer of the mainpath is set to 'bypass' mode - it just forward the 82 frame without operating on it. 83 84 rkisp1_isp - Image Signal Processing Subdevice Node 85 --------------------------------------------------- 86 This is the isp entity. It is connected to the sensor on sink pad 0 and 87 receives the frames using the CSI-2 protocol. It is responsible of configuring 88 the CSI-2 protocol. It has a cropping capability on sink pad 0 that is 89 connected to the sensor and on source pad 2 connected to the resizer entities. 90 Cropping on sink pad 0 defines the image region from the sensor. 91 Cropping on source pad 2 defines the region for the Image Stabilizer (IS). 92 93 .. _rkisp1_stats: 94 95 rkisp1_stats - Statistics Video Node 96 ------------------------------------ 97 The statistics video node outputs the 3A (auto focus, auto exposure and auto 98 white balance) statistics, and also histogram statistics for the frames that 99 are being processed by the rkisp1 to userspace applications. 100 Using these data, applications can implement algorithms and re-parameterize 101 the driver through the rkisp_params node to improve image quality during a 102 video stream. 103 The buffer format is defined by struct :c:type:`rkisp1_stat_buffer`, and 104 userspace should set 105 :ref:`V4L2_META_FMT_RK_ISP1_STAT_3A <v4l2-meta-fmt-rk-isp1-stat-3a>` as the 106 dataformat. 107 108 .. _rkisp1_params: 109 110 rkisp1_params - Parameters Video Node 111 ------------------------------------- 112 The rkisp1_params video node receives a set of parameters from userspace 113 to be applied to the hardware during a video stream, allowing userspace 114 to dynamically modify values such as black level, cross talk corrections 115 and others. 116 117 The buffer format is defined by struct :c:type:`rkisp1_params_cfg`, and 118 userspace should set 119 :ref:`V4L2_META_FMT_RK_ISP1_PARAMS <v4l2-meta-fmt-rk-isp1-params>` as the 120 dataformat. 121 122 123 Capturing Video Frames Example 124 ============================== 125 126 In the following example, the sensor connected to pad 0 of 'rkisp1_isp' is 127 imx219. 128 129 The following commands can be used to capture video from the selfpath video 130 node with dimension 900x800 planar format YUV 4:2:2. It uses all cropping 131 capabilities possible, (see explanation right below) 132 133 .. code-block:: bash 134 135 # set the links 136 "media-ctl" "-d" "platform:rkisp1" "-r" 137 "media-ctl" "-d" "platform:rkisp1" "-l" "'imx219 4-0010':0 -> 'rkisp1_isp':0 [1]" 138 "media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]" 139 "media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_mainpath':0 [0]" 140 141 # set format for imx219 4-0010:0 142 "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx219 4-0010":0 [fmt:SRGGB10_1X10/1640x1232]' 143 144 # set format for rkisp1_isp pads: 145 "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SRGGB10_1X10/1640x1232 crop: (0,0)/1600x1200]' 146 "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2 [fmt:YUYV8_2X8/1600x1200 crop: (0,0)/1500x1100]' 147 148 # set format for rkisp1_resizer_selfpath pads: 149 "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:YUYV8_2X8/1500x1100 crop: (300,400)/1400x1000]' 150 "media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":1 [fmt:YUYV8_2X8/900x800]' 151 152 # set format for rkisp1_selfpath: 153 "v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=900,height=800," 154 "v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "pixelformat=422P" 155 156 # start streaming: 157 v4l2-ctl "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "10" 158 159 160 In the above example the sensor is configured to bayer format: 161 `SRGGB10_1X10/1640x1232`. The rkisp1_isp:0 pad should be configured to the 162 same mbus format and dimensions as the sensor, otherwise streaming will fail 163 with 'EPIPE' error. So it is also configured to `SRGGB10_1X10/1640x1232`. 164 In addition, the rkisp1_isp:0 pad is configured to cropping `(0,0)/1600x1200`. 165 166 The cropping dimensions are automatically propagated to be the format of the 167 isp source pad `rkisp1_isp:2`. Another cropping operation is configured on 168 the isp source pad: `(0,0)/1500x1100`. 169 170 The resizer's sink pad `rkisp1_resizer_selfpath` should be configured to format 171 `YUYV8_2X8/1500x1100` in order to match the format on the other side of the 172 link. In addition a cropping `(300,400)/1400x1000` is configured on it. 173 174 The source pad of the resizer, `rkisp1_resizer_selfpath:1` is configured to 175 format `YUYV8_2X8/900x800`. That means that the resizer first crop a window 176 of `(300,400)/1400x100` from the received frame and then scales this window 177 to dimension `900x800`. 178 179 Note that the above example does not uses the stats-params control loop. 180 Therefore the capture frames will not go through the 3A algorithms and 181 probably won't have a good quality, and can even look dark and greenish. 182 183 Configuring Quantization 184 ======================== 185 186 The driver supports limited and full range quantization on YUV formats, 187 where limited is the default. 188 To switch between one or the other, userspace should use the Colorspace 189 Conversion API (CSC) for subdevices on source pad 2 of the 190 isp (`rkisp1_isp:2`). The quantization configured on this pad is the 191 quantization of the captured video frames on the mainpath and selfpath 192 video nodes. 193 Note that the resizer and capture entities will always report 194 ``V4L2_QUANTIZATION_DEFAULT`` even if the quantization is configured to full 195 range on `rkisp1_isp:2`. So in order to get the configured quantization, 196 application should get it from pad `rkisp1_isp:2`. 197
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.