~ [ source navigation ] ~ [ diff markup ] ~ [ identifier search ] ~

TOMOYO Linux Cross Reference
Linux/Documentation/admin-guide/media/ipu3.rst

Version: ~ [ linux-6.11.5 ] ~ [ linux-6.10.14 ] ~ [ linux-6.9.12 ] ~ [ linux-6.8.12 ] ~ [ linux-6.7.12 ] ~ [ linux-6.6.58 ] ~ [ linux-6.5.13 ] ~ [ linux-6.4.16 ] ~ [ linux-6.3.13 ] ~ [ linux-6.2.16 ] ~ [ linux-6.1.114 ] ~ [ linux-6.0.19 ] ~ [ linux-5.19.17 ] ~ [ linux-5.18.19 ] ~ [ linux-5.17.15 ] ~ [ linux-5.16.20 ] ~ [ linux-5.15.169 ] ~ [ linux-5.14.21 ] ~ [ linux-5.13.19 ] ~ [ linux-5.12.19 ] ~ [ linux-5.11.22 ] ~ [ linux-5.10.228 ] ~ [ linux-5.9.16 ] ~ [ linux-5.8.18 ] ~ [ linux-5.7.19 ] ~ [ linux-5.6.19 ] ~ [ linux-5.5.19 ] ~ [ linux-5.4.284 ] ~ [ linux-5.3.18 ] ~ [ linux-5.2.21 ] ~ [ linux-5.1.21 ] ~ [ linux-5.0.21 ] ~ [ linux-4.20.17 ] ~ [ linux-4.19.322 ] ~ [ linux-4.18.20 ] ~ [ linux-4.17.19 ] ~ [ linux-4.16.18 ] ~ [ linux-4.15.18 ] ~ [ linux-4.14.336 ] ~ [ linux-4.13.16 ] ~ [ linux-4.12.14 ] ~ [ linux-4.11.12 ] ~ [ linux-4.10.17 ] ~ [ linux-4.9.337 ] ~ [ linux-4.4.302 ] ~ [ linux-3.10.108 ] ~ [ linux-2.6.32.71 ] ~ [ linux-2.6.0 ] ~ [ linux-2.4.37.11 ] ~ [ unix-v6-master ] ~ [ ccs-tools-1.8.9 ] ~ [ policy-sample ] ~
Architecture: ~ [ i386 ] ~ [ alpha ] ~ [ m68k ] ~ [ mips ] ~ [ ppc ] ~ [ sparc ] ~ [ sparc64 ] ~

  1 .. SPDX-License-Identifier: GPL-2.0
  2 
  3 .. include:: <isonum.txt>
  4 
  5 ===============================================================
  6 Intel Image Processing Unit 3 (IPU3) Imaging Unit (ImgU) driver
  7 ===============================================================
  8 
  9 Copyright |copy| 2018 Intel Corporation
 10 
 11 Introduction
 12 ============
 13 
 14 This file documents the Intel IPU3 (3rd generation Image Processing Unit)
 15 Imaging Unit drivers located under drivers/media/pci/intel/ipu3 (CIO2) as well
 16 as under drivers/staging/media/ipu3 (ImgU).
 17 
 18 The Intel IPU3 found in certain Kaby Lake (as well as certain Sky Lake)
 19 platforms (U/Y processor lines) is made up of two parts namely the Imaging Unit
 20 (ImgU) and the CIO2 device (MIPI CSI2 receiver).
 21 
 22 The CIO2 device receives the raw Bayer data from the sensors and outputs the
 23 frames in a format that is specific to the IPU3 (for consumption by the IPU3
 24 ImgU). The CIO2 driver is available as drivers/media/pci/intel/ipu3/ipu3-cio2*
 25 and is enabled through the CONFIG_VIDEO_IPU3_CIO2 config option.
 26 
 27 The Imaging Unit (ImgU) is responsible for processing images captured
 28 by the IPU3 CIO2 device. The ImgU driver sources can be found under
 29 drivers/staging/media/ipu3 directory. The driver is enabled through the
 30 CONFIG_VIDEO_IPU3_IMGU config option.
 31 
 32 The two driver modules are named ipu3_csi2 and ipu3_imgu, respectively.
 33 
 34 The drivers has been tested on Kaby Lake platforms (U/Y processor lines).
 35 
 36 Both of the drivers implement V4L2, Media Controller and V4L2 sub-device
 37 interfaces. The IPU3 CIO2 driver supports camera sensors connected to the CIO2
 38 MIPI CSI-2 interfaces through V4L2 sub-device sensor drivers.
 39 
 40 CIO2
 41 ====
 42 
 43 The CIO2 is represented as a single V4L2 subdev, which provides a V4L2 subdev
 44 interface to the user space. There is a video node for each CSI-2 receiver,
 45 with a single media controller interface for the entire device.
 46 
 47 The CIO2 contains four independent capture channel, each with its own MIPI CSI-2
 48 receiver and DMA engine. Each channel is modelled as a V4L2 sub-device exposed
 49 to userspace as a V4L2 sub-device node and has two pads:
 50 
 51 .. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}|
 52 
 53 .. flat-table::
 54     :header-rows: 1
 55 
 56     * - Pad
 57       - Direction
 58       - Purpose
 59 
 60     * - 0
 61       - sink
 62       - MIPI CSI-2 input, connected to the sensor subdev
 63 
 64     * - 1
 65       - source
 66       - Raw video capture, connected to the V4L2 video interface
 67 
 68 The V4L2 video interfaces model the DMA engines. They are exposed to userspace
 69 as V4L2 video device nodes.
 70 
 71 Capturing frames in raw Bayer format
 72 ------------------------------------
 73 
 74 CIO2 MIPI CSI2 receiver is used to capture frames (in packed raw Bayer format)
 75 from the raw sensors connected to the CSI2 ports. The captured frames are used
 76 as input to the ImgU driver.
 77 
 78 Image processing using IPU3 ImgU requires tools such as raw2pnm [#f1]_, and
 79 yavta [#f2]_ due to the following unique requirements and / or features specific
 80 to IPU3.
 81 
 82 -- The IPU3 CSI2 receiver outputs the captured frames from the sensor in packed
 83 raw Bayer format that is specific to IPU3.
 84 
 85 -- Multiple video nodes have to be operated simultaneously.
 86 
 87 Let us take the example of ov5670 sensor connected to CSI2 port 0, for a
 88 2592x1944 image capture.
 89 
 90 Using the media controller APIs, the ov5670 sensor is configured to send
 91 frames in packed raw Bayer format to IPU3 CSI2 receiver.
 92 
 93 .. code-block:: none
 94 
 95     # This example assumes /dev/media0 as the CIO2 media device
 96     export MDEV=/dev/media0
 97 
 98     # and that ov5670 sensor is connected to i2c bus 10 with address 0x36
 99     export SDEV=$(media-ctl -d $MDEV -e "ov5670 10-0036")
100 
101     # Establish the link for the media devices using media-ctl [#f3]_
102     media-ctl -d $MDEV -l "ov5670:0 -> ipu3-csi2 0:0[1]"
103 
104     # Set the format for the media devices
105     media-ctl -d $MDEV -V "ov5670:0 [fmt:SGRBG10/2592x1944]"
106     media-ctl -d $MDEV -V "ipu3-csi2 0:0 [fmt:SGRBG10/2592x1944]"
107     media-ctl -d $MDEV -V "ipu3-csi2 0:1 [fmt:SGRBG10/2592x1944]"
108 
109 Once the media pipeline is configured, desired sensor specific settings
110 (such as exposure and gain settings) can be set, using the yavta tool.
111 
112 e.g
113 
114 .. code-block:: none
115 
116     yavta -w 0x009e0903 444 $SDEV
117     yavta -w 0x009e0913 1024 $SDEV
118     yavta -w 0x009e0911 2046 $SDEV
119 
120 Once the desired sensor settings are set, frame captures can be done as below.
121 
122 e.g
123 
124 .. code-block:: none
125 
126     yavta --data-prefix -u -c10 -n5 -I -s2592x1944 --file=/tmp/frame-#.bin \
127           -f IPU3_SGRBG10 $(media-ctl -d $MDEV -e "ipu3-cio2 0")
128 
129 With the above command, 10 frames are captured at 2592x1944 resolution, with
130 sGRBG10 format and output as IPU3_SGRBG10 format.
131 
132 The captured frames are available as /tmp/frame-#.bin files.
133 
134 ImgU
135 ====
136 
137 The ImgU is represented as two V4L2 subdevs, each of which provides a V4L2
138 subdev interface to the user space.
139 
140 Each V4L2 subdev represents a pipe, which can support a maximum of 2 streams.
141 This helps to support advanced camera features like Continuous View Finder (CVF)
142 and Snapshot During Video(SDV).
143 
144 The ImgU contains two independent pipes, each modelled as a V4L2 sub-device
145 exposed to userspace as a V4L2 sub-device node.
146 
147 Each pipe has two sink pads and three source pads for the following purpose:
148 
149 .. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}|
150 
151 .. flat-table::
152     :header-rows: 1
153 
154     * - Pad
155       - Direction
156       - Purpose
157 
158     * - 0
159       - sink
160       - Input raw video stream
161 
162     * - 1
163       - sink
164       - Processing parameters
165 
166     * - 2
167       - source
168       - Output processed video stream
169 
170     * - 3
171       - source
172       - Output viewfinder video stream
173 
174     * - 4
175       - source
176       - 3A statistics
177 
178 Each pad is connected to a corresponding V4L2 video interface, exposed to 
179 userspace as a V4L2 video device node.
180 
181 Device operation
182 ----------------
183 
184 With ImgU, once the input video node ("ipu3-imgu 0/1":0, in
185 <entity>:<pad-number> format) is queued with buffer (in packed raw Bayer
186 format), ImgU starts processing the buffer and produces the video output in YUV
187 format and statistics output on respective output nodes. The driver is expected
188 to have buffers ready for all of parameter, output and statistics nodes, when
189 input video node is queued with buffer.
190 
191 At a minimum, all of input, main output, 3A statistics and viewfinder
192 video nodes should be enabled for IPU3 to start image processing.
193 
194 Each ImgU V4L2 subdev has the following set of video nodes.
195 
196 input, output and viewfinder video nodes
197 ----------------------------------------
198 
199 The frames (in packed raw Bayer format specific to the IPU3) received by the
200 input video node is processed by the IPU3 Imaging Unit and are output to 2 video
201 nodes, with each targeting a different purpose (main output and viewfinder
202 output).
203 
204 Details onand the Bayer format specific to the IPU3 can be found in
205 :ref:`v4l2-pix-fmt-ipu3-sbggr10`.
206 
207 The driver supports V4L2 Video Capture Interface as defined at :ref:`devices`.
208 
209 Only the multi-planar API is supported. More details can be found at
210 :ref:`planar-apis`.
211 
212 Parameters video node
213 ---------------------
214 
215 The parameters video node receives the ImgU algorithm parameters that are used
216 to configure how the ImgU algorithms process the image.
217 
218 Details on processing parameters specific to the IPU3 can be found in
219 :ref:`v4l2-meta-fmt-params`.
220 
221 3A statistics video node
222 ------------------------
223 
224 3A statistics video node is used by the ImgU driver to output the 3A (auto
225 focus, auto exposure and auto white balance) statistics for the frames that are
226 being processed by the ImgU to user space applications. User space applications
227 can use this statistics data to compute the desired algorithm parameters for
228 the ImgU.
229 
230 Configuring the Intel IPU3
231 ==========================
232 
233 The IPU3 ImgU pipelines can be configured using the Media Controller, defined at
234 :ref:`media_controller`.
235 
236 Running mode and firmware binary selection
237 ------------------------------------------
238 
239 ImgU works based on firmware, currently the ImgU firmware support run 2 pipes
240 in time-sharing with single input frame data. Each pipe can run at certain mode
241 - "VIDEO" or "STILL", "VIDEO" mode is commonly used for video frames capture,
242 and "STILL" is used for still frame capture. However, you can also select
243 "VIDEO" to capture still frames if you want to capture images with less system
244 load and power. For "STILL" mode, ImgU will try to use smaller BDS factor and
245 output larger bayer frame for further YUV processing than "VIDEO" mode to get
246 high quality images. Besides, "STILL" mode need XNR3 to do noise reduction,
247 hence "STILL" mode will need more power and memory bandwidth than "VIDEO" mode.
248 TNR will be enabled in "VIDEO" mode and bypassed by "STILL" mode. ImgU is
249 running at "VIDEO" mode by default, the user can use v4l2 control
250 V4L2_CID_INTEL_IPU3_MODE (currently defined in
251 drivers/staging/media/ipu3/include/uapi/intel-ipu3.h) to query and set the
252 running mode. For user, there is no difference for buffer queueing between the
253 "VIDEO" and "STILL" mode, mandatory input and main output node should be
254 enabled and buffers need be queued, the statistics and the view-finder queues
255 are optional.
256 
257 The firmware binary will be selected according to current running mode, such log
258 "using binary if_to_osys_striped " or "using binary if_to_osys_primary_striped"
259 could be observed if you enable the ImgU dynamic debug, the binary
260 if_to_osys_striped is selected for "VIDEO" and the binary
261 "if_to_osys_primary_striped" is selected for "STILL".
262 
263 
264 Processing the image in raw Bayer format
265 ----------------------------------------
266 
267 Configuring ImgU V4L2 subdev for image processing
268 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
269 
270 The ImgU V4L2 subdevs have to be configured with media controller APIs to have
271 all the video nodes setup correctly.
272 
273 Let us take "ipu3-imgu 0" subdev as an example.
274 
275 .. code-block:: none
276 
277     media-ctl -d $MDEV -r
278     media-ctl -d $MDEV -l "ipu3-imgu 0 input":0 -> "ipu3-imgu 0":0[1]
279     media-ctl -d $MDEV -l "ipu3-imgu 0":2 -> "ipu3-imgu 0 output":0[1]
280     media-ctl -d $MDEV -l "ipu3-imgu 0":3 -> "ipu3-imgu 0 viewfinder":0[1]
281     media-ctl -d $MDEV -l "ipu3-imgu 0":4 -> "ipu3-imgu 0 3a stat":0[1]
282 
283 Also the pipe mode of the corresponding V4L2 subdev should be set as desired
284 (e.g 0 for video mode or 1 for still mode) through the control id 0x009819a1 as
285 below.
286 
287 .. code-block:: none
288 
289     yavta -w "0x009819A1 1" /dev/v4l-subdev7
290 
291 Certain hardware blocks in ImgU pipeline can change the frame resolution by
292 cropping or scaling, these hardware blocks include Input Feeder(IF), Bayer Down
293 Scaler (BDS) and Geometric Distortion Correction (GDC).
294 There is also a block which can change the frame resolution - YUV Scaler, it is
295 only applicable to the secondary output.
296 
297 RAW Bayer frames go through these ImgU pipeline hardware blocks and the final
298 processed image output to the DDR memory.
299 
300 .. kernel-figure::  ipu3_rcb.svg
301    :alt: ipu3 resolution blocks image
302 
303    IPU3 resolution change hardware blocks
304 
305 **Input Feeder**
306 
307 Input Feeder gets the Bayer frame data from the sensor, it can enable cropping
308 of lines and columns from the frame and then store pixels into device's internal
309 pixel buffer which are ready to readout by following blocks.
310 
311 **Bayer Down Scaler**
312 
313 Bayer Down Scaler is capable of performing image scaling in Bayer domain, the
314 downscale factor can be configured from 1X to 1/4X in each axis with
315 configuration steps of 0.03125 (1/32).
316 
317 **Geometric Distortion Correction**
318 
319 Geometric Distortion Correction is used to perform correction of distortions
320 and image filtering. It needs some extra filter and envelope padding pixels to
321 work, so the input resolution of GDC should be larger than the output
322 resolution.
323 
324 **YUV Scaler**
325 
326 YUV Scaler which similar with BDS, but it is mainly do image down scaling in
327 YUV domain, it can support up to 1/12X down scaling, but it can not be applied
328 to the main output.
329 
330 The ImgU V4L2 subdev has to be configured with the supported resolutions in all
331 the above hardware blocks, for a given input resolution.
332 For a given supported resolution for an input frame, the Input Feeder, Bayer
333 Down Scaler and GDC blocks should be configured with the supported resolutions
334 as each hardware block has its own alignment requirement.
335 
336 You must configure the output resolution of the hardware blocks smartly to meet
337 the hardware requirement along with keeping the maximum field of view. The
338 intermediate resolutions can be generated by specific tool -
339 
340 https://github.com/intel/intel-ipu3-pipecfg
341 
342 This tool can be used to generate intermediate resolutions. More information can
343 be obtained by looking at the following IPU3 ImgU configuration table.
344 
345 https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/master
346 
347 Under baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files/gcss
348 directory, graph_settings_ov5670.xml can be used as an example.
349 
350 The following steps prepare the ImgU pipeline for the image processing.
351 
352 1. The ImgU V4L2 subdev data format should be set by using the
353 VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained above.
354 
355 2. The ImgU V4L2 subdev cropping should be set by using the
356 VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the target,
357 using the input feeder height and width.
358 
359 3. The ImgU V4L2 subdev composing should be set by using the
360 VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the target,
361 using the BDS height and width.
362 
363 For the ov5670 example, for an input frame with a resolution of 2592x1944
364 (which is input to the ImgU subdev pad 0), the corresponding resolutions
365 for input feeder, BDS and GDC are 2592x1944, 2592x1944 and 2560x1920
366 respectively.
367 
368 Once this is done, the received raw Bayer frames can be input to the ImgU
369 V4L2 subdev as below, using the open source application v4l2n [#f1]_.
370 
371 For an image captured with 2592x1944 [#f4]_ resolution, with desired output
372 resolution as 2560x1920 and viewfinder resolution as 2560x1920, the following
373 v4l2n command can be used. This helps process the raw Bayer frames and produces
374 the desired results for the main output image and the viewfinder output, in NV12
375 format.
376 
377 .. code-block:: none
378 
379     v4l2n --pipe=4 --load=/tmp/frame-#.bin --open=/dev/video4
380           --fmt=type:VIDEO_OUTPUT_MPLANE,width=2592,height=1944,pixelformat=0X47337069 \
381           --reqbufs=type:VIDEO_OUTPUT_MPLANE,count:1 --pipe=1 \
382           --output=/tmp/frames.out --open=/dev/video5 \
383           --fmt=type:VIDEO_CAPTURE_MPLANE,width=2560,height=1920,pixelformat=NV12 \
384           --reqbufs=type:VIDEO_CAPTURE_MPLANE,count:1 --pipe=2 \
385           --output=/tmp/frames.vf --open=/dev/video6 \
386           --fmt=type:VIDEO_CAPTURE_MPLANE,width=2560,height=1920,pixelformat=NV12 \
387           --reqbufs=type:VIDEO_CAPTURE_MPLANE,count:1 --pipe=3 --open=/dev/video7 \
388           --output=/tmp/frames.3A --fmt=type:META_CAPTURE,? \
389           --reqbufs=count:1,type:META_CAPTURE --pipe=1,2,3,4 --stream=5
390 
391 You can also use yavta [#f2]_ command to do same thing as above:
392 
393 .. code-block:: none
394 
395     yavta --data-prefix -Bcapture-mplane -c10 -n5 -I -s2592x1944 \
396           --file=frame-#.out-f NV12 /dev/video5 & \
397     yavta --data-prefix -Bcapture-mplane -c10 -n5 -I -s2592x1944 \
398           --file=frame-#.vf -f NV12 /dev/video6 & \
399     yavta --data-prefix -Bmeta-capture -c10 -n5 -I \
400           --file=frame-#.3a /dev/video7 & \
401     yavta --data-prefix -Boutput-mplane -c10 -n5 -I -s2592x1944 \
402           --file=/tmp/frame-in.cio2 -f IPU3_SGRBG10 /dev/video4
403 
404 where /dev/video4, /dev/video5, /dev/video6 and /dev/video7 devices point to
405 input, output, viewfinder and 3A statistics video nodes respectively.
406 
407 Converting the raw Bayer image into YUV domain
408 ----------------------------------------------
409 
410 The processed images after the above step, can be converted to YUV domain
411 as below.
412 
413 Main output frames
414 ~~~~~~~~~~~~~~~~~~
415 
416 .. code-block:: none
417 
418     raw2pnm -x2560 -y1920 -fNV12 /tmp/frames.out /tmp/frames.out.ppm
419 
420 where 2560x1920 is output resolution, NV12 is the video format, followed
421 by input frame and output PNM file.
422 
423 Viewfinder output frames
424 ~~~~~~~~~~~~~~~~~~~~~~~~
425 
426 .. code-block:: none
427 
428     raw2pnm -x2560 -y1920 -fNV12 /tmp/frames.vf /tmp/frames.vf.ppm
429 
430 where 2560x1920 is output resolution, NV12 is the video format, followed
431 by input frame and output PNM file.
432 
433 Example user space code for IPU3
434 ================================
435 
436 User space code that configures and uses IPU3 is available here.
437 
438 https://chromium.googlesource.com/chromiumos/platform/arc-camera/+/master/
439 
440 The source can be located under hal/intel directory.
441 
442 Overview of IPU3 pipeline
443 =========================
444 
445 IPU3 pipeline has a number of image processing stages, each of which takes a
446 set of parameters as input. The major stages of pipelines are shown here:
447 
448 .. kernel-render:: DOT
449    :alt: IPU3 ImgU Pipeline
450    :caption: IPU3 ImgU Pipeline Diagram
451 
452    digraph "IPU3 ImgU" {
453        node [shape=box]
454        splines="ortho"
455        rankdir="LR"
456 
457        a [label="Raw pixels"]
458        b [label="Bayer Downscaling"]
459        c [label="Optical Black Correction"]
460        d [label="Linearization"]
461        e [label="Lens Shading Correction"]
462        f [label="White Balance / Exposure / Focus Apply"]
463        g [label="Bayer Noise Reduction"]
464        h [label="ANR"]
465        i [label="Demosaicing"]
466        j [label="Color Correction Matrix"]
467        k [label="Gamma correction"]
468        l [label="Color Space Conversion"]
469        m [label="Chroma Down Scaling"]
470        n [label="Chromatic Noise Reduction"]
471        o [label="Total Color Correction"]
472        p [label="XNR3"]
473        q [label="TNR"]
474        r [label="DDR", style=filled, fillcolor=yellow, shape=cylinder]
475        s [label="YUV Downscaling"]
476        t [label="DDR", style=filled, fillcolor=yellow, shape=cylinder]
477 
478        { rank=same; a -> b -> c -> d -> e -> f -> g -> h -> i }
479        { rank=same; j -> k -> l -> m -> n -> o -> p -> q -> s -> t}
480 
481        a -> j [style=invis, weight=10]
482        i -> j
483        q -> r
484    }
485 
486 The table below presents a description of the above algorithms.
487 
488 ======================== =======================================================
489 Name                     Description
490 ======================== =======================================================
491 Optical Black Correction Optical Black Correction block subtracts a pre-defined
492                          value from the respective pixel values to obtain better
493                          image quality.
494                          Defined in struct ipu3_uapi_obgrid_param.
495 Linearization            This algo block uses linearization parameters to
496                          address non-linearity sensor effects. The Lookup table
497                          table is defined in
498                          struct ipu3_uapi_isp_lin_vmem_params.
499 SHD                      Lens shading correction is used to correct spatial
500                          non-uniformity of the pixel response due to optical
501                          lens shading. This is done by applying a different gain
502                          for each pixel. The gain, black level etc are
503                          configured in struct ipu3_uapi_shd_config_static.
504 BNR                      Bayer noise reduction block removes image noise by
505                          applying a bilateral filter.
506                          See struct ipu3_uapi_bnr_static_config for details.
507 ANR                      Advanced Noise Reduction is a block based algorithm
508                          that performs noise reduction in the Bayer domain. The
509                          convolution matrix etc can be found in
510                          struct ipu3_uapi_anr_config.
511 DM                       Demosaicing converts raw sensor data in Bayer format
512                          into RGB (Red, Green, Blue) presentation. Then add
513                          outputs of estimation of Y channel for following stream
514                          processing by Firmware. The struct is defined as
515                          struct ipu3_uapi_dm_config.
516 Color Correction         Color Correction algo transforms sensor specific color
517                          space to the standard "sRGB" color space. This is done
518                          by applying 3x3 matrix defined in
519                          struct ipu3_uapi_ccm_mat_config.
520 Gamma correction         Gamma correction struct ipu3_uapi_gamma_config is a
521                          basic non-linear tone mapping correction that is
522                          applied per pixel for each pixel component.
523 CSC                      Color space conversion transforms each pixel from the
524                          RGB primary presentation to YUV (Y: brightness,
525                          UV: Luminance) presentation. This is done by applying
526                          a 3x3 matrix defined in
527                          struct ipu3_uapi_csc_mat_config
528 CDS                      Chroma down sampling
529                          After the CSC is performed, the Chroma Down Sampling
530                          is applied for a UV plane down sampling by a factor
531                          of 2 in each direction for YUV 4:2:0 using a 4x2
532                          configurable filter struct ipu3_uapi_cds_params.
533 CHNR                     Chroma noise reduction
534                          This block processes only the chrominance pixels and
535                          performs noise reduction by cleaning the high
536                          frequency noise.
537                          See struct struct ipu3_uapi_yuvp1_chnr_config.
538 TCC                      Total color correction as defined in struct
539                          struct ipu3_uapi_yuvp2_tcc_static_config.
540 XNR3                     eXtreme Noise Reduction V3 is the third revision of
541                          noise reduction algorithm used to improve image
542                          quality. This removes the low frequency noise in the
543                          captured image. Two related structs are  being defined,
544                          struct ipu3_uapi_isp_xnr3_params for ISP data memory
545                          and struct ipu3_uapi_isp_xnr3_vmem_params for vector
546                          memory.
547 TNR                      Temporal Noise Reduction block compares successive
548                          frames in time to remove anomalies / noise in pixel
549                          values. struct ipu3_uapi_isp_tnr3_vmem_params and
550                          struct ipu3_uapi_isp_tnr3_params are defined for ISP
551                          vector and data memory respectively.
552 ======================== =======================================================
553 
554 Other often encountered acronyms not listed in above table:
555 
556         ACC
557                 Accelerator cluster
558         AWB_FR
559                 Auto white balance filter response statistics
560         BDS
561                 Bayer downscaler parameters
562         CCM
563                 Color correction matrix coefficients
564         IEFd
565                 Image enhancement filter directed
566         Obgrid
567                 Optical black level compensation
568         OSYS
569                 Output system configuration
570         ROI
571                 Region of interest
572         YDS
573                 Y down sampling
574         YTM
575                 Y-tone mapping
576 
577 A few stages of the pipeline will be executed by firmware running on the ISP
578 processor, while many others will use a set of fixed hardware blocks also
579 called accelerator cluster (ACC) to crunch pixel data and produce statistics.
580 
581 ACC parameters of individual algorithms, as defined by
582 struct ipu3_uapi_acc_param, can be chosen to be applied by the user
583 space through struct struct ipu3_uapi_flags embedded in
584 struct ipu3_uapi_params structure. For parameters that are configured as
585 not enabled by the user space, the corresponding structs are ignored by the
586 driver, in which case the existing configuration of the algorithm will be
587 preserved.
588 
589 References
590 ==========
591 
592 .. [#f5] drivers/staging/media/ipu3/include/uapi/intel-ipu3.h
593 
594 .. [#f1] https://github.com/intel/nvt
595 
596 .. [#f2] http://git.ideasonboard.org/yavta.git
597 
598 .. [#f3] http://git.ideasonboard.org/?p=media-ctl.git;a=summary
599 
600 .. [#f4] ImgU limitation requires an additional 16x16 for all input resolutions

~ [ source navigation ] ~ [ diff markup ] ~ [ identifier search ] ~

kernel.org | git.kernel.org | LWN.net | Project Home | SVN repository | Mail admin

Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.

sflogo.php