1 .. SPDX-License-Identifier: GFDL-1.1-no-invari !! 1 .. Permission is granted to copy, distribute and/or modify this
>> 2 .. document under the terms of the GNU Free Documentation License,
>> 3 .. Version 1.1 or any later version published by the Free Software
>> 4 .. Foundation, with no Invariant Sections, no Front-Cover Texts
>> 5 .. and no Back-Cover Texts. A copy of the license is included at
>> 6 .. Documentation/userspace-api/media/fdl-appendix.rst.
>> 7 ..
>> 8 .. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
2 9
3 .. _crop: 10 .. _crop:
4 11
5 ********************************************** 12 *****************************************************
6 Image Cropping, Insertion and Scaling -- the C 13 Image Cropping, Insertion and Scaling -- the CROP API
7 ********************************************** 14 *****************************************************
8 15
9 .. note:: 16 .. note::
10 17
11 The CROP API is mostly superseded by the ne 18 The CROP API is mostly superseded by the newer :ref:`SELECTION API
12 <selection-api>`. The new API should be pre 19 <selection-api>`. The new API should be preferred in most cases,
13 with the exception of pixel aspect ratio de 20 with the exception of pixel aspect ratio detection, which is
14 implemented by :ref:`VIDIOC_CROPCAP <VIDIOC 21 implemented by :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` and has no
15 equivalent in the SELECTION API. See :ref:` 22 equivalent in the SELECTION API. See :ref:`selection-vs-crop` for a
16 comparison of the two APIs. 23 comparison of the two APIs.
17 24
18 Some video capture devices can sample a subsec 25 Some video capture devices can sample a subsection of the picture and
19 shrink or enlarge it to an image of arbitrary 26 shrink or enlarge it to an image of arbitrary size. We call these
20 abilities cropping and scaling. Some video out 27 abilities cropping and scaling. Some video output devices can scale an
21 image up or down and insert it at an arbitrary 28 image up or down and insert it at an arbitrary scan line and horizontal
22 offset into a video signal. 29 offset into a video signal.
23 30
24 Applications can use the following API to sele 31 Applications can use the following API to select an area in the video
25 signal, query the default area and the hardwar 32 signal, query the default area and the hardware limits.
26 33
27 .. note:: 34 .. note::
28 35
29 Despite their name, the :ref:`VIDIOC_CROPCA 36 Despite their name, the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>`,
30 :ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and :r 37 :ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and :ref:`VIDIOC_S_CROP
31 <VIDIOC_G_CROP>` ioctls apply to input as w 38 <VIDIOC_G_CROP>` ioctls apply to input as well as output devices.
32 39
33 Scaling requires a source and a target. On a v 40 Scaling requires a source and a target. On a video capture or overlay
34 device the source is the video signal, and the 41 device the source is the video signal, and the cropping ioctls determine
35 the area actually sampled. The target are imag 42 the area actually sampled. The target are images read by the application
36 or overlaid onto the graphics screen. Their si 43 or overlaid onto the graphics screen. Their size (and position for an
37 overlay) is negotiated with the :ref:`VIDIOC_G 44 overlay) is negotiated with the :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>`
38 and :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctls. 45 and :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctls.
39 46
40 On a video output device the source are the im 47 On a video output device the source are the images passed in by the
41 application, and their size is again negotiate 48 application, and their size is again negotiated with the
42 :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` and :ref:`V 49 :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` and :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`
43 ioctls, or may be encoded in a compressed vide 50 ioctls, or may be encoded in a compressed video stream. The target is
44 the video signal, and the cropping ioctls dete 51 the video signal, and the cropping ioctls determine the area where the
45 images are inserted. 52 images are inserted.
46 53
47 Source and target rectangles are defined even 54 Source and target rectangles are defined even if the device does not
48 support scaling or the :ref:`VIDIOC_G_CROP <VI 55 support scaling or the :ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and
49 :ref:`VIDIOC_S_CROP <VIDIOC_G_CROP>` ioctls. T 56 :ref:`VIDIOC_S_CROP <VIDIOC_G_CROP>` ioctls. Their size (and position
50 where applicable) will be fixed in this case. 57 where applicable) will be fixed in this case.
51 58
52 .. note:: 59 .. note::
53 60
54 All capture and output devices that support 61 All capture and output devices that support the CROP or SELECTION
55 API will also support the :ref:`VIDIOC_CROP 62 API will also support the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>`
56 ioctl. 63 ioctl.
57 64
58 Cropping Structures 65 Cropping Structures
59 =================== 66 ===================
60 67
61 68
62 .. _crop-scale: 69 .. _crop-scale:
63 70
64 .. kernel-figure:: crop.svg 71 .. kernel-figure:: crop.svg
65 :alt: crop.svg 72 :alt: crop.svg
66 :align: center 73 :align: center
67 74
68 Image Cropping, Insertion and Scaling 75 Image Cropping, Insertion and Scaling
69 76
70 The cropping, insertion and scaling proces 77 The cropping, insertion and scaling process
71 78
72 79
73 80
74 For capture devices the coordinates of the top 81 For capture devices the coordinates of the top left corner, width and
75 height of the area which can be sampled is giv 82 height of the area which can be sampled is given by the ``bounds``
76 substructure of the struct :c:type:`v4l2_cropc 83 substructure of the struct :c:type:`v4l2_cropcap` returned
77 by the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` 84 by the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` ioctl. To support a wide
78 range of hardware this specification does not 85 range of hardware this specification does not define an origin or units.
79 However by convention drivers should horizonta 86 However by convention drivers should horizontally count unscaled samples
80 relative to 0H (the leading edge of the horizo 87 relative to 0H (the leading edge of the horizontal sync pulse, see
81 :ref:`vbi-hsync`). Vertically ITU-R line numbe 88 :ref:`vbi-hsync`). Vertically ITU-R line numbers of the first field
82 (see ITU R-525 line numbering for :ref:`525 li 89 (see ITU R-525 line numbering for :ref:`525 lines <vbi-525>` and for
83 :ref:`625 lines <vbi-625>`), multiplied by two 90 :ref:`625 lines <vbi-625>`), multiplied by two if the driver
84 can capture both fields. 91 can capture both fields.
85 92
86 The top left corner, width and height of the s 93 The top left corner, width and height of the source rectangle, that is
87 the area actually sampled, is given by struct 94 the area actually sampled, is given by struct
88 :c:type:`v4l2_crop` using the same coordinate 95 :c:type:`v4l2_crop` using the same coordinate system as
89 struct :c:type:`v4l2_cropcap`. Applications ca 96 struct :c:type:`v4l2_cropcap`. Applications can use the
90 :ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and :ref: 97 :ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and :ref:`VIDIOC_S_CROP <VIDIOC_G_CROP>`
91 ioctls to get and set this rectangle. It must 98 ioctls to get and set this rectangle. It must lie completely within the
92 capture boundaries and the driver may further 99 capture boundaries and the driver may further adjust the requested size
93 and/or position according to hardware limitati 100 and/or position according to hardware limitations.
94 101
95 Each capture device has a default source recta 102 Each capture device has a default source rectangle, given by the
96 ``defrect`` substructure of struct 103 ``defrect`` substructure of struct
97 :c:type:`v4l2_cropcap`. The center of this rec 104 :c:type:`v4l2_cropcap`. The center of this rectangle
98 shall align with the center of the active pict 105 shall align with the center of the active picture area of the video
99 signal, and cover what the driver writer consi 106 signal, and cover what the driver writer considers the complete picture.
100 Drivers shall reset the source rectangle to th 107 Drivers shall reset the source rectangle to the default when the driver
101 is first loaded, but not later. 108 is first loaded, but not later.
102 109
103 For output devices these structures and ioctls 110 For output devices these structures and ioctls are used accordingly,
104 defining the *target* rectangle where the imag 111 defining the *target* rectangle where the images will be inserted into
105 the video signal. 112 the video signal.
106 113
107 114
108 Scaling Adjustments 115 Scaling Adjustments
109 =================== 116 ===================
110 117
111 Video hardware can have various cropping, inse 118 Video hardware can have various cropping, insertion and scaling
112 limitations. It may only scale up or down, sup 119 limitations. It may only scale up or down, support only discrete scaling
113 factors, or have different scaling abilities i 120 factors, or have different scaling abilities in horizontal and vertical
114 direction. Also it may not support scaling at 121 direction. Also it may not support scaling at all. At the same time the
115 struct :c:type:`v4l2_crop` rectangle may have 122 struct :c:type:`v4l2_crop` rectangle may have to be aligned,
116 and both the source and target rectangles may 123 and both the source and target rectangles may have arbitrary upper and
117 lower size limits. In particular the maximum ` 124 lower size limits. In particular the maximum ``width`` and ``height`` in
118 struct :c:type:`v4l2_crop` may be smaller than 125 struct :c:type:`v4l2_crop` may be smaller than the struct
119 :c:type:`v4l2_cropcap`. ``bounds`` area. There 126 :c:type:`v4l2_cropcap`. ``bounds`` area. Therefore, as
120 usual, drivers are expected to adjust the requ 127 usual, drivers are expected to adjust the requested parameters and
121 return the actual values selected. 128 return the actual values selected.
122 129
123 Applications can change the source or the targ 130 Applications can change the source or the target rectangle first, as
124 they may prefer a particular image size or a c 131 they may prefer a particular image size or a certain area in the video
125 signal. If the driver has to adjust both to sa 132 signal. If the driver has to adjust both to satisfy hardware
126 limitations, the last requested rectangle shal 133 limitations, the last requested rectangle shall take priority, and the
127 driver should preferably adjust the opposite o 134 driver should preferably adjust the opposite one. The
128 :ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl how 135 :ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl however shall not change
129 the driver state and therefore only adjust the 136 the driver state and therefore only adjust the requested rectangle.
130 137
131 Suppose scaling on a video capture device is r 138 Suppose scaling on a video capture device is restricted to a factor 1:1
132 or 2:1 in either direction and the target imag 139 or 2:1 in either direction and the target image size must be a multiple
133 of 16 × 16 pixels. The source cropping rectan 140 of 16 × 16 pixels. The source cropping rectangle is set to defaults,
134 which are also the upper limit in this example 141 which are also the upper limit in this example, of 640 × 400 pixels at
135 offset 0, 0. An application requests an image 142 offset 0, 0. An application requests an image size of 300 × 225 pixels,
136 assuming video will be scaled down from the "f 143 assuming video will be scaled down from the "full picture" accordingly.
137 The driver sets the image size to the closest 144 The driver sets the image size to the closest possible values 304 × 224,
138 then chooses the cropping rectangle closest to 145 then chooses the cropping rectangle closest to the requested size, that
139 is 608 × 224 (224 × 2:1 would exceed the lim 146 is 608 × 224 (224 × 2:1 would exceed the limit 400). The offset 0, 0 is
140 still valid, thus unmodified. Given the defaul 147 still valid, thus unmodified. Given the default cropping rectangle
141 reported by :ref:`VIDIOC_CROPCAP <VIDIOC_CROPC 148 reported by :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` the application can
142 easily propose another offset to center the cr 149 easily propose another offset to center the cropping rectangle.
143 150
144 Now the application may insist on covering an 151 Now the application may insist on covering an area using a picture
145 aspect ratio closer to the original request, s 152 aspect ratio closer to the original request, so it asks for a cropping
146 rectangle of 608 × 456 pixels. The present sc 153 rectangle of 608 × 456 pixels. The present scaling factors limit
147 cropping to 640 × 384, so the driver returns 154 cropping to 640 × 384, so the driver returns the cropping size 608 × 384
148 and adjusts the image size to closest possible 155 and adjusts the image size to closest possible 304 × 192.
149 156
150 157
151 Examples 158 Examples
152 ======== 159 ========
153 160
154 Source and target rectangles shall remain unch 161 Source and target rectangles shall remain unchanged across closing and
155 reopening a device, such that piping data into 162 reopening a device, such that piping data into or out of a device will
156 work without special preparations. More advanc 163 work without special preparations. More advanced applications should
157 ensure the parameters are suitable before star 164 ensure the parameters are suitable before starting I/O.
158 165
159 .. note:: 166 .. note::
160 167
161 On the next two examples, a video capture d 168 On the next two examples, a video capture device is assumed;
162 change ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` for 169 change ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` for other types of device.
163 170
164 Example: Resetting the cropping parameters 171 Example: Resetting the cropping parameters
165 ========================================== 172 ==========================================
166 173
167 .. code-block:: c 174 .. code-block:: c
168 175
169 struct v4l2_cropcap cropcap; 176 struct v4l2_cropcap cropcap;
170 struct v4l2_crop crop; 177 struct v4l2_crop crop;
171 178
172 memset (&cropcap, 0, sizeof (cropcap)); 179 memset (&cropcap, 0, sizeof (cropcap));
173 cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE 180 cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
174 181
175 if (-1 == ioctl (fd, VIDIOC_CROPCAP, &crop 182 if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) {
176 perror ("VIDIOC_CROPCAP"); 183 perror ("VIDIOC_CROPCAP");
177 exit (EXIT_FAILURE); 184 exit (EXIT_FAILURE);
178 } 185 }
179 186
180 memset (&crop, 0, sizeof (crop)); 187 memset (&crop, 0, sizeof (crop));
181 crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; 188 crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
182 crop.c = cropcap.defrect; 189 crop.c = cropcap.defrect;
183 190
184 /* Ignore if cropping is not supported (EI 191 /* Ignore if cropping is not supported (EINVAL). */
185 192
186 if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop) 193 if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop)
187 && errno != EINVAL) { 194 && errno != EINVAL) {
188 perror ("VIDIOC_S_CROP"); 195 perror ("VIDIOC_S_CROP");
189 exit (EXIT_FAILURE); 196 exit (EXIT_FAILURE);
190 } 197 }
191 198
192 199
193 Example: Simple downscaling 200 Example: Simple downscaling
194 =========================== 201 ===========================
195 202
196 .. code-block:: c 203 .. code-block:: c
197 204
198 struct v4l2_cropcap cropcap; 205 struct v4l2_cropcap cropcap;
199 struct v4l2_format format; 206 struct v4l2_format format;
200 207
201 reset_cropping_parameters (); 208 reset_cropping_parameters ();
202 209
203 /* Scale down to 1/4 size of full picture. 210 /* Scale down to 1/4 size of full picture. */
204 211
205 memset (&format, 0, sizeof (format)); /* d 212 memset (&format, 0, sizeof (format)); /* defaults */
206 213
207 format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; 214 format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
208 215
209 format.fmt.pix.width = cropcap.defrect.wid 216 format.fmt.pix.width = cropcap.defrect.width >> 1;
210 format.fmt.pix.height = cropcap.defrect.he 217 format.fmt.pix.height = cropcap.defrect.height >> 1;
211 format.fmt.pix.pixelformat = V4L2_PIX_FMT_ 218 format.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
212 219
213 if (-1 == ioctl (fd, VIDIOC_S_FMT, &format 220 if (-1 == ioctl (fd, VIDIOC_S_FMT, &format)) {
214 perror ("VIDIOC_S_FORMAT"); 221 perror ("VIDIOC_S_FORMAT");
215 exit (EXIT_FAILURE); 222 exit (EXIT_FAILURE);
216 } 223 }
217 224
218 /* We could check the actual image size no 225 /* We could check the actual image size now, the actual scaling factor
219 or if the driver can scale at all. */ 226 or if the driver can scale at all. */
220 227
221 Example: Selecting an output area 228 Example: Selecting an output area
222 ================================= 229 =================================
223 230
224 .. note:: This example assumes an output devic 231 .. note:: This example assumes an output device.
225 232
226 .. code-block:: c 233 .. code-block:: c
227 234
228 struct v4l2_cropcap cropcap; 235 struct v4l2_cropcap cropcap;
229 struct v4l2_crop crop; 236 struct v4l2_crop crop;
230 237
231 memset (&cropcap, 0, sizeof (cropcap)); 238 memset (&cropcap, 0, sizeof (cropcap));
232 cropcap.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; 239 cropcap.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
233 240
234 if (-1 == ioctl (fd, VIDIOC_CROPCAP;, &cro 241 if (-1 == ioctl (fd, VIDIOC_CROPCAP;, &cropcap)) {
235 perror ("VIDIOC_CROPCAP"); 242 perror ("VIDIOC_CROPCAP");
236 exit (EXIT_FAILURE); 243 exit (EXIT_FAILURE);
237 } 244 }
238 245
239 memset (&crop, 0, sizeof (crop)); 246 memset (&crop, 0, sizeof (crop));
240 247
241 crop.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; 248 crop.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
242 crop.c = cropcap.defrect; 249 crop.c = cropcap.defrect;
243 250
244 /* Scale the width and height to 50 % of t 251 /* Scale the width and height to 50 % of their original size
245 and center the output. */ 252 and center the output. */
246 253
247 crop.c.width /= 2; 254 crop.c.width /= 2;
248 crop.c.height /= 2; 255 crop.c.height /= 2;
249 crop.c.left += crop.c.width / 2; 256 crop.c.left += crop.c.width / 2;
250 crop.c.top += crop.c.height / 2; 257 crop.c.top += crop.c.height / 2;
251 258
252 /* Ignore if cropping is not supported (EI 259 /* Ignore if cropping is not supported (EINVAL). */
253 260
254 if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop) 261 if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop)
255 && errno != EINVAL) { 262 && errno != EINVAL) {
256 perror ("VIDIOC_S_CROP"); 263 perror ("VIDIOC_S_CROP");
257 exit (EXIT_FAILURE); 264 exit (EXIT_FAILURE);
258 } 265 }
259 266
260 Example: Current scaling factor and pixel aspe 267 Example: Current scaling factor and pixel aspect
261 ============================================== 268 ================================================
262 269
263 .. note:: This example assumes a video capture 270 .. note:: This example assumes a video capture device.
264 271
265 .. code-block:: c 272 .. code-block:: c
266 273
267 struct v4l2_cropcap cropcap; 274 struct v4l2_cropcap cropcap;
268 struct v4l2_crop crop; 275 struct v4l2_crop crop;
269 struct v4l2_format format; 276 struct v4l2_format format;
270 double hscale, vscale; 277 double hscale, vscale;
271 double aspect; 278 double aspect;
272 int dwidth, dheight; 279 int dwidth, dheight;
273 280
274 memset (&cropcap, 0, sizeof (cropcap)); 281 memset (&cropcap, 0, sizeof (cropcap));
275 cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE 282 cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
276 283
277 if (-1 == ioctl (fd, VIDIOC_CROPCAP, &crop 284 if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) {
278 perror ("VIDIOC_CROPCAP"); 285 perror ("VIDIOC_CROPCAP");
279 exit (EXIT_FAILURE); 286 exit (EXIT_FAILURE);
280 } 287 }
281 288
282 memset (&crop, 0, sizeof (crop)); 289 memset (&crop, 0, sizeof (crop));
283 crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; 290 crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
284 291
285 if (-1 == ioctl (fd, VIDIOC_G_CROP, &crop) 292 if (-1 == ioctl (fd, VIDIOC_G_CROP, &crop)) {
286 if (errno != EINVAL) { 293 if (errno != EINVAL) {
287 perror ("VIDIOC_G_CROP"); 294 perror ("VIDIOC_G_CROP");
288 exit (EXIT_FAILURE); 295 exit (EXIT_FAILURE);
289 } 296 }
290 297
291 /* Cropping not supported. */ 298 /* Cropping not supported. */
292 crop.c = cropcap.defrect; 299 crop.c = cropcap.defrect;
293 } 300 }
294 301
295 memset (&format, 0, sizeof (format)); 302 memset (&format, 0, sizeof (format));
296 format.fmt.type = V4L2_BUF_TYPE_VIDEO_CAPT 303 format.fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
297 304
298 if (-1 == ioctl (fd, VIDIOC_G_FMT, &format 305 if (-1 == ioctl (fd, VIDIOC_G_FMT, &format)) {
299 perror ("VIDIOC_G_FMT"); 306 perror ("VIDIOC_G_FMT");
300 exit (EXIT_FAILURE); 307 exit (EXIT_FAILURE);
301 } 308 }
302 309
303 /* The scaling applied by the driver. */ 310 /* The scaling applied by the driver. */
304 311
305 hscale = format.fmt.pix.width / (double) c 312 hscale = format.fmt.pix.width / (double) crop.c.width;
306 vscale = format.fmt.pix.height / (double) 313 vscale = format.fmt.pix.height / (double) crop.c.height;
307 314
308 aspect = cropcap.pixelaspect.numerator / 315 aspect = cropcap.pixelaspect.numerator /
309 (double) cropcap.pixelaspect.denomina 316 (double) cropcap.pixelaspect.denominator;
310 aspect = aspect * hscale / vscale; 317 aspect = aspect * hscale / vscale;
311 318
312 /* Devices following ITU-R BT.601 do not c 319 /* Devices following ITU-R BT.601 do not capture
313 square pixels. For playback on a comput 320 square pixels. For playback on a computer monitor
314 we should scale the images to this size 321 we should scale the images to this size. */
315 322
316 dwidth = format.fmt.pix.width / aspect; 323 dwidth = format.fmt.pix.width / aspect;
317 dheight = format.fmt.pix.height; 324 dheight = format.fmt.pix.height;
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.