1 .. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
2
3 .. _crop:
4
5 *****************************************************
6 Image Cropping, Insertion and Scaling -- the CROP API
7 *****************************************************
8
9 .. note::
10
11 The CROP API is mostly superseded by the newer :ref:`SELECTION API
12 <selection-api>`. The new API should be preferred in most cases,
13 with the exception of pixel aspect ratio detection, which is
14 implemented by :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` and has no
15 equivalent in the SELECTION API. See :ref:`selection-vs-crop` for a
16 comparison of the two APIs.
17
18 Some video capture devices can sample a subsection of the picture and
19 shrink or enlarge it to an image of arbitrary size. We call these
20 abilities cropping and scaling. Some video output devices can scale an
21 image up or down and insert it at an arbitrary scan line and horizontal
22 offset into a video signal.
23
24 Applications can use the following API to select an area in the video
25 signal, query the default area and the hardware limits.
26
27 .. note::
28
29 Despite their name, the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>`,
30 :ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and :ref:`VIDIOC_S_CROP
31 <VIDIOC_G_CROP>` ioctls apply to input as well as output devices.
32
33 Scaling requires a source and a target. On a video capture or overlay
34 device the source is the video signal, and the cropping ioctls determine
35 the area actually sampled. The target are images read by the application
36 or overlaid onto the graphics screen. Their size (and position for an
37 overlay) is negotiated with the :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>`
38 and :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctls.
39
40 On a video output device the source are the images passed in by the
41 application, and their size is again negotiated with the
42 :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` and :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`
43 ioctls, or may be encoded in a compressed video stream. The target is
44 the video signal, and the cropping ioctls determine the area where the
45 images are inserted.
46
47 Source and target rectangles are defined even if the device does not
48 support scaling or the :ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and
49 :ref:`VIDIOC_S_CROP <VIDIOC_G_CROP>` ioctls. Their size (and position
50 where applicable) will be fixed in this case.
51
52 .. note::
53
54 All capture and output devices that support the CROP or SELECTION
55 API will also support the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>`
56 ioctl.
57
58 Cropping Structures
59 ===================
60
61
62 .. _crop-scale:
63
64 .. kernel-figure:: crop.svg
65 :alt: crop.svg
66 :align: center
67
68 Image Cropping, Insertion and Scaling
69
70 The cropping, insertion and scaling process
71
72
73
74 For capture devices the coordinates of the top left corner, width and
75 height of the area which can be sampled is given by the ``bounds``
76 substructure of the struct :c:type:`v4l2_cropcap` returned
77 by the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` ioctl. To support a wide
78 range of hardware this specification does not define an origin or units.
79 However by convention drivers should horizontally count unscaled samples
80 relative to 0H (the leading edge of the horizontal sync pulse, see
81 :ref:`vbi-hsync`). Vertically ITU-R line numbers of the first field
82 (see ITU R-525 line numbering for :ref:`525 lines <vbi-525>` and for
83 :ref:`625 lines <vbi-625>`), multiplied by two if the driver
84 can capture both fields.
85
86 The top left corner, width and height of the source rectangle, that is
87 the area actually sampled, is given by struct
88 :c:type:`v4l2_crop` using the same coordinate system as
89 struct :c:type:`v4l2_cropcap`. Applications can use the
90 :ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and :ref:`VIDIOC_S_CROP <VIDIOC_G_CROP>`
91 ioctls to get and set this rectangle. It must lie completely within the
92 capture boundaries and the driver may further adjust the requested size
93 and/or position according to hardware limitations.
94
95 Each capture device has a default source rectangle, given by the
96 ``defrect`` substructure of struct
97 :c:type:`v4l2_cropcap`. The center of this rectangle
98 shall align with the center of the active picture area of the video
99 signal, and cover what the driver writer considers the complete picture.
100 Drivers shall reset the source rectangle to the default when the driver
101 is first loaded, but not later.
102
103 For output devices these structures and ioctls are used accordingly,
104 defining the *target* rectangle where the images will be inserted into
105 the video signal.
106
107
108 Scaling Adjustments
109 ===================
110
111 Video hardware can have various cropping, insertion and scaling
112 limitations. It may only scale up or down, support only discrete scaling
113 factors, or have different scaling abilities in horizontal and vertical
114 direction. Also it may not support scaling at all. At the same time the
115 struct :c:type:`v4l2_crop` rectangle may have to be aligned,
116 and both the source and target rectangles may have arbitrary upper and
117 lower size limits. In particular the maximum ``width`` and ``height`` in
118 struct :c:type:`v4l2_crop` may be smaller than the struct
119 :c:type:`v4l2_cropcap`. ``bounds`` area. Therefore, as
120 usual, drivers are expected to adjust the requested parameters and
121 return the actual values selected.
122
123 Applications can change the source or the target rectangle first, as
124 they may prefer a particular image size or a certain area in the video
125 signal. If the driver has to adjust both to satisfy hardware
126 limitations, the last requested rectangle shall take priority, and the
127 driver should preferably adjust the opposite one. The
128 :ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl however shall not change
129 the driver state and therefore only adjust the requested rectangle.
130
131 Suppose scaling on a video capture device is restricted to a factor 1:1
132 or 2:1 in either direction and the target image size must be a multiple
133 of 16 × 16 pixels. The source cropping rectangle is set to defaults,
134 which are also the upper limit in this example, of 640 × 400 pixels at
135 offset 0, 0. An application requests an image size of 300 × 225 pixels,
136 assuming video will be scaled down from the "full picture" accordingly.
137 The driver sets the image size to the closest possible values 304 × 224,
138 then chooses the cropping rectangle closest to the requested size, that
139 is 608 × 224 (224 × 2:1 would exceed the limit 400). The offset 0, 0 is
140 still valid, thus unmodified. Given the default cropping rectangle
141 reported by :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` the application can
142 easily propose another offset to center the cropping rectangle.
143
144 Now the application may insist on covering an area using a picture
145 aspect ratio closer to the original request, so it asks for a cropping
146 rectangle of 608 × 456 pixels. The present scaling factors limit
147 cropping to 640 × 384, so the driver returns the cropping size 608 × 384
148 and adjusts the image size to closest possible 304 × 192.
149
150
151 Examples
152 ========
153
154 Source and target rectangles shall remain unchanged across closing and
155 reopening a device, such that piping data into or out of a device will
156 work without special preparations. More advanced applications should
157 ensure the parameters are suitable before starting I/O.
158
159 .. note::
160
161 On the next two examples, a video capture device is assumed;
162 change ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` for other types of device.
163
164 Example: Resetting the cropping parameters
165 ==========================================
166
167 .. code-block:: c
168
169 struct v4l2_cropcap cropcap;
170 struct v4l2_crop crop;
171
172 memset (&cropcap, 0, sizeof (cropcap));
173 cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
174
175 if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) {
176 perror ("VIDIOC_CROPCAP");
177 exit (EXIT_FAILURE);
178 }
179
180 memset (&crop, 0, sizeof (crop));
181 crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
182 crop.c = cropcap.defrect;
183
184 /* Ignore if cropping is not supported (EINVAL). */
185
186 if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop)
187 && errno != EINVAL) {
188 perror ("VIDIOC_S_CROP");
189 exit (EXIT_FAILURE);
190 }
191
192
193 Example: Simple downscaling
194 ===========================
195
196 .. code-block:: c
197
198 struct v4l2_cropcap cropcap;
199 struct v4l2_format format;
200
201 reset_cropping_parameters ();
202
203 /* Scale down to 1/4 size of full picture. */
204
205 memset (&format, 0, sizeof (format)); /* defaults */
206
207 format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
208
209 format.fmt.pix.width = cropcap.defrect.width >> 1;
210 format.fmt.pix.height = cropcap.defrect.height >> 1;
211 format.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
212
213 if (-1 == ioctl (fd, VIDIOC_S_FMT, &format)) {
214 perror ("VIDIOC_S_FORMAT");
215 exit (EXIT_FAILURE);
216 }
217
218 /* We could check the actual image size now, the actual scaling factor
219 or if the driver can scale at all. */
220
221 Example: Selecting an output area
222 =================================
223
224 .. note:: This example assumes an output device.
225
226 .. code-block:: c
227
228 struct v4l2_cropcap cropcap;
229 struct v4l2_crop crop;
230
231 memset (&cropcap, 0, sizeof (cropcap));
232 cropcap.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
233
234 if (-1 == ioctl (fd, VIDIOC_CROPCAP;, &cropcap)) {
235 perror ("VIDIOC_CROPCAP");
236 exit (EXIT_FAILURE);
237 }
238
239 memset (&crop, 0, sizeof (crop));
240
241 crop.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
242 crop.c = cropcap.defrect;
243
244 /* Scale the width and height to 50 % of their original size
245 and center the output. */
246
247 crop.c.width /= 2;
248 crop.c.height /= 2;
249 crop.c.left += crop.c.width / 2;
250 crop.c.top += crop.c.height / 2;
251
252 /* Ignore if cropping is not supported (EINVAL). */
253
254 if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop)
255 && errno != EINVAL) {
256 perror ("VIDIOC_S_CROP");
257 exit (EXIT_FAILURE);
258 }
259
260 Example: Current scaling factor and pixel aspect
261 ================================================
262
263 .. note:: This example assumes a video capture device.
264
265 .. code-block:: c
266
267 struct v4l2_cropcap cropcap;
268 struct v4l2_crop crop;
269 struct v4l2_format format;
270 double hscale, vscale;
271 double aspect;
272 int dwidth, dheight;
273
274 memset (&cropcap, 0, sizeof (cropcap));
275 cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
276
277 if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) {
278 perror ("VIDIOC_CROPCAP");
279 exit (EXIT_FAILURE);
280 }
281
282 memset (&crop, 0, sizeof (crop));
283 crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
284
285 if (-1 == ioctl (fd, VIDIOC_G_CROP, &crop)) {
286 if (errno != EINVAL) {
287 perror ("VIDIOC_G_CROP");
288 exit (EXIT_FAILURE);
289 }
290
291 /* Cropping not supported. */
292 crop.c = cropcap.defrect;
293 }
294
295 memset (&format, 0, sizeof (format));
296 format.fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
297
298 if (-1 == ioctl (fd, VIDIOC_G_FMT, &format)) {
299 perror ("VIDIOC_G_FMT");
300 exit (EXIT_FAILURE);
301 }
302
303 /* The scaling applied by the driver. */
304
305 hscale = format.fmt.pix.width / (double) crop.c.width;
306 vscale = format.fmt.pix.height / (double) crop.c.height;
307
308 aspect = cropcap.pixelaspect.numerator /
309 (double) cropcap.pixelaspect.denominator;
310 aspect = aspect * hscale / vscale;
311
312 /* Devices following ITU-R BT.601 do not capture
313 square pixels. For playback on a computer monitor
314 we should scale the images to this size. */
315
316 dwidth = format.fmt.pix.width / aspect;
317 dheight = format.fmt.pix.height;
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.