1 .. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
2
3 .. _overlay:
4
5 ***********************
6 Video Overlay Interface
7 ***********************
8
9 **Also known as Framebuffer Overlay or Previewing.**
10
11 Video overlay devices have the ability to genlock (TV-)video into the
12 (VGA-)video signal of a graphics card, or to store captured images
13 directly in video memory of a graphics card, typically with clipping.
14 This can be considerable more efficient than capturing images and
15 displaying them by other means. In the old days when only nuclear power
16 plants needed cooling towers this used to be the only way to put live
17 video into a window.
18
19 Video overlay devices are accessed through the same character special
20 files as :ref:`video capture <capture>` devices.
21
22 .. note::
23
24 The default function of a ``/dev/video`` device is video
25 capturing. The overlay function is only available after calling
26 the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl.
27
28 The driver may support simultaneous overlay and capturing using the
29 read/write and streaming I/O methods. If so, operation at the nominal
30 frame rate of the video standard is not guaranteed. Frames may be
31 directed away from overlay to capture, or one field may be used for
32 overlay and the other for capture if the capture parameters permit this.
33
34 Applications should use different file descriptors for capturing and
35 overlay. This must be supported by all drivers capable of simultaneous
36 capturing and overlay. Optionally these drivers may also permit
37 capturing and overlay with a single file descriptor for compatibility
38 with V4L and earlier versions of V4L2. [#f1]_
39
40 A common application of two file descriptors is the X11
41 :ref:`Xv/V4L <xvideo>` interface driver and a V4L2 application.
42 While the X server controls video overlay, the application can take
43 advantage of memory mapping and DMA.
44
45 Querying Capabilities
46 =====================
47
48 Devices supporting the video overlay interface set the
49 ``V4L2_CAP_VIDEO_OVERLAY`` flag in the ``capabilities`` field of struct
50 :c:type:`v4l2_capability` returned by the
51 :ref:`VIDIOC_QUERYCAP` ioctl. The overlay I/O
52 method specified below must be supported. Tuners and audio inputs are
53 optional.
54
55
56 Supplemental Functions
57 ======================
58
59 Video overlay devices shall support :ref:`audio input <audio>`,
60 :ref:`tuner`, :ref:`controls <control>`,
61 :ref:`cropping and scaling <crop>` and
62 :ref:`streaming parameter <streaming-par>` ioctls as needed. The
63 :ref:`video input <video>` and :ref:`video standard <standard>`
64 ioctls must be supported by all video overlay devices.
65
66
67 Setup
68 =====
69
70 *Note: support for this has been removed.*
71 Before overlay can commence applications must program the driver with
72 frame buffer parameters, namely the address and size of the frame buffer
73 and the image format, for example RGB 5:6:5. The
74 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` and
75 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctls are available to get and
76 set these parameters, respectively. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is
77 privileged because it allows to set up DMA into physical memory,
78 bypassing the memory protection mechanisms of the kernel. Only the
79 superuser can change the frame buffer address and size. Users are not
80 supposed to run TV applications as root or with SUID bit set. A small
81 helper application with suitable privileges should query the graphics
82 system and program the V4L2 driver at the appropriate time.
83
84 Some devices add the video overlay to the output signal of the graphics
85 card. In this case the frame buffer is not modified by the video device,
86 and the frame buffer address and pixel format are not needed by the
87 driver. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is not privileged. An application
88 can check for this type of device by calling the :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`
89 ioctl.
90
91 A driver may support any (or none) of five clipping/blending methods:
92
93 1. Chroma-keying displays the overlaid image only where pixels in the
94 primary graphics surface assume a certain color.
95
96 2. *Note: support for this has been removed.*
97 A bitmap can be specified where each bit corresponds to a pixel in
98 the overlaid image. When the bit is set, the corresponding video
99 pixel is displayed, otherwise a pixel of the graphics surface.
100
101 3. *Note: support for this has been removed.*
102 A list of clipping rectangles can be specified. In these regions *no*
103 video is displayed, so the graphics surface can be seen here.
104
105 4. The framebuffer has an alpha channel that can be used to clip or
106 blend the framebuffer with the video.
107
108 5. A global alpha value can be specified to blend the framebuffer
109 contents with video images.
110
111 When simultaneous capturing and overlay is supported and the hardware
112 prohibits different image and frame buffer formats, the format requested
113 first takes precedence. The attempt to capture
114 (:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) or overlay
115 (:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`) may fail with an ``EBUSY`` error
116 code or return accordingly modified parameters..
117
118
119 Overlay Window
120 ==============
121
122 The overlaid image is determined by cropping and overlay window
123 parameters. The former select an area of the video picture to capture,
124 the latter how images are overlaid and clipped. Cropping initialization
125 at minimum requires to reset the parameters to defaults. An example is
126 given in :ref:`crop`.
127
128 The overlay window is described by a struct
129 :c:type:`v4l2_window`. It defines the size of the image,
130 its position over the graphics surface and the clipping to be applied.
131 To get the current parameters applications set the ``type`` field of a
132 struct :c:type:`v4l2_format` to
133 ``V4L2_BUF_TYPE_VIDEO_OVERLAY`` and call the
134 :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl. The driver fills the
135 struct :c:type:`v4l2_window` substructure named ``win``. It is not
136 possible to retrieve a previously programmed clipping list or bitmap.
137
138 To program the overlay window applications set the ``type`` field of a
139 struct :c:type:`v4l2_format` to
140 ``V4L2_BUF_TYPE_VIDEO_OVERLAY``, initialize the ``win`` substructure and
141 call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. The driver
142 adjusts the parameters against hardware limits and returns the actual
143 parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does. Like :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`, the
144 :ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl can be used to learn
145 about driver capabilities without actually changing driver state. Unlike
146 :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` this also works after the overlay has been enabled.
147
148 The scaling factor of the overlaid image is implied by the width and
149 height given in struct :c:type:`v4l2_window` and the size
150 of the cropping rectangle. For more information see :ref:`crop`.
151
152 When simultaneous capturing and overlay is supported and the hardware
153 prohibits different image and window sizes, the size requested first
154 takes precedence. The attempt to capture or overlay as well
155 (:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) may fail with an ``EBUSY`` error
156 code or return accordingly modified parameters.
157
158
159 .. c:type:: v4l2_window
160
161 struct v4l2_window
162 ------------------
163
164 ``struct v4l2_rect w``
165 Size and position of the window relative to the top, left corner of
166 the frame buffer defined with
167 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`. The window can extend the
168 frame buffer width and height, the ``x`` and ``y`` coordinates can
169 be negative, and it can lie completely outside the frame buffer. The
170 driver clips the window accordingly, or if that is not possible,
171 modifies its size and/or position.
172
173 ``enum v4l2_field field``
174 Applications set this field to determine which video field shall be
175 overlaid, typically one of ``V4L2_FIELD_ANY`` (0),
176 ``V4L2_FIELD_TOP``, ``V4L2_FIELD_BOTTOM`` or
177 ``V4L2_FIELD_INTERLACED``. Drivers may have to choose a different
178 field order and return the actual setting here.
179
180 ``__u32 chromakey``
181 When chroma-keying has been negotiated with
182 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` applications set this field
183 to the desired pixel value for the chroma key. The format is the
184 same as the pixel format of the framebuffer (struct
185 :c:type:`v4l2_framebuffer` ``fmt.pixelformat``
186 field), with bytes in host order. E. g. for
187 :ref:`V4L2_PIX_FMT_BGR24 <V4L2-PIX-FMT-BGR32>` the value should
188 be 0xRRGGBB on a little endian, 0xBBGGRR on a big endian host.
189
190 ``struct v4l2_clip * clips``
191 *Note: support for this has been removed.*
192 When chroma-keying has *not* been negotiated and
193 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability,
194 applications can set this field to point to an array of clipping
195 rectangles.
196
197 Like the window coordinates w, clipping rectangles are defined
198 relative to the top, left corner of the frame buffer. However
199 clipping rectangles must not extend the frame buffer width and
200 height, and they must not overlap. If possible applications
201 should merge adjacent rectangles. Whether this must create
202 x-y or y-x bands, or the order of rectangles, is not defined. When
203 clip lists are not supported the driver ignores this field. Its
204 contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`
205 are undefined.
206
207 ``__u32 clipcount``
208 *Note: support for this has been removed.*
209 When the application set the ``clips`` field, this field must
210 contain the number of clipping rectangles in the list. When clip
211 lists are not supported the driver ignores this field, its contents
212 after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are undefined. When clip lists are
213 supported but no clipping is desired this field must be set to zero.
214
215 ``void * bitmap``
216 *Note: support for this has been removed.*
217 When chroma-keying has *not* been negotiated and
218 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability,
219 applications can set this field to point to a clipping bit mask.
220
221 It must be of the same size as the window, ``w.width`` and ``w.height``.
222 Each bit corresponds to a pixel in the overlaid image, which is
223 displayed only when the bit is *set*. Pixel coordinates translate to
224 bits like:
225
226
227 .. code-block:: c
228
229 ((__u8 *) bitmap)[w.width * y + x / 8] & (1 << (x & 7))
230
231 where ``0`` ≤ x < ``w.width`` and ``0`` ≤ y <``w.height``. [#f2]_
232
233 When a clipping bit mask is not supported the driver ignores this field,
234 its contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are
235 undefined. When a bit mask is supported but no clipping is desired this
236 field must be set to ``NULL``.
237
238 Applications need not create a clip list or bit mask. When they pass
239 both, or despite negotiating chroma-keying, the results are undefined.
240 Regardless of the chosen method, the clipping abilities of the hardware
241 may be limited in quantity or quality. The results when these limits are
242 exceeded are undefined. [#f3]_
243
244 ``__u8 global_alpha``
245 The global alpha value used to blend the framebuffer with video
246 images, if global alpha blending has been negotiated
247 (``V4L2_FBUF_FLAG_GLOBAL_ALPHA``, see
248 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`,
249 :ref:`framebuffer-flags`).
250
251 .. note::
252
253 This field was added in Linux 2.6.23, extending the
254 structure. However the :ref:`VIDIOC_[G|S|TRY]_FMT <VIDIOC_G_FMT>`
255 ioctls, which take a pointer to a :c:type:`v4l2_format`
256 parent structure with padding bytes at the end, are not affected.
257
258
259 .. c:type:: v4l2_clip
260
261 struct v4l2_clip [#f4]_
262 -----------------------
263
264 ``struct v4l2_rect c``
265 Coordinates of the clipping rectangle, relative to the top, left
266 corner of the frame buffer. Only window pixels *outside* all
267 clipping rectangles are displayed.
268
269 ``struct v4l2_clip * next``
270 Pointer to the next clipping rectangle, ``NULL`` when this is the last
271 rectangle. Drivers ignore this field, it cannot be used to pass a
272 linked list of clipping rectangles.
273
274
275 .. c:type:: v4l2_rect
276
277 struct v4l2_rect
278 ----------------
279
280 ``__s32 left``
281 Horizontal offset of the top, left corner of the rectangle, in
282 pixels.
283
284 ``__s32 top``
285 Vertical offset of the top, left corner of the rectangle, in pixels.
286 Offsets increase to the right and down.
287
288 ``__u32 width``
289 Width of the rectangle, in pixels.
290
291 ``__u32 height``
292 Height of the rectangle, in pixels.
293
294
295 Enabling Overlay
296 ================
297
298 To start or stop the frame buffer overlay applications call the
299 :ref:`VIDIOC_OVERLAY` ioctl.
300
301 .. [#f1]
302 In the opinion of the designers of this API, no driver writer taking
303 the efforts to support simultaneous capturing and overlay will
304 restrict this ability by requiring a single file descriptor, as in
305 V4L and earlier versions of V4L2. Making this optional means
306 applications depending on two file descriptors need backup routines
307 to be compatible with all drivers, which is considerable more work
308 than using two fds in applications which do not. Also two fd's fit
309 the general concept of one file descriptor for each logical stream.
310 Hence as a complexity trade-off drivers *must* support two file
311 descriptors and *may* support single fd operation.
312
313 .. [#f2]
314 Should we require ``w.width`` to be a multiple of eight?
315
316 .. [#f3]
317 When the image is written into frame buffer memory it will be
318 undesirable if the driver clips out less pixels than expected,
319 because the application and graphics system are not aware these
320 regions need to be refreshed. The driver should clip out more pixels
321 or not write the image at all.
322
323 .. [#f4]
324 The X Window system defines "regions" which are vectors of ``struct
325 BoxRec { short x1, y1, x2, y2; }`` with ``width = x2 - x1`` and
326 ``height = y2 - y1``, so one cannot pass X11 clip lists directly.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.