1 .. SPDX-License-Identifier: GFDL-1.1-no-invari !! 1 .. Permission is granted to copy, distribute and/or modify this
2 .. c:namespace:: V4L !! 2 .. document under the terms of the GNU Free Documentation License,
>> 3 .. Version 1.1 or any later version published by the Free Software
>> 4 .. Foundation, with no Invariant Sections, no Front-Cover Texts
>> 5 .. and no Back-Cover Texts. A copy of the license is included at
>> 6 .. Documentation/userspace-api/media/fdl-appendix.rst.
>> 7 ..
>> 8 .. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
3 9
4 .. _mmap: 10 .. _mmap:
5 11
6 ****************************** 12 ******************************
7 Streaming I/O (Memory Mapping) 13 Streaming I/O (Memory Mapping)
8 ****************************** 14 ******************************
9 15
10 Input and output devices support this I/O meth 16 Input and output devices support this I/O method when the
11 ``V4L2_CAP_STREAMING`` flag in the ``capabilit 17 ``V4L2_CAP_STREAMING`` flag in the ``capabilities`` field of struct
12 :c:type:`v4l2_capability` returned by the 18 :c:type:`v4l2_capability` returned by the
13 :ref:`VIDIOC_QUERYCAP` ioctl is set. There are 19 :ref:`VIDIOC_QUERYCAP` ioctl is set. There are two
14 streaming methods, to determine if the memory 20 streaming methods, to determine if the memory mapping flavor is
15 supported applications must call the :ref:`VID 21 supported applications must call the :ref:`VIDIOC_REQBUFS` ioctl
16 with the memory type set to ``V4L2_MEMORY_MMAP 22 with the memory type set to ``V4L2_MEMORY_MMAP``.
17 23
18 Streaming is an I/O method where only pointers 24 Streaming is an I/O method where only pointers to buffers are exchanged
19 between application and driver, the data itsel 25 between application and driver, the data itself is not copied. Memory
20 mapping is primarily intended to map buffers i 26 mapping is primarily intended to map buffers in device memory into the
21 application's address space. Device memory can 27 application's address space. Device memory can be for example the video
22 memory on a graphics card with a video capture 28 memory on a graphics card with a video capture add-on. However, being
23 the most efficient I/O method available for a 29 the most efficient I/O method available for a long time, many other
24 drivers support streaming as well, allocating 30 drivers support streaming as well, allocating buffers in DMA-able main
25 memory. 31 memory.
26 32
27 A driver can support many sets of buffers. Eac 33 A driver can support many sets of buffers. Each set is identified by a
28 unique buffer type value. The sets are indepen 34 unique buffer type value. The sets are independent and each set can hold
29 a different type of data. To access different 35 a different type of data. To access different sets at the same time
30 different file descriptors must be used. [#f1] 36 different file descriptors must be used. [#f1]_
31 37
32 To allocate device buffers applications call t 38 To allocate device buffers applications call the
33 :ref:`VIDIOC_REQBUFS` ioctl with the desired n 39 :ref:`VIDIOC_REQBUFS` ioctl with the desired number
34 of buffers and buffer type, for example ``V4L2 40 of buffers and buffer type, for example ``V4L2_BUF_TYPE_VIDEO_CAPTURE``.
35 This ioctl can also be used to change the numb 41 This ioctl can also be used to change the number of buffers or to free
36 the allocated memory, provided none of the buf 42 the allocated memory, provided none of the buffers are still mapped.
37 43
38 Before applications can access the buffers the 44 Before applications can access the buffers they must map them into their
39 address space with the :c:func:`mmap()` functi !! 45 address space with the :ref:`mmap() <func-mmap>` function. The
40 location of the buffers in device memory can b 46 location of the buffers in device memory can be determined with the
41 :ref:`VIDIOC_QUERYBUF` ioctl. In the single-pl 47 :ref:`VIDIOC_QUERYBUF` ioctl. In the single-planar
42 API case, the ``m.offset`` and ``length`` retu 48 API case, the ``m.offset`` and ``length`` returned in a struct
43 :c:type:`v4l2_buffer` are passed as sixth and 49 :c:type:`v4l2_buffer` are passed as sixth and second
44 parameter to the :c:func:`mmap()` function. Wh !! 50 parameter to the :ref:`mmap() <func-mmap>` function. When using the
45 multi-planar API, struct :c:type:`v4l2_buffer` 51 multi-planar API, struct :c:type:`v4l2_buffer` contains an
46 array of struct :c:type:`v4l2_plane` structure 52 array of struct :c:type:`v4l2_plane` structures, each
47 containing its own ``m.offset`` and ``length`` 53 containing its own ``m.offset`` and ``length``. When using the
48 multi-planar API, every plane of every buffer 54 multi-planar API, every plane of every buffer has to be mapped
49 separately, so the number of calls to :c:func: !! 55 separately, so the number of calls to :ref:`mmap() <func-mmap>` should
50 be equal to number of buffers times number of 56 be equal to number of buffers times number of planes in each buffer. The
51 offset and length values must not be modified. 57 offset and length values must not be modified. Remember, the buffers are
52 allocated in physical memory, as opposed to vi 58 allocated in physical memory, as opposed to virtual memory, which can be
53 swapped out to disk. Applications should free 59 swapped out to disk. Applications should free the buffers as soon as
54 possible with the :c:func:`munmap()` function. !! 60 possible with the :ref:`munmap() <func-munmap>` function.
55 61
56 Example: Mapping buffers in the single-planar 62 Example: Mapping buffers in the single-planar API
57 ============================================== 63 =================================================
58 64
59 .. code-block:: c 65 .. code-block:: c
60 66
61 struct v4l2_requestbuffers reqbuf; 67 struct v4l2_requestbuffers reqbuf;
62 struct { 68 struct {
63 void *start; 69 void *start;
64 size_t length; 70 size_t length;
65 } *buffers; 71 } *buffers;
66 unsigned int i; 72 unsigned int i;
67 73
68 memset(&reqbuf, 0, sizeof(reqbuf)); 74 memset(&reqbuf, 0, sizeof(reqbuf));
69 reqbuf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; 75 reqbuf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
70 reqbuf.memory = V4L2_MEMORY_MMAP; 76 reqbuf.memory = V4L2_MEMORY_MMAP;
71 reqbuf.count = 20; 77 reqbuf.count = 20;
72 78
73 if (-1 == ioctl (fd, VIDIOC_REQBUFS, &reqb 79 if (-1 == ioctl (fd, VIDIOC_REQBUFS, &reqbuf)) {
74 if (errno == EINVAL) 80 if (errno == EINVAL)
75 printf("Video capturing or mmap-st 81 printf("Video capturing or mmap-streaming is not supported\\n");
76 else 82 else
77 perror("VIDIOC_REQBUFS"); 83 perror("VIDIOC_REQBUFS");
78 84
79 exit(EXIT_FAILURE); 85 exit(EXIT_FAILURE);
80 } 86 }
81 87
82 /* We want at least five buffers. */ 88 /* We want at least five buffers. */
83 89
84 if (reqbuf.count < 5) { 90 if (reqbuf.count < 5) {
85 /* You may need to free the buffers he 91 /* You may need to free the buffers here. */
86 printf("Not enough buffer memory\\n"); 92 printf("Not enough buffer memory\\n");
87 exit(EXIT_FAILURE); 93 exit(EXIT_FAILURE);
88 } 94 }
89 95
90 buffers = calloc(reqbuf.count, sizeof(*buf 96 buffers = calloc(reqbuf.count, sizeof(*buffers));
91 assert(buffers != NULL); 97 assert(buffers != NULL);
92 98
93 for (i = 0; i < reqbuf.count; i++) { 99 for (i = 0; i < reqbuf.count; i++) {
94 struct v4l2_buffer buffer; 100 struct v4l2_buffer buffer;
95 101
96 memset(&buffer, 0, sizeof(buffer)); 102 memset(&buffer, 0, sizeof(buffer));
97 buffer.type = reqbuf.type; 103 buffer.type = reqbuf.type;
98 buffer.memory = V4L2_MEMORY_MMAP; 104 buffer.memory = V4L2_MEMORY_MMAP;
99 buffer.index = i; 105 buffer.index = i;
100 106
101 if (-1 == ioctl (fd, VIDIOC_QUERYBUF, 107 if (-1 == ioctl (fd, VIDIOC_QUERYBUF, &buffer)) {
102 perror("VIDIOC_QUERYBUF"); 108 perror("VIDIOC_QUERYBUF");
103 exit(EXIT_FAILURE); 109 exit(EXIT_FAILURE);
104 } 110 }
105 111
106 buffers[i].length = buffer.length; /* 112 buffers[i].length = buffer.length; /* remember for munmap() */
107 113
108 buffers[i].start = mmap(NULL, buffer.l 114 buffers[i].start = mmap(NULL, buffer.length,
109 PROT_READ | PROT_WRITE, /* 115 PROT_READ | PROT_WRITE, /* recommended */
110 MAP_SHARED, /* 116 MAP_SHARED, /* recommended */
111 fd, buffer.m.offset); 117 fd, buffer.m.offset);
112 118
113 if (MAP_FAILED == buffers[i].start) { 119 if (MAP_FAILED == buffers[i].start) {
114 /* If you do not exit here you sho 120 /* If you do not exit here you should unmap() and free()
115 the buffers mapped so far. */ 121 the buffers mapped so far. */
116 perror("mmap"); 122 perror("mmap");
117 exit(EXIT_FAILURE); 123 exit(EXIT_FAILURE);
118 } 124 }
119 } 125 }
120 126
121 /* Cleanup. */ 127 /* Cleanup. */
122 128
123 for (i = 0; i < reqbuf.count; i++) 129 for (i = 0; i < reqbuf.count; i++)
124 munmap(buffers[i].start, buffers[i].le 130 munmap(buffers[i].start, buffers[i].length);
125 131
>> 132
126 Example: Mapping buffers in the multi-planar A 133 Example: Mapping buffers in the multi-planar API
127 ============================================== 134 ================================================
128 135
129 .. code-block:: c 136 .. code-block:: c
130 137
131 struct v4l2_requestbuffers reqbuf; 138 struct v4l2_requestbuffers reqbuf;
132 /* Our current format uses 3 planes per bu 139 /* Our current format uses 3 planes per buffer */
133 #define FMT_NUM_PLANES = 3 140 #define FMT_NUM_PLANES = 3
134 141
135 struct { 142 struct {
136 void *start[FMT_NUM_PLANES]; 143 void *start[FMT_NUM_PLANES];
137 size_t length[FMT_NUM_PLANES]; 144 size_t length[FMT_NUM_PLANES];
138 } *buffers; 145 } *buffers;
139 unsigned int i, j; 146 unsigned int i, j;
140 147
141 memset(&reqbuf, 0, sizeof(reqbuf)); 148 memset(&reqbuf, 0, sizeof(reqbuf));
142 reqbuf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_ 149 reqbuf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
143 reqbuf.memory = V4L2_MEMORY_MMAP; 150 reqbuf.memory = V4L2_MEMORY_MMAP;
144 reqbuf.count = 20; 151 reqbuf.count = 20;
145 152
146 if (ioctl(fd, VIDIOC_REQBUFS, &reqbuf) < 0 153 if (ioctl(fd, VIDIOC_REQBUFS, &reqbuf) < 0) {
147 if (errno == EINVAL) 154 if (errno == EINVAL)
148 printf("Video capturing or mmap-st 155 printf("Video capturing or mmap-streaming is not supported\\n");
149 else 156 else
150 perror("VIDIOC_REQBUFS"); 157 perror("VIDIOC_REQBUFS");
151 158
152 exit(EXIT_FAILURE); 159 exit(EXIT_FAILURE);
153 } 160 }
154 161
155 /* We want at least five buffers. */ 162 /* We want at least five buffers. */
156 163
157 if (reqbuf.count < 5) { 164 if (reqbuf.count < 5) {
158 /* You may need to free the buffers he 165 /* You may need to free the buffers here. */
159 printf("Not enough buffer memory\\n"); 166 printf("Not enough buffer memory\\n");
160 exit(EXIT_FAILURE); 167 exit(EXIT_FAILURE);
161 } 168 }
162 169
163 buffers = calloc(reqbuf.count, sizeof(*buf 170 buffers = calloc(reqbuf.count, sizeof(*buffers));
164 assert(buffers != NULL); 171 assert(buffers != NULL);
165 172
166 for (i = 0; i < reqbuf.count; i++) { 173 for (i = 0; i < reqbuf.count; i++) {
167 struct v4l2_buffer buffer; 174 struct v4l2_buffer buffer;
168 struct v4l2_plane planes[FMT_NUM_PLANE 175 struct v4l2_plane planes[FMT_NUM_PLANES];
169 176
170 memset(&buffer, 0, sizeof(buffer)); 177 memset(&buffer, 0, sizeof(buffer));
171 buffer.type = reqbuf.type; 178 buffer.type = reqbuf.type;
172 buffer.memory = V4L2_MEMORY_MMAP; 179 buffer.memory = V4L2_MEMORY_MMAP;
173 buffer.index = i; 180 buffer.index = i;
174 /* length in struct v4l2_buffer in mul 181 /* length in struct v4l2_buffer in multi-planar API stores the size
175 * of planes array. */ 182 * of planes array. */
176 buffer.length = FMT_NUM_PLANES; 183 buffer.length = FMT_NUM_PLANES;
177 buffer.m.planes = planes; 184 buffer.m.planes = planes;
178 185
179 if (ioctl(fd, VIDIOC_QUERYBUF, &buffer 186 if (ioctl(fd, VIDIOC_QUERYBUF, &buffer) < 0) {
180 perror("VIDIOC_QUERYBUF"); 187 perror("VIDIOC_QUERYBUF");
181 exit(EXIT_FAILURE); 188 exit(EXIT_FAILURE);
182 } 189 }
183 190
184 /* Every plane has to be mapped separa 191 /* Every plane has to be mapped separately */
185 for (j = 0; j < FMT_NUM_PLANES; j++) { 192 for (j = 0; j < FMT_NUM_PLANES; j++) {
186 buffers[i].length[j] = buffer.m.pl 193 buffers[i].length[j] = buffer.m.planes[j].length; /* remember for munmap() */
187 194
188 buffers[i].start[j] = mmap(NULL, b 195 buffers[i].start[j] = mmap(NULL, buffer.m.planes[j].length,
189 PROT_READ | PROT_WRITE, / 196 PROT_READ | PROT_WRITE, /* recommended */
190 MAP_SHARED, / 197 MAP_SHARED, /* recommended */
191 fd, buffer.m.planes[j].m. !! 198 fd, buffer.m.planes[j].m.offset);
192 199
193 if (MAP_FAILED == buffers[i].start 200 if (MAP_FAILED == buffers[i].start[j]) {
194 /* If you do not exit here you 201 /* If you do not exit here you should unmap() and free()
195 the buffers and planes mapp 202 the buffers and planes mapped so far. */
196 perror("mmap"); 203 perror("mmap");
197 exit(EXIT_FAILURE); 204 exit(EXIT_FAILURE);
198 } 205 }
199 } 206 }
200 } 207 }
201 208
202 /* Cleanup. */ 209 /* Cleanup. */
203 210
204 for (i = 0; i < reqbuf.count; i++) 211 for (i = 0; i < reqbuf.count; i++)
205 for (j = 0; j < FMT_NUM_PLANES; j++) 212 for (j = 0; j < FMT_NUM_PLANES; j++)
206 munmap(buffers[i].start[j], buffer 213 munmap(buffers[i].start[j], buffers[i].length[j]);
207 214
208 Conceptually streaming drivers maintain two bu 215 Conceptually streaming drivers maintain two buffer queues, an incoming
209 and an outgoing queue. They separate the synch 216 and an outgoing queue. They separate the synchronous capture or output
210 operation locked to a video clock from the app 217 operation locked to a video clock from the application which is subject
211 to random disk or network delays and preemptio 218 to random disk or network delays and preemption by other processes,
212 thereby reducing the probability of data loss. 219 thereby reducing the probability of data loss. The queues are organized
213 as FIFOs, buffers will be output in the order 220 as FIFOs, buffers will be output in the order enqueued in the incoming
214 FIFO, and were captured in the order dequeued 221 FIFO, and were captured in the order dequeued from the outgoing FIFO.
215 222
216 The driver may require a minimum number of buf 223 The driver may require a minimum number of buffers enqueued at all times
217 to function, apart of this no limit exists on 224 to function, apart of this no limit exists on the number of buffers
218 applications can enqueue in advance, or dequeu 225 applications can enqueue in advance, or dequeue and process. They can
219 also enqueue in a different order than buffers 226 also enqueue in a different order than buffers have been dequeued, and
220 the driver can *fill* enqueued *empty* buffers 227 the driver can *fill* enqueued *empty* buffers in any order. [#f2]_ The
221 index number of a buffer (struct :c:type:`v4l2 228 index number of a buffer (struct :c:type:`v4l2_buffer`
222 ``index``) plays no role here, it only identif 229 ``index``) plays no role here, it only identifies the buffer.
223 230
224 Initially all mapped buffers are in dequeued s 231 Initially all mapped buffers are in dequeued state, inaccessible by the
225 driver. For capturing applications it is custo 232 driver. For capturing applications it is customary to first enqueue all
226 mapped buffers, then to start capturing and en 233 mapped buffers, then to start capturing and enter the read loop. Here
227 the application waits until a filled buffer ca 234 the application waits until a filled buffer can be dequeued, and
228 re-enqueues the buffer when the data is no lon 235 re-enqueues the buffer when the data is no longer needed. Output
229 applications fill and enqueue buffers, when en 236 applications fill and enqueue buffers, when enough buffers are stacked
230 up the output is started with :ref:`VIDIOC_STR 237 up the output is started with :ref:`VIDIOC_STREAMON <VIDIOC_STREAMON>`.
231 In the write loop, when the application runs o 238 In the write loop, when the application runs out of free buffers, it
232 must wait until an empty buffer can be dequeue 239 must wait until an empty buffer can be dequeued and reused.
233 240
234 To enqueue and dequeue a buffer applications u 241 To enqueue and dequeue a buffer applications use the
235 :ref:`VIDIOC_QBUF <VIDIOC_QBUF>` and :ref:`VID !! 242 :ref:`VIVIOC_QBUF <VIDIOC_QBUF>` and :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>`
236 ioctl. The status of a buffer being mapped, en 243 ioctl. The status of a buffer being mapped, enqueued, full or empty can
237 be determined at any time using the :ref:`VIDI 244 be determined at any time using the :ref:`VIDIOC_QUERYBUF` ioctl. Two
238 methods exist to suspend execution of the appl 245 methods exist to suspend execution of the application until one or more
239 buffers can be dequeued. By default :ref:`VID 246 buffers can be dequeued. By default :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>`
240 blocks when no buffer is in the outgoing queue 247 blocks when no buffer is in the outgoing queue. When the ``O_NONBLOCK``
241 flag was given to the :c:func:`open()` functio !! 248 flag was given to the :ref:`open() <func-open>` function,
242 :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>` returns imme 249 :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>` returns immediately with an ``EAGAIN``
243 error code when no buffer is available. The :c !! 250 error code when no buffer is available. The :ref:`select() <func-select>`
244 or :c:func:`poll()` functions are always avail !! 251 or :ref:`poll() <func-poll>` functions are always available.
245 252
246 To start and stop capturing or output applicat 253 To start and stop capturing or output applications call the
247 :ref:`VIDIOC_STREAMON <VIDIOC_STREAMON>` and : 254 :ref:`VIDIOC_STREAMON <VIDIOC_STREAMON>` and :ref:`VIDIOC_STREAMOFF
248 <VIDIOC_STREAMON>` ioctl. 255 <VIDIOC_STREAMON>` ioctl.
249 256
250 .. note:::ref:`VIDIOC_STREAMOFF <VIDIOC_STREAM 257 .. note:::ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>`
251 removes all buffers from both queues as a s 258 removes all buffers from both queues as a side effect. Since there is
252 no notion of doing anything "now" on a mult 259 no notion of doing anything "now" on a multitasking system, if an
253 application needs to synchronize with anoth 260 application needs to synchronize with another event it should examine
254 the struct ::c:type:`v4l2_buffer` ``timesta 261 the struct ::c:type:`v4l2_buffer` ``timestamp`` of captured
255 or outputted buffers. 262 or outputted buffers.
256 263
257 Drivers implementing memory mapping I/O must s 264 Drivers implementing memory mapping I/O must support the
258 :ref:`VIDIOC_REQBUFS <VIDIOC_REQBUFS>`, :ref:` 265 :ref:`VIDIOC_REQBUFS <VIDIOC_REQBUFS>`, :ref:`VIDIOC_QUERYBUF
259 <VIDIOC_QUERYBUF>`, :ref:`VIDIOC_QBUF <VIDIOC_ 266 <VIDIOC_QUERYBUF>`, :ref:`VIDIOC_QBUF <VIDIOC_QBUF>`, :ref:`VIDIOC_DQBUF
260 <VIDIOC_QBUF>`, :ref:`VIDIOC_STREAMON <VIDIOC_ 267 <VIDIOC_QBUF>`, :ref:`VIDIOC_STREAMON <VIDIOC_STREAMON>`
261 and :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` 268 and :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` ioctls, the :ref:`mmap()
262 <func-mmap>`, :c:func:`munmap()`, :ref:`select !! 269 <func-mmap>`, :ref:`munmap() <func-munmap>`, :ref:`select()
263 <func-select>` and :c:func:`poll()` function. !! 270 <func-select>` and :ref:`poll() <func-poll>` function. [#f3]_
264 271
265 [capture example] 272 [capture example]
266 273
267 .. [#f1] 274 .. [#f1]
268 One could use one file descriptor and set t 275 One could use one file descriptor and set the buffer type field
269 accordingly when calling :ref:`VIDIOC_QBUF` 276 accordingly when calling :ref:`VIDIOC_QBUF` etc.,
270 but it makes the :c:func:`select()` functio !! 277 but it makes the :ref:`select() <func-select>` function ambiguous. We also
271 like the clean approach of one file descrip 278 like the clean approach of one file descriptor per logical stream.
272 Video overlay for example is also a logical 279 Video overlay for example is also a logical stream, although the CPU
273 is not needed for continuous operation. 280 is not needed for continuous operation.
274 281
275 .. [#f2] 282 .. [#f2]
276 Random enqueue order permits applications p 283 Random enqueue order permits applications processing images out of
277 order (such as video codecs) to return buff 284 order (such as video codecs) to return buffers earlier, reducing the
278 probability of data loss. Random fill order 285 probability of data loss. Random fill order allows drivers to reuse
279 buffers on a LIFO-basis, taking advantage o 286 buffers on a LIFO-basis, taking advantage of caches holding
280 scatter-gather lists and the like. 287 scatter-gather lists and the like.
281 288
282 .. [#f3] 289 .. [#f3]
283 At the driver level :c:func:`select()` and !! 290 At the driver level :ref:`select() <func-select>` and :ref:`poll() <func-poll>` are
284 the same, and :c:func:`select()` is too imp !! 291 the same, and :ref:`select() <func-select>` is too important to be optional.
285 The rest should be evident. 292 The rest should be evident.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.