1 .. SPDX-License-Identifier: GFDL-1.1-no-invari 1 .. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
2 2
3 .. _v4l2-meta-fmt-d4xx: 3 .. _v4l2-meta-fmt-d4xx:
4 4
5 ******************************* 5 *******************************
6 V4L2_META_FMT_D4XX ('D4XX') 6 V4L2_META_FMT_D4XX ('D4XX')
7 ******************************* 7 *******************************
8 8
9 Intel D4xx UVC Cameras Metadata 9 Intel D4xx UVC Cameras Metadata
10 10
11 11
12 Description 12 Description
13 =========== 13 ===========
14 14
15 Intel D4xx (D435, D455 and others) cameras inc 15 Intel D4xx (D435, D455 and others) cameras include per-frame metadata in their UVC
16 payload headers, following the Microsoft(R) UV 16 payload headers, following the Microsoft(R) UVC extension proposal [1_]. That
17 means, that the private D4XX metadata, followi 17 means, that the private D4XX metadata, following the standard UVC header, is
18 organised in blocks. D4XX cameras implement se 18 organised in blocks. D4XX cameras implement several standard block types,
19 proposed by Microsoft, and several proprietary 19 proposed by Microsoft, and several proprietary ones. Supported standard metadata
20 types are MetadataId_CaptureStats (ID 3), Meta 20 types are MetadataId_CaptureStats (ID 3), MetadataId_CameraExtrinsics (ID 4),
21 and MetadataId_CameraIntrinsics (ID 5). For th 21 and MetadataId_CameraIntrinsics (ID 5). For their description see [1_]. This
22 document describes proprietary metadata types, 22 document describes proprietary metadata types, used by D4xx cameras.
23 23
24 V4L2_META_FMT_D4XX buffers follow the metadata 24 V4L2_META_FMT_D4XX buffers follow the metadata buffer layout of
25 V4L2_META_FMT_UVC with the only difference, th 25 V4L2_META_FMT_UVC with the only difference, that it also includes proprietary
26 payload header data. D4xx cameras use bulk tra 26 payload header data. D4xx cameras use bulk transfers and only send one payload
27 per frame, therefore their headers cannot be l 27 per frame, therefore their headers cannot be larger than 255 bytes.
28 28
29 This document implements Intel Configuration v 29 This document implements Intel Configuration version 3 [9_].
30 30
31 Below are proprietary Microsoft style metadata 31 Below are proprietary Microsoft style metadata types, used by D4xx cameras,
32 where all fields are in little endian order: 32 where all fields are in little endian order:
33 33
34 .. tabularcolumns:: |p{5.0cm}|p{12.5cm}| 34 .. tabularcolumns:: |p{5.0cm}|p{12.5cm}|
35 35
36 36
37 .. flat-table:: D4xx metadata 37 .. flat-table:: D4xx metadata
38 :widths: 1 2 38 :widths: 1 2
39 :header-rows: 1 39 :header-rows: 1
40 :stub-columns: 0 40 :stub-columns: 0
41 41
42 * - **Field** 42 * - **Field**
43 - **Description** 43 - **Description**
44 * - :cspan:`1` *Depth Control* 44 * - :cspan:`1` *Depth Control*
45 * - __u32 ID 45 * - __u32 ID
46 - 0x80000000 46 - 0x80000000
47 * - __u32 Size 47 * - __u32 Size
48 - Size in bytes, include ID (all protoco 48 - Size in bytes, include ID (all protocol versions: 60)
49 * - __u32 Version 49 * - __u32 Version
50 - Version of this structure. The documen 50 - Version of this structure. The documentation herein covers versions 1,
51 2 and 3. The version number will be in 51 2 and 3. The version number will be incremented when new fields are
52 added. 52 added.
53 * - __u32 Flags 53 * - __u32 Flags
54 - A bitmask of flags: see [2_] below 54 - A bitmask of flags: see [2_] below
55 * - __u32 Gain 55 * - __u32 Gain
56 - Gain value in internal units, same as 56 - Gain value in internal units, same as the V4L2_CID_GAIN control, used to
57 capture the frame 57 capture the frame
58 * - __u32 Exposure 58 * - __u32 Exposure
59 - Exposure time (in microseconds) used t 59 - Exposure time (in microseconds) used to capture the frame
60 * - __u32 Laser power 60 * - __u32 Laser power
61 - Power of the laser LED 0-360, used for 61 - Power of the laser LED 0-360, used for depth measurement
62 * - __u32 AE mode 62 * - __u32 AE mode
63 - 0: manual; 1: automatic exposure 63 - 0: manual; 1: automatic exposure
64 * - __u32 Exposure priority 64 * - __u32 Exposure priority
65 - Exposure priority value: 0 - constant 65 - Exposure priority value: 0 - constant frame rate
66 * - __u32 AE ROI left 66 * - __u32 AE ROI left
67 - Left border of the AE Region of Intere 67 - Left border of the AE Region of Interest (all ROI values are in pixels
68 and lie between 0 and maximum width or 68 and lie between 0 and maximum width or height respectively)
69 * - __u32 AE ROI right 69 * - __u32 AE ROI right
70 - Right border of the AE Region of Inter 70 - Right border of the AE Region of Interest
71 * - __u32 AE ROI top 71 * - __u32 AE ROI top
72 - Top border of the AE Region of Interes 72 - Top border of the AE Region of Interest
73 * - __u32 AE ROI bottom 73 * - __u32 AE ROI bottom
74 - Bottom border of the AE Region of Inte 74 - Bottom border of the AE Region of Interest
75 * - __u32 Preset 75 * - __u32 Preset
76 - Preset selector value, default: 0, unl 76 - Preset selector value, default: 0, unless changed by the user
77 * - __u8 Emitter mode (v3 only) (__u32 Las 77 * - __u8 Emitter mode (v3 only) (__u32 Laser mode for v1) [8_]
78 - 0: off, 1: on, same as __u32 Laser mod 78 - 0: off, 1: on, same as __u32 Laser mode for v1
79 * - __u8 RFU byte (v3 only) 79 * - __u8 RFU byte (v3 only)
80 - Spare byte for future use 80 - Spare byte for future use
81 * - __u16 LED Power (v3 only) 81 * - __u16 LED Power (v3 only)
82 - Led power value 0-360 (F416 SKU) 82 - Led power value 0-360 (F416 SKU)
83 * - :cspan:`1` *Capture Timing* 83 * - :cspan:`1` *Capture Timing*
84 * - __u32 ID 84 * - __u32 ID
85 - 0x80000001 85 - 0x80000001
86 * - __u32 Size 86 * - __u32 Size
87 - Size in bytes, include ID (all protoco 87 - Size in bytes, include ID (all protocol versions: 40)
88 * - __u32 Version 88 * - __u32 Version
89 - Version of this structure. The documen 89 - Version of this structure. The documentation herein corresponds to
90 version xxx. The version number will b 90 version xxx. The version number will be incremented when new fields are
91 added. 91 added.
92 * - __u32 Flags 92 * - __u32 Flags
93 - A bitmask of flags: see [3_] below 93 - A bitmask of flags: see [3_] below
94 * - __u32 Frame counter 94 * - __u32 Frame counter
95 - Monotonically increasing counter 95 - Monotonically increasing counter
96 * - __u32 Optical time 96 * - __u32 Optical time
97 - Time in microseconds from the beginnin 97 - Time in microseconds from the beginning of a frame till its middle
98 * - __u32 Readout time 98 * - __u32 Readout time
99 - Time, used to read out a frame in micr 99 - Time, used to read out a frame in microseconds
100 * - __u32 Exposure time 100 * - __u32 Exposure time
101 - Frame exposure time in microseconds 101 - Frame exposure time in microseconds
102 * - __u32 Frame interval 102 * - __u32 Frame interval
103 - In microseconds = 1000000 / framerate 103 - In microseconds = 1000000 / framerate
104 * - __u32 Pipe latency 104 * - __u32 Pipe latency
105 - Time in microseconds from start of fra 105 - Time in microseconds from start of frame to data in USB buffer
106 * - :cspan:`1` *Configuration* 106 * - :cspan:`1` *Configuration*
107 * - __u32 ID 107 * - __u32 ID
108 - 0x80000002 108 - 0x80000002
109 * - __u32 Size 109 * - __u32 Size
110 - Size in bytes, include ID (v1:36, v3:4 110 - Size in bytes, include ID (v1:36, v3:40)
111 * - __u32 Version 111 * - __u32 Version
112 - Version of this structure. The documen 112 - Version of this structure. The documentation herein corresponds to
113 version xxx. The version number will b 113 version xxx. The version number will be incremented when new fields are
114 added. 114 added.
115 * - __u32 Flags 115 * - __u32 Flags
116 - A bitmask of flags: see [4_] below 116 - A bitmask of flags: see [4_] below
117 * - __u8 Hardware type 117 * - __u8 Hardware type
118 - Camera hardware version [5_] 118 - Camera hardware version [5_]
119 * - __u8 SKU ID 119 * - __u8 SKU ID
120 - Camera hardware configuration [6_] 120 - Camera hardware configuration [6_]
121 * - __u32 Cookie 121 * - __u32 Cookie
122 - Internal synchronisation 122 - Internal synchronisation
123 * - __u16 Format 123 * - __u16 Format
124 - Image format code [7_] 124 - Image format code [7_]
125 * - __u16 Width 125 * - __u16 Width
126 - Width in pixels 126 - Width in pixels
127 * - __u16 Height 127 * - __u16 Height
128 - Height in pixels 128 - Height in pixels
129 * - __u16 Framerate 129 * - __u16 Framerate
130 - Requested frame rate per second 130 - Requested frame rate per second
131 * - __u16 Trigger 131 * - __u16 Trigger
132 - Byte 0: bit 0: depth and RGB are synch 132 - Byte 0: bit 0: depth and RGB are synchronised, bit 1: external trigger
133 * - __u16 Calibration count (v3 only) 133 * - __u16 Calibration count (v3 only)
134 - Calibration counter, see [4_] below 134 - Calibration counter, see [4_] below
135 * - __u8 GPIO input data (v3 only) 135 * - __u8 GPIO input data (v3 only)
136 - GPIO readout, see [4_] below (Supporte 136 - GPIO readout, see [4_] below (Supported from FW 5.12.7.0)
137 * - __u32 Sub-preset info (v3 only) 137 * - __u32 Sub-preset info (v3 only)
138 - Sub-preset choice information, see [4_ 138 - Sub-preset choice information, see [4_] below
139 * - __u8 reserved (v3 only) 139 * - __u8 reserved (v3 only)
140 - RFU byte. 140 - RFU byte.
141 141
142 .. _1: 142 .. _1:
143 143
144 [1] https://docs.microsoft.com/en-us/windows-h 144 [1] https://docs.microsoft.com/en-us/windows-hardware/drivers/stream/uvc-extensions-1-5
145 145
146 .. _2: 146 .. _2:
147 147
148 [2] Depth Control flags specify which fields a 148 [2] Depth Control flags specify which fields are valid: ::
149 149
150 0x00000001 Gain 150 0x00000001 Gain
151 0x00000002 Exposure 151 0x00000002 Exposure
152 0x00000004 Laser power 152 0x00000004 Laser power
153 0x00000008 AE mode 153 0x00000008 AE mode
154 0x00000010 Exposure priority 154 0x00000010 Exposure priority
155 0x00000020 AE ROI 155 0x00000020 AE ROI
156 0x00000040 Preset 156 0x00000040 Preset
157 0x00000080 Emitter mode 157 0x00000080 Emitter mode
158 0x00000100 LED Power 158 0x00000100 LED Power
159 159
160 .. _3: 160 .. _3:
161 161
162 [3] Capture Timing flags specify which fields 162 [3] Capture Timing flags specify which fields are valid: ::
163 163
164 0x00000001 Frame counter 164 0x00000001 Frame counter
165 0x00000002 Optical time 165 0x00000002 Optical time
166 0x00000004 Readout time 166 0x00000004 Readout time
167 0x00000008 Exposure time 167 0x00000008 Exposure time
168 0x00000010 Frame interval 168 0x00000010 Frame interval
169 0x00000020 Pipe latency 169 0x00000020 Pipe latency
170 170
171 .. _4: 171 .. _4:
172 172
173 [4] Configuration flags specify which fields a 173 [4] Configuration flags specify which fields are valid: ::
174 174
175 0x00000001 Hardware type 175 0x00000001 Hardware type
176 0x00000002 SKU ID 176 0x00000002 SKU ID
177 0x00000004 Cookie 177 0x00000004 Cookie
178 0x00000008 Format 178 0x00000008 Format
179 0x00000010 Width 179 0x00000010 Width
180 0x00000020 Height 180 0x00000020 Height
181 0x00000040 Framerate 181 0x00000040 Framerate
182 0x00000080 Trigger 182 0x00000080 Trigger
183 0x00000100 Cal count 183 0x00000100 Cal count
184 0x00000200 GPIO Input Data 184 0x00000200 GPIO Input Data
185 0x00000400 Sub-preset Info 185 0x00000400 Sub-preset Info
186 186
187 .. _5: 187 .. _5:
188 188
189 [5] Camera model: :: 189 [5] Camera model: ::
190 190
191 0 DS5 191 0 DS5
192 1 IVCAM2 192 1 IVCAM2
193 193
194 .. _6: 194 .. _6:
195 195
196 [6] 8-bit camera hardware configuration bitfie 196 [6] 8-bit camera hardware configuration bitfield: ::
197 197
198 [1:0] depthCamera 198 [1:0] depthCamera
199 00: no depth 199 00: no depth
200 01: standard depth 200 01: standard depth
201 10: wide depth 201 10: wide depth
202 11: reserved 202 11: reserved
203 [2] depthIsActive - has a laser projector 203 [2] depthIsActive - has a laser projector
204 [3] RGB presence 204 [3] RGB presence
205 [4] Inertial Measurement Unit (IMU) presen 205 [4] Inertial Measurement Unit (IMU) presence
206 [5] projectorType 206 [5] projectorType
207 0: HPTG 207 0: HPTG
208 1: Princeton 208 1: Princeton
209 [6] 0: a projector, 1: an LED 209 [6] 0: a projector, 1: an LED
210 [7] reserved 210 [7] reserved
211 211
212 .. _7: 212 .. _7:
213 213
214 [7] Image format codes per video streaming int 214 [7] Image format codes per video streaming interface:
215 215
216 Depth: :: 216 Depth: ::
217 217
218 1 Z16 218 1 Z16
219 2 Z 219 2 Z
220 220
221 Left sensor: :: 221 Left sensor: ::
222 222
223 1 Y8 223 1 Y8
224 2 UYVY 224 2 UYVY
225 3 R8L8 225 3 R8L8
226 4 Calibration 226 4 Calibration
227 5 W10 227 5 W10
228 228
229 Fish Eye sensor: :: 229 Fish Eye sensor: ::
230 230
231 1 RAW8 231 1 RAW8
232 232
233 .. _8: 233 .. _8:
234 234
235 [8] The "Laser mode" has been replaced in vers 235 [8] The "Laser mode" has been replaced in version 3 by three different fields.
236 "Laser" has been renamed to "Emitter" as there 236 "Laser" has been renamed to "Emitter" as there are multiple technologies for
237 camera projectors. As we have another field fo 237 camera projectors. As we have another field for "Laser Power" we introduced
238 "LED Power" for extra emitter. 238 "LED Power" for extra emitter.
239 239
240 The "Laser mode" __u32 fields has been split i 240 The "Laser mode" __u32 fields has been split into: ::
241 1 __u8 Emitter mode 241 1 __u8 Emitter mode
242 2 __u8 RFU byte 242 2 __u8 RFU byte
243 3 __u16 LED Power 243 3 __u16 LED Power
244 244
245 This is a change between versions 1 and 3. All 245 This is a change between versions 1 and 3. All versions 1, 2 and 3 are backward
246 compatible with the same data format and they 246 compatible with the same data format and they are supported. See [2_] for which
247 attributes are valid. 247 attributes are valid.
248 248
249 .. _9: 249 .. _9:
250 250
251 [9] LibRealSense SDK metadata source: 251 [9] LibRealSense SDK metadata source:
252 https://github.com/IntelRealSense/librealsense 252 https://github.com/IntelRealSense/librealsense/blob/master/src/metadata.h
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.