1 =================================
2 (How to avoid) Botching up ioctls
3 =================================
4
5 From: https://blog.ffwll.ch/2013/11/botching-u
6
7 By: Daniel Vetter, Copyright © 2013 Intel Cor
8
9 One clear insight kernel graphics hackers gain
10 trying to come up with a unified interface to
11 memory on completely different GPUs is a futil
12 driver has its own set of ioctls to allocate m
13 Which is nice, since there's no more insanity
14 actually only used once interfaces. But the cl
15 more potential to screw things up.
16
17 To avoid repeating all the same mistakes again
18 lessons learned while botching the job for the
19 only cover technicalities and not the big-pict
20 submission ioctl exactly should look like. Lea
21 something every GPU driver has to do on its ow
22
23
24 Prerequisites
25 -------------
26
27 First the prerequisites. Without these you hav
28 will need to add a 32-bit compat layer:
29
30 * Only use fixed sized integers. To avoid con
31 the kernel has special types like __u32, __
32
33 * Align everything to the natural size and us
34 platforms don't necessarily align 64-bit va
35 64-bit platforms do. So we always need padd
36 this right.
37
38 * Pad the entire struct to a multiple of 64-b
39 64-bit types - the structure size will othe
40 64-bit. Having a different structure size h
41 structures to the kernel, or if the kernel
42 e.g. the drm core does.
43
44 * Pointers are __u64, cast from/to a uintptr_
45 from/to a void __user * in the kernel. Try
46 conversion or worse, fiddle the raw __u64 t
47 diminishes the checking tools like sparse c
48 u64_to_user_ptr can be used in the kernel t
49 and pointers of different sizes.
50
51
52 Basics
53 ------
54
55 With the joys of writing a compat layer avoide
56 fumbles. Neglecting these will make backward a
57 pain. And since getting things wrong on the fi
58 will have a second iteration or at least an ex
59
60 * Have a clear way for userspace to figure ou
61 extension is supported on a given kernel. I
62 rejecting the new flags/modes or ioctls (si
63 past) then you need a driver feature flag o
64
65 * Have a plan for extending ioctls with new f
66 the structure. The drm core checks the pass
67 and zero-extends any mismatches between ker
68 but isn't a complete solution since newer u
69 notice that the newly added fields at the e
70 needs a new driver feature flags.
71
72 * Check all unused fields and flags and all t
73 and reject the ioctl if that's not the case
74 future extensions is going right down the g
75 an ioctl struct with random stack garbage i
76 then bakes in the ABI that those fields can
77 but garbage. This is also the reason why yo
78 structures, even if you never use them in a
79 might insert could contain garbage.
80
81 * Have simple testcases for all of the above.
82
83
84 Fun with Error Paths
85 --------------------
86
87 Nowadays we don't have any excuse left any mor
88 little root exploits. This means we both need
89 error handling paths - GPUs will die eventuall
90 anyway:
91
92 * The ioctl must check for array overflows. A
93 over/underflows and clamping issues of inte
94 example is sprite positioning values fed di
95 hardware just having 12 bits or so. Works n
96 server doesn't bother with clamping itself
97 screen.
98
99 * Have simple testcases for every input valid
100 Check that the error code matches your expe
101 that you only test for one single error pat
102 otherwise perfectly valid data. Without thi
103 the ioctl already and shadow the codepath y
104 bugs and regressions.
105
106 * Make all your ioctls restartable. First X r
107 this will allow you to test 90% of all erro
108 interrupting your main test suite constantl
109 love for signal you'll get an excellent bas
110 paths pretty much for free for graphics dri
111 how you handle ioctl restarting - e.g. drm
112 userspace library. The i915 driver botched
113 now we're stuck forever with some arcane se
114 userspace.
115
116 * If you can't make a given codepath restarta
117 killable. GPUs just die and your users won'
118 entire box (by means of an unkillable X pro
119 still too tricky have a timeout or hangchec
120 effort in case the hardware has gone banana
121
122 * Have testcases for the really tricky corner
123 - it's way too easy to create a deadlock be
124 waiters.
125
126
127 Time, Waiting and Missing it
128 ----------------------------
129
130 GPUs do most everything asynchronously, so we
131 wait for outstanding ones. This is really tric
132 the ioctls supported by the drm/i915 get this
133 still tons more lessons to learn here.
134
135 * Use CLOCK_MONOTONIC as your reference time,
136 v4l use by default nowadays. But let usersp
137 derived from different clock domains like y
138 by the kernel) or some independent hardware
139 will mismatch if you look close enough, but
140 have this information they can at least com
141 get at the raw values of some clocks (e.g.
142 performance counter sampling instructions)
143
144 * Use __s64 seconds plus __u64 nanoseconds to
145 convenient time specification, but it's mos
146
147 * Check that input time values are normalized
148 that the kernel native struct ktime has a s
149 and nanoseconds, so beware here.
150
151 * For timeouts, use absolute times. If you're
152 ioctl restartable relative timeouts tend to
153 indefinitely extend your wait time due to r
154 Especially if your reference clock is somet
155 frame counter. With a spec lawyer hat on th
156 always be extended - but users will surely
157 starts to stutter due to this.
158
159 * Consider ditching any synchronous wait ioct
160 an asynchronous event on a pollable file de
161 into event driven applications' main loop.
162
163 * Have testcases for corner-cases, especially
164 already-completed events, successful waits
165 and suiting to your needs.
166
167
168 Leaking Resources, Not
169 ----------------------
170
171 A full-blown drm driver essentially implements
172 the given GPU platforms. This means a driver n
173 for different objects and other resources to u
174 entails its own little set of pitfalls:
175
176 * Always attach the lifetime of your dynamica
177 lifetime of a file descriptor. Consider usi
178 needs to be shared across processes - fd-p
179 also simplifies lifetime management for use
180
181 * Always have O_CLOEXEC support.
182
183 * Ensure that you have sufficient insulation
184 default pick a private per-fd namespace whi
185 explicitly. Only go with a more global per-
186 are truly device-unique. One counterexample
187 that the per-device modeset objects like co
188 framebuffer objects, which mostly are not s
189 namespace, private by default, for framebuf
190 suitable.
191
192 * Think about uniqueness requirements for use
193 drivers it's a userspace bug to submit the
194 command submission ioctl. But then if objec
195 to know whether it has seen an imported obj
196 already or not. I haven't tried this myself
197 of objects, but consider using inode number
198 as unique identifiers - it's how real files
199 Unfortunately this requires a full-blown vi
200
201
202 Last, but not Least
203 -------------------
204
205 Not every problem needs a new ioctl:
206
207 * Think hard whether you really want a driver
208 it's much quicker to push a driver-private
209 lengthy discussions for a more generic solu
210 private interface to spearhead a new concep
211 end, once the generic interface comes aroun
212 interfaces. Indefinitely.
213
214 * Consider other interfaces than ioctls. A sy
215 per-device settings, or for child objects w
216 output connectors in drm with all the detec
217 maybe only your testsuite needs this interf
218 disclaimer of not having a stable ABI would
219
220 Finally, the name of the game is to get it rig
221 your driver proves popular and your hardware p
222 be stuck with a given ioctl essentially foreve
223 horrible ioctls on newer iterations of your ha
224 years to accomplish this. And then again years
225 complain about regressions disappears, too.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.