1 .. _development_process: 1 .. _development_process:
2 2
3 How the development process works 3 How the development process works
4 ================================= 4 =================================
5 5
6 Linux kernel development in the early 1990's w 6 Linux kernel development in the early 1990's was a pretty loose affair,
7 with relatively small numbers of users and dev 7 with relatively small numbers of users and developers involved. With a
8 user base in the millions and with some 2,000 8 user base in the millions and with some 2,000 developers involved over the
9 course of one year, the kernel has since had t 9 course of one year, the kernel has since had to evolve a number of
10 processes to keep development happening smooth 10 processes to keep development happening smoothly. A solid understanding of
11 how the process works is required in order to 11 how the process works is required in order to be an effective part of it.
12 12
13 The big picture 13 The big picture
14 --------------- 14 ---------------
15 15
16 The kernel developers use a loosely time-based 16 The kernel developers use a loosely time-based release process, with a new
17 major kernel release happening every two or th 17 major kernel release happening every two or three months. The recent
18 release history looks like this: 18 release history looks like this:
19 19
20 ====== ================= 20 ====== =================
21 5.0 March 3, 2019 !! 21 4.11 April 30, 2017
22 5.1 May 5, 2019 !! 22 4.12 July 2, 2017
23 5.2 July 7, 2019 !! 23 4.13 September 3, 2017
24 5.3 September 15, 2019 !! 24 4.14 November 12, 2017
25 5.4 November 24, 2019 !! 25 4.15 January 28, 2018
26 5.5 January 6, 2020 !! 26 4.16 April 1, 2018
27 ====== ================= 27 ====== =================
28 28
29 Every 5.x release is a major kernel release wi !! 29 Every 4.x release is a major kernel release with new features, internal
30 API changes, and more. A typical release can !! 30 API changes, and more. A typical 4.x release contain about 13,000
31 changesets with changes to several hundred tho !! 31 changesets with changes to several hundred thousand lines of code. 4.x is
32 the leading edge of Linux kernel development; !! 32 thus the leading edge of Linux kernel development; the kernel uses a
33 rolling development model which is continually 33 rolling development model which is continually integrating major changes.
34 34
35 A relatively straightforward discipline is fol 35 A relatively straightforward discipline is followed with regard to the
36 merging of patches for each release. At the b 36 merging of patches for each release. At the beginning of each development
37 cycle, the "merge window" is said to be open. 37 cycle, the "merge window" is said to be open. At that time, code which is
38 deemed to be sufficiently stable (and which is 38 deemed to be sufficiently stable (and which is accepted by the development
39 community) is merged into the mainline kernel. 39 community) is merged into the mainline kernel. The bulk of changes for a
40 new development cycle (and all of the major ch 40 new development cycle (and all of the major changes) will be merged during
41 this time, at a rate approaching 1,000 changes 41 this time, at a rate approaching 1,000 changes ("patches," or "changesets")
42 per day. 42 per day.
43 43
44 (As an aside, it is worth noting that the chan 44 (As an aside, it is worth noting that the changes integrated during the
45 merge window do not come out of thin air; they 45 merge window do not come out of thin air; they have been collected, tested,
46 and staged ahead of time. How that process wo 46 and staged ahead of time. How that process works will be described in
47 detail later on). 47 detail later on).
48 48
49 The merge window lasts for approximately two w 49 The merge window lasts for approximately two weeks. At the end of this
50 time, Linus Torvalds will declare that the win 50 time, Linus Torvalds will declare that the window is closed and release the
51 first of the "rc" kernels. For the kernel whi !! 51 first of the "rc" kernels. For the kernel which is destined to be 2.6.40,
52 for example, the release which happens at the 52 for example, the release which happens at the end of the merge window will
53 be called 5.6-rc1. The -rc1 release is the si !! 53 be called 2.6.40-rc1. The -rc1 release is the signal that the time to
54 merge new features has passed, and that the ti 54 merge new features has passed, and that the time to stabilize the next
55 kernel has begun. 55 kernel has begun.
56 56
57 Over the next six to ten weeks, only patches w 57 Over the next six to ten weeks, only patches which fix problems should be
58 submitted to the mainline. On occasion a more 58 submitted to the mainline. On occasion a more significant change will be
59 allowed, but such occasions are rare; develope 59 allowed, but such occasions are rare; developers who try to merge new
60 features outside of the merge window tend to g 60 features outside of the merge window tend to get an unfriendly reception.
61 As a general rule, if you miss the merge windo 61 As a general rule, if you miss the merge window for a given feature, the
62 best thing to do is to wait for the next devel 62 best thing to do is to wait for the next development cycle. (An occasional
63 exception is made for drivers for previously-u 63 exception is made for drivers for previously-unsupported hardware; if they
64 touch no in-tree code, they cannot cause regre 64 touch no in-tree code, they cannot cause regressions and should be safe to
65 add at any time). 65 add at any time).
66 66
67 As fixes make their way into the mainline, the 67 As fixes make their way into the mainline, the patch rate will slow over
68 time. Linus releases new -rc kernels about on 68 time. Linus releases new -rc kernels about once a week; a normal series
69 will get up to somewhere between -rc6 and -rc9 69 will get up to somewhere between -rc6 and -rc9 before the kernel is
70 considered to be sufficiently stable and the f !! 70 considered to be sufficiently stable and the final 2.6.x release is made.
71 At that point the whole process starts over ag 71 At that point the whole process starts over again.
72 72
73 As an example, here is how the 5.4 development !! 73 As an example, here is how the 4.16 development cycle went (all dates in
74 2019): !! 74 2018):
75 75
76 ============== ====================== 76 ============== ===============================
77 September 15 5.3 stable release !! 77 January 28 4.15 stable release
78 September 30 5.4-rc1, merge window !! 78 February 11 4.16-rc1, merge window closes
79 October 6 5.4-rc2 !! 79 February 18 4.16-rc2
80 October 13 5.4-rc3 !! 80 February 25 4.16-rc3
81 October 20 5.4-rc4 !! 81 March 4 4.16-rc4
82 October 27 5.4-rc5 !! 82 March 11 4.16-rc5
83 November 3 5.4-rc6 !! 83 March 18 4.16-rc6
84 November 10 5.4-rc7 !! 84 March 25 4.16-rc7
85 November 17 5.4-rc8 !! 85 April 1 4.16 stable release
86 November 24 5.4 stable release <<
87 ============== ====================== 86 ============== ===============================
88 87
89 How do the developers decide when to close the 88 How do the developers decide when to close the development cycle and create
90 the stable release? The most significant metr 89 the stable release? The most significant metric used is the list of
91 regressions from previous releases. No bugs a 90 regressions from previous releases. No bugs are welcome, but those which
92 break systems which worked in the past are con 91 break systems which worked in the past are considered to be especially
93 serious. For this reason, patches which cause 92 serious. For this reason, patches which cause regressions are looked upon
94 unfavorably and are quite likely to be reverte 93 unfavorably and are quite likely to be reverted during the stabilization
95 period. 94 period.
96 95
97 The developers' goal is to fix all known regre 96 The developers' goal is to fix all known regressions before the stable
98 release is made. In the real world, this kind 97 release is made. In the real world, this kind of perfection is hard to
99 achieve; there are just too many variables in 98 achieve; there are just too many variables in a project of this size.
100 There comes a point where delaying the final r 99 There comes a point where delaying the final release just makes the problem
101 worse; the pile of changes waiting for the nex 100 worse; the pile of changes waiting for the next merge window will grow
102 larger, creating even more regressions the nex !! 101 larger, creating even more regressions the next time around. So most 4.x
103 kernels go out with a handful of known regress 102 kernels go out with a handful of known regressions though, hopefully, none
104 of them are serious. 103 of them are serious.
105 104
106 Once a stable release is made, its ongoing mai 105 Once a stable release is made, its ongoing maintenance is passed off to the
107 "stable team," currently Greg Kroah-Hartman. T !! 106 "stable team," currently consisting of Greg Kroah-Hartman. The stable team
108 occasional updates to the stable release using !! 107 will release occasional updates to the stable release using the 4.x.y
109 To be considered for an update release, a patc !! 108 numbering scheme. To be considered for an update release, a patch must (1)
110 bug, and (2) already be merged into the mainli !! 109 fix a significant bug, and (2) already be merged into the mainline for the
111 kernel. Kernels will typically receive stable !! 110 next development kernel. Kernels will typically receive stable updates for
112 than one development cycle past their initial !! 111 a little more than one development cycle past their initial release. So,
113 5.2 kernel's history looked like this (all dat !! 112 for example, the 4.13 kernel's history looked like:
114 113
115 ============== ====================== 114 ============== ===============================
116 July 7 5.2 stable release !! 115 September 3 4.13 stable release
117 July 14 5.2.1 !! 116 September 13 4.13.1
118 July 21 5.2.2 !! 117 September 20 4.13.2
119 July 26 5.2.3 !! 118 September 27 4.13.3
120 July 28 5.2.4 !! 119 October 5 4.13.4
121 July 31 5.2.5 !! 120 October 12 4.13.5
122 ... ... 121 ... ...
123 October 11 5.2.21 !! 122 November 24 4.13.16
124 ============== ====================== 123 ============== ===============================
125 124
126 5.2.21 was the final stable update of the 5.2 !! 125 4.13.16 was the final stable update of the 4.13 release.
127 126
128 Some kernels are designated "long term" kernel 127 Some kernels are designated "long term" kernels; they will receive support
129 for a longer period. Please refer to the foll !! 128 for a longer period. As of this writing, the current long term kernels
130 long term kernel versions and their maintainer !! 129 and their maintainers are:
131 130
132 https://www.kernel.org/category/releas !! 131 ====== ====================== ==============================
>> 132 3.16 Ben Hutchings (very long-term stable kernel)
>> 133 4.1 Sasha Levin
>> 134 4.4 Greg Kroah-Hartman (very long-term stable kernel)
>> 135 4.9 Greg Kroah-Hartman
>> 136 4.14 Greg Kroah-Hartman
>> 137 ====== ====================== ==============================
133 138
134 The selection of a kernel for long-term suppor 139 The selection of a kernel for long-term support is purely a matter of a
135 maintainer having the need and the time to mai 140 maintainer having the need and the time to maintain that release. There
136 are no known plans for long-term support for a 141 are no known plans for long-term support for any specific upcoming
137 release. 142 release.
138 143
139 144
140 The lifecycle of a patch 145 The lifecycle of a patch
141 ------------------------ 146 ------------------------
142 147
143 Patches do not go directly from the developer' 148 Patches do not go directly from the developer's keyboard into the mainline
144 kernel. There is, instead, a somewhat involve 149 kernel. There is, instead, a somewhat involved (if somewhat informal)
145 process designed to ensure that each patch is 150 process designed to ensure that each patch is reviewed for quality and that
146 each patch implements a change which is desira 151 each patch implements a change which is desirable to have in the mainline.
147 This process can happen quickly for minor fixe 152 This process can happen quickly for minor fixes, or, in the case of large
148 and controversial changes, go on for years. M 153 and controversial changes, go on for years. Much developer frustration
149 comes from a lack of understanding of this pro 154 comes from a lack of understanding of this process or from attempts to
150 circumvent it. 155 circumvent it.
151 156
152 In the hopes of reducing that frustration, thi 157 In the hopes of reducing that frustration, this document will describe how
153 a patch gets into the kernel. What follows be 158 a patch gets into the kernel. What follows below is an introduction which
154 describes the process in a somewhat idealized 159 describes the process in a somewhat idealized way. A much more detailed
155 treatment will come in later sections. 160 treatment will come in later sections.
156 161
157 The stages that a patch goes through are, gene 162 The stages that a patch goes through are, generally:
158 163
159 - Design. This is where the real requirement 164 - Design. This is where the real requirements for the patch - and the way
160 those requirements will be met - are laid o 165 those requirements will be met - are laid out. Design work is often
161 done without involving the community, but i 166 done without involving the community, but it is better to do this work
162 in the open if at all possible; it can save 167 in the open if at all possible; it can save a lot of time redesigning
163 things later. 168 things later.
164 169
165 - Early review. Patches are posted to the re 170 - Early review. Patches are posted to the relevant mailing list, and
166 developers on that list reply with any comm 171 developers on that list reply with any comments they may have. This
167 process should turn up any major problems w 172 process should turn up any major problems with a patch if all goes
168 well. 173 well.
169 174
170 - Wider review. When the patch is getting cl 175 - Wider review. When the patch is getting close to ready for mainline
171 inclusion, it should be accepted by a relev 176 inclusion, it should be accepted by a relevant subsystem maintainer -
172 though this acceptance is not a guarantee t 177 though this acceptance is not a guarantee that the patch will make it
173 all the way to the mainline. The patch wil 178 all the way to the mainline. The patch will show up in the maintainer's
174 subsystem tree and into the -next trees (de 179 subsystem tree and into the -next trees (described below). When the
175 process works, this step leads to more exte 180 process works, this step leads to more extensive review of the patch and
176 the discovery of any problems resulting fro 181 the discovery of any problems resulting from the integration of this
177 patch with work being done by others. 182 patch with work being done by others.
178 183
179 - Please note that most maintainers also have 184 - Please note that most maintainers also have day jobs, so merging
180 your patch may not be their highest priorit 185 your patch may not be their highest priority. If your patch is
181 getting feedback about changes that are nee 186 getting feedback about changes that are needed, you should either
182 make those changes or justify why they shou 187 make those changes or justify why they should not be made. If your
183 patch has no review complaints but is not b 188 patch has no review complaints but is not being merged by its
184 appropriate subsystem or driver maintainer, 189 appropriate subsystem or driver maintainer, you should be persistent
185 in updating the patch to the current kernel 190 in updating the patch to the current kernel so that it applies cleanly
186 and keep sending it for review and merging. 191 and keep sending it for review and merging.
187 192
188 - Merging into the mainline. Eventually, a s 193 - Merging into the mainline. Eventually, a successful patch will be
189 merged into the mainline repository managed 194 merged into the mainline repository managed by Linus Torvalds. More
190 comments and/or problems may surface at thi 195 comments and/or problems may surface at this time; it is important that
191 the developer be responsive to these and fi 196 the developer be responsive to these and fix any issues which arise.
192 197
193 - Stable release. The number of users potent 198 - Stable release. The number of users potentially affected by the patch
194 is now large, so, once again, new problems 199 is now large, so, once again, new problems may arise.
195 200
196 - Long-term maintenance. While it is certain 201 - Long-term maintenance. While it is certainly possible for a developer
197 to forget about code after merging it, that 202 to forget about code after merging it, that sort of behavior tends to
198 leave a poor impression in the development 203 leave a poor impression in the development community. Merging code
199 eliminates some of the maintenance burden, 204 eliminates some of the maintenance burden, in that others will fix
200 problems caused by API changes. But the or 205 problems caused by API changes. But the original developer should
201 continue to take responsibility for the cod 206 continue to take responsibility for the code if it is to remain useful
202 in the longer term. 207 in the longer term.
203 208
204 One of the largest mistakes made by kernel dev 209 One of the largest mistakes made by kernel developers (or their employers)
205 is to try to cut the process down to a single 210 is to try to cut the process down to a single "merging into the mainline"
206 step. This approach invariably leads to frust 211 step. This approach invariably leads to frustration for everybody
207 involved. 212 involved.
208 213
209 How patches get into the Kernel 214 How patches get into the Kernel
210 ------------------------------- 215 -------------------------------
211 216
212 There is exactly one person who can merge patc 217 There is exactly one person who can merge patches into the mainline kernel
213 repository: Linus Torvalds. But, for example, !! 218 repository: Linus Torvalds. But, of the over 9,500 patches which went
214 which went into the 2.6.38 kernel, only 112 (a !! 219 into the 2.6.38 kernel, only 112 (around 1.3%) were directly chosen by Linus
215 chosen by Linus himself. The kernel project ha !! 220 himself. The kernel project has long since grown to a size where no single
216 where no single developer could possibly inspe !! 221 developer could possibly inspect and select every patch unassisted. The
217 unassisted. The way the kernel developers have !! 222 way the kernel developers have addressed this growth is through the use of
218 through the use of a lieutenant system built a !! 223 a lieutenant system built around a chain of trust.
219 224
220 The kernel code base is logically broken down 225 The kernel code base is logically broken down into a set of subsystems:
221 networking, specific architecture support, mem 226 networking, specific architecture support, memory management, video
222 devices, etc. Most subsystems have a designat 227 devices, etc. Most subsystems have a designated maintainer, a developer
223 who has overall responsibility for the code wi 228 who has overall responsibility for the code within that subsystem. These
224 subsystem maintainers are the gatekeepers (in 229 subsystem maintainers are the gatekeepers (in a loose way) for the portion
225 of the kernel they manage; they are the ones w 230 of the kernel they manage; they are the ones who will (usually) accept a
226 patch for inclusion into the mainline kernel. 231 patch for inclusion into the mainline kernel.
227 232
228 Subsystem maintainers each manage their own ve 233 Subsystem maintainers each manage their own version of the kernel source
229 tree, usually (but certainly not always) using 234 tree, usually (but certainly not always) using the git source management
230 tool. Tools like git (and related tools like 235 tool. Tools like git (and related tools like quilt or mercurial) allow
231 maintainers to track a list of patches, includ 236 maintainers to track a list of patches, including authorship information
232 and other metadata. At any given time, the ma 237 and other metadata. At any given time, the maintainer can identify which
233 patches in his or her repository are not found 238 patches in his or her repository are not found in the mainline.
234 239
235 When the merge window opens, top-level maintai 240 When the merge window opens, top-level maintainers will ask Linus to "pull"
236 the patches they have selected for merging fro 241 the patches they have selected for merging from their repositories. If
237 Linus agrees, the stream of patches will flow 242 Linus agrees, the stream of patches will flow up into his repository,
238 becoming part of the mainline kernel. The amo 243 becoming part of the mainline kernel. The amount of attention that Linus
239 pays to specific patches received in a pull op 244 pays to specific patches received in a pull operation varies. It is clear
240 that, sometimes, he looks quite closely. But, 245 that, sometimes, he looks quite closely. But, as a general rule, Linus
241 trusts the subsystem maintainers to not send b 246 trusts the subsystem maintainers to not send bad patches upstream.
242 247
243 Subsystem maintainers, in turn, can pull patch 248 Subsystem maintainers, in turn, can pull patches from other maintainers.
244 For example, the networking tree is built from 249 For example, the networking tree is built from patches which accumulated
245 first in trees dedicated to network device dri 250 first in trees dedicated to network device drivers, wireless networking,
246 etc. This chain of repositories can be arbitr 251 etc. This chain of repositories can be arbitrarily long, though it rarely
247 exceeds two or three links. Since each mainta 252 exceeds two or three links. Since each maintainer in the chain trusts
248 those managing lower-level trees, this process 253 those managing lower-level trees, this process is known as the "chain of
249 trust." 254 trust."
250 255
251 Clearly, in a system like this, getting patche 256 Clearly, in a system like this, getting patches into the kernel depends on
252 finding the right maintainer. Sending patches 257 finding the right maintainer. Sending patches directly to Linus is not
253 normally the right way to go. 258 normally the right way to go.
254 259
255 260
256 Next trees 261 Next trees
257 ---------- 262 ----------
258 263
259 The chain of subsystem trees guides the flow o 264 The chain of subsystem trees guides the flow of patches into the kernel,
260 but it also raises an interesting question: wh 265 but it also raises an interesting question: what if somebody wants to look
261 at all of the patches which are being prepared 266 at all of the patches which are being prepared for the next merge window?
262 Developers will be interested in what other ch 267 Developers will be interested in what other changes are pending to see
263 whether there are any conflicts to worry about 268 whether there are any conflicts to worry about; a patch which changes a
264 core kernel function prototype, for example, w 269 core kernel function prototype, for example, will conflict with any other
265 patches which use the older form of that funct 270 patches which use the older form of that function. Reviewers and testers
266 want access to the changes in their integrated 271 want access to the changes in their integrated form before all of those
267 changes land in the mainline kernel. One coul 272 changes land in the mainline kernel. One could pull changes from all of
268 the interesting subsystem trees, but that woul 273 the interesting subsystem trees, but that would be a big and error-prone
269 job. 274 job.
270 275
271 The answer comes in the form of -next trees, w 276 The answer comes in the form of -next trees, where subsystem trees are
272 collected for testing and review. The older o 277 collected for testing and review. The older of these trees, maintained by
273 Andrew Morton, is called "-mm" (for memory man 278 Andrew Morton, is called "-mm" (for memory management, which is how it got
274 started). The -mm tree integrates patches fro 279 started). The -mm tree integrates patches from a long list of subsystem
275 trees; it also has some patches aimed at helpi 280 trees; it also has some patches aimed at helping with debugging.
276 281
277 Beyond that, -mm contains a significant collec 282 Beyond that, -mm contains a significant collection of patches which have
278 been selected by Andrew directly. These patch 283 been selected by Andrew directly. These patches may have been posted on a
279 mailing list, or they may apply to a part of t 284 mailing list, or they may apply to a part of the kernel for which there is
280 no designated subsystem tree. As a result, -m 285 no designated subsystem tree. As a result, -mm operates as a sort of
281 subsystem tree of last resort; if there is no 286 subsystem tree of last resort; if there is no other obvious path for a
282 patch into the mainline, it is likely to end u 287 patch into the mainline, it is likely to end up in -mm. Miscellaneous
283 patches which accumulate in -mm will eventuall 288 patches which accumulate in -mm will eventually either be forwarded on to
284 an appropriate subsystem tree or be sent direc 289 an appropriate subsystem tree or be sent directly to Linus. In a typical
285 development cycle, approximately 5-10% of the 290 development cycle, approximately 5-10% of the patches going into the
286 mainline get there via -mm. 291 mainline get there via -mm.
287 292
288 The current -mm patch is available in the "mmo 293 The current -mm patch is available in the "mmotm" (-mm of the moment)
289 directory at: 294 directory at:
290 295
291 https://www.ozlabs.org/~akpm/mmotm/ !! 296 http://www.ozlabs.org/~akpm/mmotm/
292 297
293 Use of the MMOTM tree is likely to be a frustr 298 Use of the MMOTM tree is likely to be a frustrating experience, though;
294 there is a definite chance that it will not ev 299 there is a definite chance that it will not even compile.
295 300
296 The primary tree for next-cycle patch merging 301 The primary tree for next-cycle patch merging is linux-next, maintained by
297 Stephen Rothwell. The linux-next tree is, by 302 Stephen Rothwell. The linux-next tree is, by design, a snapshot of what
298 the mainline is expected to look like after th 303 the mainline is expected to look like after the next merge window closes.
299 Linux-next trees are announced on the linux-ke 304 Linux-next trees are announced on the linux-kernel and linux-next mailing
300 lists when they are assembled; they can be dow 305 lists when they are assembled; they can be downloaded from:
301 306
302 https://www.kernel.org/pub/linux/kerne !! 307 http://www.kernel.org/pub/linux/kernel/next/
303 308
304 Linux-next has become an integral part of the 309 Linux-next has become an integral part of the kernel development process;
305 all patches merged during a given merge window 310 all patches merged during a given merge window should really have found
306 their way into linux-next some time before the 311 their way into linux-next some time before the merge window opens.
307 312
308 313
309 Staging trees 314 Staging trees
310 ------------- 315 -------------
311 316
312 The kernel source tree contains the drivers/st 317 The kernel source tree contains the drivers/staging/ directory, where
313 many sub-directories for drivers or filesystem 318 many sub-directories for drivers or filesystems that are on their way to
314 being added to the kernel tree live. They rem 319 being added to the kernel tree live. They remain in drivers/staging while
315 they still need more work; once complete, they 320 they still need more work; once complete, they can be moved into the
316 kernel proper. This is a way to keep track of 321 kernel proper. This is a way to keep track of drivers that aren't
317 up to Linux kernel coding or quality standards 322 up to Linux kernel coding or quality standards, but people may want to use
318 them and track development. 323 them and track development.
319 324
320 Greg Kroah-Hartman currently maintains the sta 325 Greg Kroah-Hartman currently maintains the staging tree. Drivers that
321 still need work are sent to him, with each dri 326 still need work are sent to him, with each driver having its own
322 subdirectory in drivers/staging/. Along with 327 subdirectory in drivers/staging/. Along with the driver source files, a
323 TODO file should be present in the directory a 328 TODO file should be present in the directory as well. The TODO file lists
324 the pending work that the driver needs for acc 329 the pending work that the driver needs for acceptance into the kernel
325 proper, as well as a list of people that shoul 330 proper, as well as a list of people that should be Cc'd for any patches to
326 the driver. Current rules require that driver 331 the driver. Current rules require that drivers contributed to staging
327 must, at a minimum, compile properly. 332 must, at a minimum, compile properly.
328 333
329 Staging can be a relatively easy way to get ne 334 Staging can be a relatively easy way to get new drivers into the mainline
330 where, with luck, they will come to the attent 335 where, with luck, they will come to the attention of other developers and
331 improve quickly. Entry into staging is not th 336 improve quickly. Entry into staging is not the end of the story, though;
332 code in staging which is not seeing regular pr 337 code in staging which is not seeing regular progress will eventually be
333 removed. Distributors also tend to be relativ 338 removed. Distributors also tend to be relatively reluctant to enable
334 staging drivers. So staging is, at best, a st 339 staging drivers. So staging is, at best, a stop on the way toward becoming
335 a proper mainline driver. 340 a proper mainline driver.
336 341
337 342
338 Tools 343 Tools
339 ----- 344 -----
340 345
341 As can be seen from the above text, the kernel 346 As can be seen from the above text, the kernel development process depends
342 heavily on the ability to herd collections of 347 heavily on the ability to herd collections of patches in various
343 directions. The whole thing would not work an 348 directions. The whole thing would not work anywhere near as well as it
344 does without suitably powerful tools. Tutoria 349 does without suitably powerful tools. Tutorials on how to use these tools
345 are well beyond the scope of this document, bu 350 are well beyond the scope of this document, but there is space for a few
346 pointers. 351 pointers.
347 352
348 By far the dominant source code management sys 353 By far the dominant source code management system used by the kernel
349 community is git. Git is one of a number of d 354 community is git. Git is one of a number of distributed version control
350 systems being developed in the free software c 355 systems being developed in the free software community. It is well tuned
351 for kernel development, in that it performs qu 356 for kernel development, in that it performs quite well when dealing with
352 large repositories and large numbers of patche 357 large repositories and large numbers of patches. It also has a reputation
353 for being difficult to learn and use, though i 358 for being difficult to learn and use, though it has gotten better over
354 time. Some sort of familiarity with git is al 359 time. Some sort of familiarity with git is almost a requirement for kernel
355 developers; even if they do not use it for the 360 developers; even if they do not use it for their own work, they'll need git
356 to keep up with what other developers (and the 361 to keep up with what other developers (and the mainline) are doing.
357 362
358 Git is now packaged by almost all Linux distri 363 Git is now packaged by almost all Linux distributions. There is a home
359 page at: 364 page at:
360 365
361 https://git-scm.com/ !! 366 http://git-scm.com/
362 367
363 That page has pointers to documentation and tu 368 That page has pointers to documentation and tutorials.
364 369
365 Among the kernel developers who do not use git 370 Among the kernel developers who do not use git, the most popular choice is
366 almost certainly Mercurial: 371 almost certainly Mercurial:
367 372
368 https://www.selenic.com/mercurial/ !! 373 http://www.selenic.com/mercurial/
369 374
370 Mercurial shares many features with git, but i 375 Mercurial shares many features with git, but it provides an interface which
371 many find easier to use. 376 many find easier to use.
372 377
373 The other tool worth knowing about is Quilt: 378 The other tool worth knowing about is Quilt:
374 379
375 https://savannah.nongnu.org/projects/q !! 380 http://savannah.nongnu.org/projects/quilt/
376 381
377 Quilt is a patch management system, rather tha 382 Quilt is a patch management system, rather than a source code management
378 system. It does not track history over time; 383 system. It does not track history over time; it is, instead, oriented
379 toward tracking a specific set of changes agai 384 toward tracking a specific set of changes against an evolving code base.
380 Some major subsystem maintainers use quilt to 385 Some major subsystem maintainers use quilt to manage patches intended to go
381 upstream. For the management of certain kinds 386 upstream. For the management of certain kinds of trees (-mm, for example),
382 quilt is the best tool for the job. 387 quilt is the best tool for the job.
383 388
384 389
385 Mailing lists 390 Mailing lists
386 ------------- 391 -------------
387 392
388 A great deal of Linux kernel development work 393 A great deal of Linux kernel development work is done by way of mailing
389 lists. It is hard to be a fully-functioning m 394 lists. It is hard to be a fully-functioning member of the community
390 without joining at least one list somewhere. 395 without joining at least one list somewhere. But Linux mailing lists also
391 represent a potential hazard to developers, wh 396 represent a potential hazard to developers, who risk getting buried under a
392 load of electronic mail, running afoul of the 397 load of electronic mail, running afoul of the conventions used on the Linux
393 lists, or both. 398 lists, or both.
394 399
395 Most kernel mailing lists are hosted at kernel !! 400 Most kernel mailing lists are run on vger.kernel.org; the master list can
396 be found at: 401 be found at:
397 402
398 https://subspace.kernel.org !! 403 http://vger.kernel.org/vger-lists.html
399 404
400 There are lists hosted elsewhere; please check !! 405 There are lists hosted elsewhere, though; a number of them are at
401 the list relevant for any particular subsystem !! 406 lists.redhat.com.
402 407
403 The core mailing list for kernel development i 408 The core mailing list for kernel development is, of course, linux-kernel.
404 This list is an intimidating place to be; volu 409 This list is an intimidating place to be; volume can reach 500 messages per
405 day, the amount of noise is high, the conversa 410 day, the amount of noise is high, the conversation can be severely
406 technical, and participants are not always con 411 technical, and participants are not always concerned with showing a high
407 degree of politeness. But there is no other p 412 degree of politeness. But there is no other place where the kernel
408 development community comes together as a whol 413 development community comes together as a whole; developers who avoid this
409 list will miss important information. 414 list will miss important information.
410 415
411 There are a few hints which can help with linu 416 There are a few hints which can help with linux-kernel survival:
412 417
413 - Have the list delivered to a separate folder 418 - Have the list delivered to a separate folder, rather than your main
414 mailbox. One must be able to ignore the str 419 mailbox. One must be able to ignore the stream for sustained periods of
415 time. 420 time.
416 421
417 - Do not try to follow every conversation - no 422 - Do not try to follow every conversation - nobody else does. It is
418 important to filter on both the topic of int 423 important to filter on both the topic of interest (though note that
419 long-running conversations can drift away fr 424 long-running conversations can drift away from the original subject
420 without changing the email subject line) and 425 without changing the email subject line) and the people who are
421 participating. 426 participating.
422 427
423 - Do not feed the trolls. If somebody is tryi 428 - Do not feed the trolls. If somebody is trying to stir up an angry
424 response, ignore them. 429 response, ignore them.
425 430
426 - When responding to linux-kernel email (or th 431 - When responding to linux-kernel email (or that on other lists) preserve
427 the Cc: header for all involved. In the abs 432 the Cc: header for all involved. In the absence of a strong reason (such
428 as an explicit request), you should never re 433 as an explicit request), you should never remove recipients. Always make
429 sure that the person you are responding to i 434 sure that the person you are responding to is in the Cc: list. This
430 convention also makes it unnecessary to expl 435 convention also makes it unnecessary to explicitly ask to be copied on
431 replies to your postings. 436 replies to your postings.
432 437
433 - Search the list archives (and the net as a w 438 - Search the list archives (and the net as a whole) before asking
434 questions. Some developers can get impatien 439 questions. Some developers can get impatient with people who clearly
435 have not done their homework. 440 have not done their homework.
436 441
437 - Use interleaved ("inline") replies, which ma !! 442 - Avoid top-posting (the practice of putting your answer above the quoted
438 read. (i.e. avoid top-posting -- the practic !! 443 text you are responding to). It makes your response harder to read and
439 the quoted text you are responding to.) For !! 444 makes a poor impression.
440 :ref:`Documentation/process/submitting-patch <<
441 445
442 - Ask on the correct mailing list. Linux-kern 446 - Ask on the correct mailing list. Linux-kernel may be the general meeting
443 point, but it is not the best place to find 447 point, but it is not the best place to find developers from all
444 subsystems. 448 subsystems.
445 449
446 The last point - finding the correct mailing l 450 The last point - finding the correct mailing list - is a common place for
447 beginning developers to go wrong. Somebody wh 451 beginning developers to go wrong. Somebody who asks a networking-related
448 question on linux-kernel will almost certainly 452 question on linux-kernel will almost certainly receive a polite suggestion
449 to ask on the netdev list instead, as that is 453 to ask on the netdev list instead, as that is the list frequented by most
450 networking developers. Other lists exist for 454 networking developers. Other lists exist for the SCSI, video4linux, IDE,
451 filesystem, etc. subsystems. The best place t 455 filesystem, etc. subsystems. The best place to look for mailing lists is
452 in the MAINTAINERS file packaged with the kern 456 in the MAINTAINERS file packaged with the kernel source.
453 457
454 458
455 Getting started with Kernel development 459 Getting started with Kernel development
456 --------------------------------------- 460 ---------------------------------------
457 461
458 Questions about how to get started with the ke 462 Questions about how to get started with the kernel development process are
459 common - from both individuals and companies. 463 common - from both individuals and companies. Equally common are missteps
460 which make the beginning of the relationship h 464 which make the beginning of the relationship harder than it has to be.
461 465
462 Companies often look to hire well-known develo 466 Companies often look to hire well-known developers to get a development
463 group started. This can, in fact, be an effec 467 group started. This can, in fact, be an effective technique. But it also
464 tends to be expensive and does not do much to 468 tends to be expensive and does not do much to grow the pool of experienced
465 kernel developers. It is possible to bring in 469 kernel developers. It is possible to bring in-house developers up to speed
466 on Linux kernel development, given the investm 470 on Linux kernel development, given the investment of a bit of time. Taking
467 this time can endow an employer with a group o 471 this time can endow an employer with a group of developers who understand
468 the kernel and the company both, and who can h 472 the kernel and the company both, and who can help to train others as well.
469 Over the medium term, this is often the more p 473 Over the medium term, this is often the more profitable approach.
470 474
471 Individual developers are often, understandabl 475 Individual developers are often, understandably, at a loss for a place to
472 start. Beginning with a large project can be 476 start. Beginning with a large project can be intimidating; one often wants
473 to test the waters with something smaller firs 477 to test the waters with something smaller first. This is the point where
474 some developers jump into the creation of patc 478 some developers jump into the creation of patches fixing spelling errors or
475 minor coding style issues. Unfortunately, suc 479 minor coding style issues. Unfortunately, such patches create a level of
476 noise which is distracting for the development 480 noise which is distracting for the development community as a whole, so,
477 increasingly, they are looked down upon. New 481 increasingly, they are looked down upon. New developers wishing to
478 introduce themselves to the community will not 482 introduce themselves to the community will not get the sort of reception
479 they wish for by these means. 483 they wish for by these means.
480 484
481 Andrew Morton gives this advice for aspiring k 485 Andrew Morton gives this advice for aspiring kernel developers
482 486
483 :: 487 ::
484 488
485 The #1 project for all kernel beginner 489 The #1 project for all kernel beginners should surely be "make sure
486 that the kernel runs perfectly at all 490 that the kernel runs perfectly at all times on all machines which
487 you can lay your hands on". Usually t 491 you can lay your hands on". Usually the way to do this is to work
488 with others on getting things fixed up 492 with others on getting things fixed up (this can require
489 persistence!) but that's fine - it's a 493 persistence!) but that's fine - it's a part of kernel development.
490 494
491 (https://lwn.net/Articles/283982/). !! 495 (http://lwn.net/Articles/283982/).
492 496
493 In the absence of obvious problems to fix, dev 497 In the absence of obvious problems to fix, developers are advised to look
494 at the current lists of regressions and open b 498 at the current lists of regressions and open bugs in general. There is
495 never any shortage of issues in need of fixing 499 never any shortage of issues in need of fixing; by addressing these issues,
496 developers will gain experience with the proce 500 developers will gain experience with the process while, at the same time,
497 building respect with the rest of the developm 501 building respect with the rest of the development community.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.