1 .. SPDX-License-Identifier: GPL-2.0 1 .. SPDX-License-Identifier: GPL-2.0
2 2
3 ======================== 3 ========================
4 RCU and lockdep checking 4 RCU and lockdep checking
5 ======================== 5 ========================
6 6
7 All flavors of RCU have lockdep checking avail 7 All flavors of RCU have lockdep checking available, so that lockdep is
8 aware of when each task enters and leaves any 8 aware of when each task enters and leaves any flavor of RCU read-side
9 critical section. Each flavor of RCU is track 9 critical section. Each flavor of RCU is tracked separately (but note
10 that this is not the case in 2.6.32 and earlie 10 that this is not the case in 2.6.32 and earlier). This allows lockdep's
11 tracking to include RCU state, which can somet 11 tracking to include RCU state, which can sometimes help when debugging
12 deadlocks and the like. 12 deadlocks and the like.
13 13
14 In addition, RCU provides the following primit 14 In addition, RCU provides the following primitives that check lockdep's
15 state:: 15 state::
16 16
17 rcu_read_lock_held() for normal RCU. 17 rcu_read_lock_held() for normal RCU.
18 rcu_read_lock_bh_held() for RCU-bh. 18 rcu_read_lock_bh_held() for RCU-bh.
19 rcu_read_lock_sched_held() for RCU-sch 19 rcu_read_lock_sched_held() for RCU-sched.
20 rcu_read_lock_any_held() for any of no <<
21 srcu_read_lock_held() for SRCU. 20 srcu_read_lock_held() for SRCU.
22 rcu_read_lock_trace_held() for RCU Tas <<
23 21
24 These functions are conservative, and will the 22 These functions are conservative, and will therefore return 1 if they
25 aren't certain (for example, if CONFIG_DEBUG_L 23 aren't certain (for example, if CONFIG_DEBUG_LOCK_ALLOC is not set).
26 This prevents things like WARN_ON(!rcu_read_lo 24 This prevents things like WARN_ON(!rcu_read_lock_held()) from giving false
27 positives when lockdep is disabled. 25 positives when lockdep is disabled.
28 26
29 In addition, a separate kernel config paramete 27 In addition, a separate kernel config parameter CONFIG_PROVE_RCU enables
30 checking of rcu_dereference() primitives: 28 checking of rcu_dereference() primitives:
31 29
32 rcu_dereference(p): 30 rcu_dereference(p):
33 Check for RCU read-side critic 31 Check for RCU read-side critical section.
34 rcu_dereference_bh(p): 32 rcu_dereference_bh(p):
35 Check for RCU-bh read-side cri 33 Check for RCU-bh read-side critical section.
36 rcu_dereference_sched(p): 34 rcu_dereference_sched(p):
37 Check for RCU-sched read-side 35 Check for RCU-sched read-side critical section.
38 srcu_dereference(p, sp): 36 srcu_dereference(p, sp):
39 Check for SRCU read-side criti 37 Check for SRCU read-side critical section.
40 rcu_dereference_check(p, c): 38 rcu_dereference_check(p, c):
41 Use explicit check expression 39 Use explicit check expression "c" along with
42 rcu_read_lock_held(). This is 40 rcu_read_lock_held(). This is useful in code that is
43 invoked by both RCU readers an 41 invoked by both RCU readers and updaters.
44 rcu_dereference_bh_check(p, c): 42 rcu_dereference_bh_check(p, c):
45 Use explicit check expression 43 Use explicit check expression "c" along with
46 rcu_read_lock_bh_held(). This 44 rcu_read_lock_bh_held(). This is useful in code that
47 is invoked by both RCU-bh read 45 is invoked by both RCU-bh readers and updaters.
48 rcu_dereference_sched_check(p, c): 46 rcu_dereference_sched_check(p, c):
49 Use explicit check expression 47 Use explicit check expression "c" along with
50 rcu_read_lock_sched_held(). T 48 rcu_read_lock_sched_held(). This is useful in code that
51 is invoked by both RCU-sched r 49 is invoked by both RCU-sched readers and updaters.
52 srcu_dereference_check(p, c): 50 srcu_dereference_check(p, c):
53 Use explicit check expression 51 Use explicit check expression "c" along with
54 srcu_read_lock_held(). This i 52 srcu_read_lock_held(). This is useful in code that
55 is invoked by both SRCU reader 53 is invoked by both SRCU readers and updaters.
56 rcu_dereference_raw(p): 54 rcu_dereference_raw(p):
57 Don't check. (Use sparingly, 55 Don't check. (Use sparingly, if at all.)
58 rcu_dereference_raw_check(p): <<
59 Don't do lockdep at all. (Use <<
60 rcu_dereference_protected(p, c): 56 rcu_dereference_protected(p, c):
61 Use explicit check expression 57 Use explicit check expression "c", and omit all barriers
62 and compiler constraints. Thi 58 and compiler constraints. This is useful when the data
63 structure cannot change, for e 59 structure cannot change, for example, in code that is
64 invoked only by updaters. 60 invoked only by updaters.
65 rcu_access_pointer(p): 61 rcu_access_pointer(p):
66 Return the value of the pointe 62 Return the value of the pointer and omit all barriers,
67 but retain the compiler constr 63 but retain the compiler constraints that prevent duplicating
68 or coalescing. This is useful !! 64 or coalescsing. This is useful when when testing the
69 value of the pointer itself, f 65 value of the pointer itself, for example, against NULL.
70 66
71 The rcu_dereference_check() check expression c 67 The rcu_dereference_check() check expression can be any boolean
72 expression, but would normally include a lockd !! 68 expression, but would normally include a lockdep expression. However,
73 moderately ornate example, consider the follow !! 69 any boolean expression can be used. For a moderately ornate example,
>> 70 consider the following::
74 71
75 file = rcu_dereference_check(fdt->fd[f 72 file = rcu_dereference_check(fdt->fd[fd],
76 lockdep_i 73 lockdep_is_held(&files->file_lock) ||
77 atomic_re 74 atomic_read(&files->count) == 1);
78 75
79 This expression picks up the pointer "fdt->fd[ 76 This expression picks up the pointer "fdt->fd[fd]" in an RCU-safe manner,
80 and, if CONFIG_PROVE_RCU is configured, verifi 77 and, if CONFIG_PROVE_RCU is configured, verifies that this expression
81 is used in: 78 is used in:
82 79
83 1. An RCU read-side critical section (imp 80 1. An RCU read-side critical section (implicit), or
84 2. with files->file_lock held, or 81 2. with files->file_lock held, or
85 3. on an unshared files_struct. 82 3. on an unshared files_struct.
86 83
87 In case (1), the pointer is picked up in an RC 84 In case (1), the pointer is picked up in an RCU-safe manner for vanilla
88 RCU read-side critical sections, in case (2) t 85 RCU read-side critical sections, in case (2) the ->file_lock prevents
89 any change from taking place, and finally, in 86 any change from taking place, and finally, in case (3) the current task
90 is the only task accessing the file_struct, ag 87 is the only task accessing the file_struct, again preventing any change
91 from taking place. If the above statement was 88 from taking place. If the above statement was invoked only from updater
92 code, it could instead be written as follows:: 89 code, it could instead be written as follows::
93 90
94 file = rcu_dereference_protected(fdt-> 91 file = rcu_dereference_protected(fdt->fd[fd],
95 lockd 92 lockdep_is_held(&files->file_lock) ||
96 atomi 93 atomic_read(&files->count) == 1);
97 94
98 This would verify cases #2 and #3 above, and f 95 This would verify cases #2 and #3 above, and furthermore lockdep would
99 complain even if this was used in an RCU read- !! 96 complain if this was used in an RCU read-side critical section unless one
100 one of these two cases held. Because rcu_dere !! 97 of these two cases held. Because rcu_dereference_protected() omits all
101 all barriers and compiler constraints, it gene !! 98 barriers and compiler constraints, it generates better code than do the
102 the other flavors of rcu_dereference(). On th !! 99 other flavors of rcu_dereference(). On the other hand, it is illegal
103 to use rcu_dereference_protected() if either t 100 to use rcu_dereference_protected() if either the RCU-protected pointer
104 or the RCU-protected data that it points to ca 101 or the RCU-protected data that it points to can change concurrently.
105 102
106 Like rcu_dereference(), when lockdep is enable 103 Like rcu_dereference(), when lockdep is enabled, RCU list and hlist
107 traversal primitives check for being called fr 104 traversal primitives check for being called from within an RCU read-side
108 critical section. However, a lockdep expressi 105 critical section. However, a lockdep expression can be passed to them
109 as a additional optional argument. With this 106 as a additional optional argument. With this lockdep expression, these
110 traversal primitives will complain only if the 107 traversal primitives will complain only if the lockdep expression is
111 false and they are called from outside any RCU 108 false and they are called from outside any RCU read-side critical section.
112 109
113 For example, the workqueue for_each_pwq() macr 110 For example, the workqueue for_each_pwq() macro is intended to be used
114 either within an RCU read-side critical sectio 111 either within an RCU read-side critical section or with wq->mutex held.
115 It is thus implemented as follows:: 112 It is thus implemented as follows::
116 113
117 #define for_each_pwq(pwq, wq) 114 #define for_each_pwq(pwq, wq)
118 list_for_each_entry_rcu((pwq), 115 list_for_each_entry_rcu((pwq), &(wq)->pwqs, pwqs_node,
119 lock_i 116 lock_is_held(&(wq->mutex).dep_map))
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.