1 ==================== 1 ====================
2 Scheduler Statistics 2 Scheduler Statistics
3 ==================== 3 ====================
4 4
5 Version 16 of schedstats changed the order of <<
6 'enum cpu_idle_type', which changed the order <<
7 columns in show_schedstat(). In particular the <<
8 and __CPU_NOT_IDLE changed places. The size of <<
9 <<
10 Version 15 of schedstats dropped counters for 5 Version 15 of schedstats dropped counters for some sched_yield:
11 yld_exp_empty, yld_act_empty and yld_both_empt 6 yld_exp_empty, yld_act_empty and yld_both_empty. Otherwise, it is
12 identical to version 14. 7 identical to version 14.
13 8
14 Version 14 of schedstats includes support for 9 Version 14 of schedstats includes support for sched_domains, which hit the
15 mainline kernel in 2.6.20 although it is ident 10 mainline kernel in 2.6.20 although it is identical to the stats from version
16 12 which was in the kernel from 2.6.13-2.6.19 11 12 which was in the kernel from 2.6.13-2.6.19 (version 13 never saw a kernel
17 release). Some counters make more sense to be 12 release). Some counters make more sense to be per-runqueue; other to be
18 per-domain. Note that domains (and their asso 13 per-domain. Note that domains (and their associated information) will only
19 be pertinent and available on machines utilizi 14 be pertinent and available on machines utilizing CONFIG_SMP.
20 15
21 In version 14 of schedstat, there is at least 16 In version 14 of schedstat, there is at least one level of domain
22 statistics for each cpu listed, and there may 17 statistics for each cpu listed, and there may well be more than one
23 domain. Domains have no particular names in t 18 domain. Domains have no particular names in this implementation, but
24 the highest numbered one typically arbitrates 19 the highest numbered one typically arbitrates balancing across all the
25 cpus on the machine, while domain0 is the most 20 cpus on the machine, while domain0 is the most tightly focused domain,
26 sometimes balancing only between pairs of cpus 21 sometimes balancing only between pairs of cpus. At this time, there
27 are no architectures which need more than thre 22 are no architectures which need more than three domain levels. The first
28 field in the domain stats is a bit map indicat 23 field in the domain stats is a bit map indicating which cpus are affected
29 by that domain. 24 by that domain.
30 25
31 These fields are counters, and only increment. 26 These fields are counters, and only increment. Programs which make use
32 of these will need to start with a baseline ob 27 of these will need to start with a baseline observation and then calculate
33 the change in the counters at each subsequent 28 the change in the counters at each subsequent observation. A perl script
34 which does this for many of the fields is avai 29 which does this for many of the fields is available at
35 30
36 http://eaglet.pdxhosts.com/rick/linux/sche 31 http://eaglet.pdxhosts.com/rick/linux/schedstat/
37 32
38 Note that any such script will necessarily be 33 Note that any such script will necessarily be version-specific, as the main
39 reason to change versions is changes in the ou 34 reason to change versions is changes in the output format. For those wishing
40 to write their own scripts, the fields are des 35 to write their own scripts, the fields are described here.
41 36
42 CPU statistics 37 CPU statistics
43 -------------- 38 --------------
44 cpu<N> 1 2 3 4 5 6 7 8 9 39 cpu<N> 1 2 3 4 5 6 7 8 9
45 40
46 First field is a sched_yield() statistic: 41 First field is a sched_yield() statistic:
47 42
48 1) # of times sched_yield() was called 43 1) # of times sched_yield() was called
49 44
50 Next three are schedule() statistics: 45 Next three are schedule() statistics:
51 46
52 2) This field is a legacy array expiratio 47 2) This field is a legacy array expiration count field used in the O(1)
53 scheduler. We kept it for ABI compatib 48 scheduler. We kept it for ABI compatibility, but it is always set to zero.
54 3) # of times schedule() was called 49 3) # of times schedule() was called
55 4) # of times schedule() left the process 50 4) # of times schedule() left the processor idle
56 51
57 Next two are try_to_wake_up() statistics: 52 Next two are try_to_wake_up() statistics:
58 53
59 5) # of times try_to_wake_up() was called 54 5) # of times try_to_wake_up() was called
60 6) # of times try_to_wake_up() was called 55 6) # of times try_to_wake_up() was called to wake up the local cpu
61 56
62 Next three are statistics describing schedulin 57 Next three are statistics describing scheduling latency:
63 58
64 7) sum of all time spent running by tasks 59 7) sum of all time spent running by tasks on this processor (in nanoseconds)
65 8) sum of all time spent waiting to run b 60 8) sum of all time spent waiting to run by tasks on this processor (in
66 nanoseconds) 61 nanoseconds)
67 9) # of timeslices run on this cpu 62 9) # of timeslices run on this cpu
68 63
69 64
70 Domain statistics 65 Domain statistics
71 ----------------- 66 -----------------
72 One of these is produced per domain for each c 67 One of these is produced per domain for each cpu described. (Note that if
73 CONFIG_SMP is not defined, *no* domains are ut 68 CONFIG_SMP is not defined, *no* domains are utilized and these lines
74 will not appear in the output.) 69 will not appear in the output.)
75 70
76 domain<N> <cpumask> 1 2 3 4 5 6 7 8 9 10 11 12 71 domain<N> <cpumask> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
77 72
78 The first field is a bit mask indicating what 73 The first field is a bit mask indicating what cpus this domain operates over.
79 74
80 The next 24 are a variety of sched_balance_rq( !! 75 The next 24 are a variety of load_balance() statistics in grouped into types
81 of idleness (idle, busy, and newly idle): 76 of idleness (idle, busy, and newly idle):
82 77
83 1) # of times in this domain sched_balanc !! 78 1) # of times in this domain load_balance() was called when the
84 cpu was idle 79 cpu was idle
85 2) # of times in this domain sched_balanc !! 80 2) # of times in this domain load_balance() checked but found
86 the load did not require balancing whe 81 the load did not require balancing when the cpu was idle
87 3) # of times in this domain sched_balanc !! 82 3) # of times in this domain load_balance() tried to move one or
88 more tasks and failed, when the cpu wa 83 more tasks and failed, when the cpu was idle
89 4) sum of imbalances discovered (if any) 84 4) sum of imbalances discovered (if any) with each call to
90 sched_balance_rq() in this domain when !! 85 load_balance() in this domain when the cpu was idle
91 5) # of times in this domain pull_task() 86 5) # of times in this domain pull_task() was called when the cpu
92 was idle 87 was idle
93 6) # of times in this domain pull_task() 88 6) # of times in this domain pull_task() was called even though
94 the target task was cache-hot when idl 89 the target task was cache-hot when idle
95 7) # of times in this domain sched_balanc !! 90 7) # of times in this domain load_balance() was called but did
96 not find a busier queue while the cpu 91 not find a busier queue while the cpu was idle
97 8) # of times in this domain a busier que 92 8) # of times in this domain a busier queue was found while the
98 cpu was idle but no busier group was f 93 cpu was idle but no busier group was found
99 9) # of times in this domain sched_balanc !! 94 9) # of times in this domain load_balance() was called when the
100 cpu was busy 95 cpu was busy
101 10) # of times in this domain sched_balanc !! 96 10) # of times in this domain load_balance() checked but found the
102 load did not require balancing when bu 97 load did not require balancing when busy
103 11) # of times in this domain sched_balanc !! 98 11) # of times in this domain load_balance() tried to move one or
104 more tasks and failed, when the cpu wa 99 more tasks and failed, when the cpu was busy
105 12) sum of imbalances discovered (if any) 100 12) sum of imbalances discovered (if any) with each call to
106 sched_balance_rq() in this domain when !! 101 load_balance() in this domain when the cpu was busy
107 13) # of times in this domain pull_task() 102 13) # of times in this domain pull_task() was called when busy
108 14) # of times in this domain pull_task() 103 14) # of times in this domain pull_task() was called even though the
109 target task was cache-hot when busy 104 target task was cache-hot when busy
110 15) # of times in this domain sched_balanc !! 105 15) # of times in this domain load_balance() was called but did not
111 find a busier queue while the cpu was 106 find a busier queue while the cpu was busy
112 16) # of times in this domain a busier que 107 16) # of times in this domain a busier queue was found while the cpu
113 was busy but no busier group was found 108 was busy but no busier group was found
114 109
115 17) # of times in this domain sched_balanc !! 110 17) # of times in this domain load_balance() was called when the
116 cpu was just becoming idle 111 cpu was just becoming idle
117 18) # of times in this domain sched_balanc !! 112 18) # of times in this domain load_balance() checked but found the
118 load did not require balancing when th 113 load did not require balancing when the cpu was just becoming idle
119 19) # of times in this domain sched_balanc !! 114 19) # of times in this domain load_balance() tried to move one or more
120 tasks and failed, when the cpu was jus 115 tasks and failed, when the cpu was just becoming idle
121 20) sum of imbalances discovered (if any) 116 20) sum of imbalances discovered (if any) with each call to
122 sched_balance_rq() in this domain when !! 117 load_balance() in this domain when the cpu was just becoming idle
123 21) # of times in this domain pull_task() 118 21) # of times in this domain pull_task() was called when newly idle
124 22) # of times in this domain pull_task() 119 22) # of times in this domain pull_task() was called even though the
125 target task was cache-hot when just be 120 target task was cache-hot when just becoming idle
126 23) # of times in this domain sched_balanc !! 121 23) # of times in this domain load_balance() was called but did not
127 find a busier queue while the cpu was 122 find a busier queue while the cpu was just becoming idle
128 24) # of times in this domain a busier que 123 24) # of times in this domain a busier queue was found while the cpu
129 was just becoming idle but no busier g 124 was just becoming idle but no busier group was found
130 125
131 Next three are active_load_balance() statis 126 Next three are active_load_balance() statistics:
132 127
133 25) # of times active_load_balance() was c 128 25) # of times active_load_balance() was called
134 26) # of times active_load_balance() tried 129 26) # of times active_load_balance() tried to move a task and failed
135 27) # of times active_load_balance() succe 130 27) # of times active_load_balance() successfully moved a task
136 131
137 Next three are sched_balance_exec() statist 132 Next three are sched_balance_exec() statistics:
138 133
139 28) sbe_cnt is not used 134 28) sbe_cnt is not used
140 29) sbe_balanced is not used 135 29) sbe_balanced is not used
141 30) sbe_pushed is not used 136 30) sbe_pushed is not used
142 137
143 Next three are sched_balance_fork() statist 138 Next three are sched_balance_fork() statistics:
144 139
145 31) sbf_cnt is not used 140 31) sbf_cnt is not used
146 32) sbf_balanced is not used 141 32) sbf_balanced is not used
147 33) sbf_pushed is not used 142 33) sbf_pushed is not used
148 143
149 Next three are try_to_wake_up() statistics: 144 Next three are try_to_wake_up() statistics:
150 145
151 34) # of times in this domain try_to_wake_ 146 34) # of times in this domain try_to_wake_up() awoke a task that
152 last ran on a different cpu in this do 147 last ran on a different cpu in this domain
153 35) # of times in this domain try_to_wake_ 148 35) # of times in this domain try_to_wake_up() moved a task to the
154 waking cpu because it was cache-cold o 149 waking cpu because it was cache-cold on its own cpu anyway
155 36) # of times in this domain try_to_wake_ 150 36) # of times in this domain try_to_wake_up() started passive balancing
156 151
157 /proc/<pid>/schedstat 152 /proc/<pid>/schedstat
158 --------------------- 153 ---------------------
159 schedstats also adds a new /proc/<pid>/schedst 154 schedstats also adds a new /proc/<pid>/schedstat file to include some of
160 the same information on a per-process level. 155 the same information on a per-process level. There are three fields in
161 this file correlating for that process to: 156 this file correlating for that process to:
162 157
163 1) time spent on the cpu (in nanoseconds) 158 1) time spent on the cpu (in nanoseconds)
164 2) time spent waiting on a runqueue (in n 159 2) time spent waiting on a runqueue (in nanoseconds)
165 3) # of timeslices run on this cpu 160 3) # of timeslices run on this cpu
166 161
167 A program could be easily written to make use 162 A program could be easily written to make use of these extra fields to
168 report on how well a particular process or set 163 report on how well a particular process or set of processes is faring
169 under the scheduler's policies. A simple vers 164 under the scheduler's policies. A simple version of such a program is
170 available at 165 available at
171 166
172 http://eaglet.pdxhosts.com/rick/linux/sche 167 http://eaglet.pdxhosts.com/rick/linux/schedstat/v12/latency.c
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.