1 .. SPDX-License-Identifier: GPL-2.0 1 .. SPDX-License-Identifier: GPL-2.0
2 2
3 ======================= 3 =======================
4 Energy Model of devices 4 Energy Model of devices
5 ======================= 5 =======================
6 6
7 1. Overview 7 1. Overview
8 ----------- 8 -----------
9 9
10 The Energy Model (EM) framework serves as an i 10 The Energy Model (EM) framework serves as an interface between drivers knowing
11 the power consumed by devices at various perfo 11 the power consumed by devices at various performance levels, and the kernel
12 subsystems willing to use that information to 12 subsystems willing to use that information to make energy-aware decisions.
13 13
14 The source of the information about the power 14 The source of the information about the power consumed by devices can vary greatly
15 from one platform to another. These power cost 15 from one platform to another. These power costs can be estimated using
16 devicetree data in some cases. In others, the 16 devicetree data in some cases. In others, the firmware will know better.
17 Alternatively, userspace might be best positio 17 Alternatively, userspace might be best positioned. And so on. In order to avoid
18 each and every client subsystem to re-implemen 18 each and every client subsystem to re-implement support for each and every
19 possible source of information on its own, the 19 possible source of information on its own, the EM framework intervenes as an
20 abstraction layer which standardizes the forma 20 abstraction layer which standardizes the format of power cost tables in the
21 kernel, hence enabling to avoid redundant work 21 kernel, hence enabling to avoid redundant work.
22 22
23 The power values might be expressed in micro-W 23 The power values might be expressed in micro-Watts or in an 'abstract scale'.
24 Multiple subsystems might use the EM and it is 24 Multiple subsystems might use the EM and it is up to the system integrator to
25 check that the requirements for the power valu 25 check that the requirements for the power value scale types are met. An example
26 can be found in the Energy-Aware Scheduler doc 26 can be found in the Energy-Aware Scheduler documentation
27 Documentation/scheduler/sched-energy.rst. For 27 Documentation/scheduler/sched-energy.rst. For some subsystems like thermal or
28 powercap power values expressed in an 'abstrac 28 powercap power values expressed in an 'abstract scale' might cause issues.
29 These subsystems are more interested in estima 29 These subsystems are more interested in estimation of power used in the past,
30 thus the real micro-Watts might be needed. An 30 thus the real micro-Watts might be needed. An example of these requirements can
31 be found in the Intelligent Power Allocation i 31 be found in the Intelligent Power Allocation in
32 Documentation/driver-api/thermal/power_allocat 32 Documentation/driver-api/thermal/power_allocator.rst.
33 Kernel subsystems might implement automatic de 33 Kernel subsystems might implement automatic detection to check whether EM
34 registered devices have inconsistent scale (ba 34 registered devices have inconsistent scale (based on EM internal flag).
35 Important thing to keep in mind is that when t 35 Important thing to keep in mind is that when the power values are expressed in
36 an 'abstract scale' deriving real energy in mi 36 an 'abstract scale' deriving real energy in micro-Joules would not be possible.
37 37
38 The figure below depicts an example of drivers 38 The figure below depicts an example of drivers (Arm-specific here, but the
39 approach is applicable to any architecture) pr 39 approach is applicable to any architecture) providing power costs to the EM
40 framework, and interested clients reading the 40 framework, and interested clients reading the data from it::
41 41
42 +---------------+ +-----------------+ 42 +---------------+ +-----------------+ +---------------+
43 | Thermal (IPA) | | Scheduler (EAS) | 43 | Thermal (IPA) | | Scheduler (EAS) | | Other |
44 +---------------+ +-----------------+ 44 +---------------+ +-----------------+ +---------------+
45 | | em_cpu_en 45 | | em_cpu_energy() |
46 | | em_cpu_ge 46 | | em_cpu_get() |
47 +---------+ | + 47 +---------+ | +---------+
48 | | | 48 | | |
49 v v v 49 v v v
50 +--------------------- 50 +---------------------+
51 | Energy Model 51 | Energy Model |
52 | Framework 52 | Framework |
53 +--------------------- 53 +---------------------+
54 ^ ^ ^ 54 ^ ^ ^
55 | | | e 55 | | | em_dev_register_perf_domain()
56 +----------+ | +-- 56 +----------+ | +---------+
57 | | 57 | | |
58 +---------------+ +---------------+ 58 +---------------+ +---------------+ +--------------+
59 | cpufreq-dt | | arm_scmi | 59 | cpufreq-dt | | arm_scmi | | Other |
60 +---------------+ +---------------+ 60 +---------------+ +---------------+ +--------------+
61 ^ ^ 61 ^ ^ ^
62 | | 62 | | |
63 +--------------+ +---------------+ 63 +--------------+ +---------------+ +--------------+
64 | Device Tree | | Firmware | 64 | Device Tree | | Firmware | | ? |
65 +--------------+ +---------------+ 65 +--------------+ +---------------+ +--------------+
66 66
67 In case of CPU devices the EM framework manage 67 In case of CPU devices the EM framework manages power cost tables per
68 'performance domain' in the system. A performa 68 'performance domain' in the system. A performance domain is a group of CPUs
69 whose performance is scaled together. Performa 69 whose performance is scaled together. Performance domains generally have a
70 1-to-1 mapping with CPUFreq policies. All CPUs 70 1-to-1 mapping with CPUFreq policies. All CPUs in a performance domain are
71 required to have the same micro-architecture. 71 required to have the same micro-architecture. CPUs in different performance
72 domains can have different micro-architectures 72 domains can have different micro-architectures.
73 73
74 To better reflect power variation due to stati <<
75 supports runtime modifications of the power va <<
76 RCU to free the modifiable EM perf_state table <<
77 scheduler, also uses RCU to access this memory <<
78 API for allocating/freeing the new memory for <<
79 The old memory is freed automatically using RC <<
80 are no owners anymore for the given EM runtime <<
81 using kref mechanism. The device driver which <<
82 should call EM API to free it safely when it's <<
83 framework will handle the clean-up when it's p <<
84 <<
85 The kernel code which want to modify the EM va <<
86 access using a mutex. Therefore, the device dr <<
87 context when it tries to modify the EM. <<
88 <<
89 With the runtime modifiable EM we switch from <<
90 runtime static EM' (system property) design to <<
91 changed during runtime according e.g. to the w <<
92 property) design. <<
93 <<
94 It is possible also to modify the CPU performa <<
95 performance state. Thus, the full power and pe <<
96 is an exponential curve) can be changed accord <<
97 or system property. <<
98 <<
99 74
100 2. Core APIs 75 2. Core APIs
101 ------------ 76 ------------
102 77
103 2.1 Config options 78 2.1 Config options
104 ^^^^^^^^^^^^^^^^^^ 79 ^^^^^^^^^^^^^^^^^^
105 80
106 CONFIG_ENERGY_MODEL must be enabled to use the 81 CONFIG_ENERGY_MODEL must be enabled to use the EM framework.
107 82
108 83
109 2.2 Registration of performance domains 84 2.2 Registration of performance domains
110 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 85 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
111 86
112 Registration of 'advanced' EM 87 Registration of 'advanced' EM
113 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 88 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
114 89
115 The 'advanced' EM gets its name due to the fac !! 90 The 'advanced' EM gets it's name due to the fact that the driver is allowed
116 to provide more precised power model. It's not 91 to provide more precised power model. It's not limited to some implemented math
117 formula in the framework (like it is in 'simpl !! 92 formula in the framework (like it's in 'simple' EM case). It can better reflect
118 the real power measurements performed for each 93 the real power measurements performed for each performance state. Thus, this
119 registration method should be preferred in cas 94 registration method should be preferred in case considering EM static power
120 (leakage) is important. 95 (leakage) is important.
121 96
122 Drivers are expected to register performance d 97 Drivers are expected to register performance domains into the EM framework by
123 calling the following API:: 98 calling the following API::
124 99
125 int em_dev_register_perf_domain(struct devic 100 int em_dev_register_perf_domain(struct device *dev, unsigned int nr_states,
126 struct em_data_callback *cb, c 101 struct em_data_callback *cb, cpumask_t *cpus, bool microwatts);
127 102
128 Drivers must provide a callback function retur 103 Drivers must provide a callback function returning <frequency, power> tuples
129 for each performance state. The callback funct 104 for each performance state. The callback function provided by the driver is free
130 to fetch data from any relevant location (DT, 105 to fetch data from any relevant location (DT, firmware, ...), and by any mean
131 deemed necessary. Only for CPU devices, driver 106 deemed necessary. Only for CPU devices, drivers must specify the CPUs of the
132 performance domains using cpumask. For other d 107 performance domains using cpumask. For other devices than CPUs the last
133 argument must be set to NULL. 108 argument must be set to NULL.
134 The last argument 'microwatts' is important to 109 The last argument 'microwatts' is important to set with correct value. Kernel
135 subsystems which use EM might rely on this fla 110 subsystems which use EM might rely on this flag to check if all EM devices use
136 the same scale. If there are different scales, 111 the same scale. If there are different scales, these subsystems might decide
137 to return warning/error, stop working or panic 112 to return warning/error, stop working or panic.
138 See Section 3. for an example of driver implem 113 See Section 3. for an example of driver implementing this
139 callback, or Section 2.4 for further documenta 114 callback, or Section 2.4 for further documentation on this API
140 115
141 Registration of EM using DT 116 Registration of EM using DT
142 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 117 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
143 118
144 The EM can also be registered using OPP frame 119 The EM can also be registered using OPP framework and information in DT
145 "operating-points-v2". Each OPP entry in DT ca 120 "operating-points-v2". Each OPP entry in DT can be extended with a property
146 "opp-microwatt" containing micro-Watts power v 121 "opp-microwatt" containing micro-Watts power value. This OPP DT property
147 allows a platform to register EM power values 122 allows a platform to register EM power values which are reflecting total power
148 (static + dynamic). These power values might b 123 (static + dynamic). These power values might be coming directly from
149 experiments and measurements. 124 experiments and measurements.
150 125
151 Registration of 'artificial' EM 126 Registration of 'artificial' EM
152 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 127 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
153 128
154 There is an option to provide a custom callbac 129 There is an option to provide a custom callback for drivers missing detailed
155 knowledge about power value for each performan 130 knowledge about power value for each performance state. The callback
156 .get_cost() is optional and provides the 'cost 131 .get_cost() is optional and provides the 'cost' values used by the EAS.
157 This is useful for platforms that only provide 132 This is useful for platforms that only provide information on relative
158 efficiency between CPU types, where one could 133 efficiency between CPU types, where one could use the information to
159 create an abstract power model. But even an ab 134 create an abstract power model. But even an abstract power model can
160 sometimes be hard to fit in, given the input p 135 sometimes be hard to fit in, given the input power value size restrictions.
161 The .get_cost() allows to provide the 'cost' v 136 The .get_cost() allows to provide the 'cost' values which reflect the
162 efficiency of the CPUs. This would allow to pr 137 efficiency of the CPUs. This would allow to provide EAS information which
163 has different relation than what would be forc 138 has different relation than what would be forced by the EM internal
164 formulas calculating 'cost' values. To registe 139 formulas calculating 'cost' values. To register an EM for such platform, the
165 driver must set the flag 'microwatts' to 0, pr 140 driver must set the flag 'microwatts' to 0, provide .get_power() callback
166 and provide .get_cost() callback. The EM frame 141 and provide .get_cost() callback. The EM framework would handle such platform
167 properly during registration. A flag EM_PERF_D 142 properly during registration. A flag EM_PERF_DOMAIN_ARTIFICIAL is set for such
168 platform. Special care should be taken by othe 143 platform. Special care should be taken by other frameworks which are using EM
169 to test and treat this flag properly. 144 to test and treat this flag properly.
170 145
171 Registration of 'simple' EM 146 Registration of 'simple' EM
172 ~~~~~~~~~~~~~~~~~~~~~~~~~~~ 147 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
173 148
174 The 'simple' EM is registered using the framew 149 The 'simple' EM is registered using the framework helper function
175 cpufreq_register_em_with_opp(). It implements 150 cpufreq_register_em_with_opp(). It implements a power model which is tight to
176 math formula:: 151 math formula::
177 152
178 Power = C * V^2 * f 153 Power = C * V^2 * f
179 154
180 The EM which is registered using this method m 155 The EM which is registered using this method might not reflect correctly the
181 physics of a real device, e.g. when static pow 156 physics of a real device, e.g. when static power (leakage) is important.
182 157
183 158
184 2.3 Accessing performance domains 159 2.3 Accessing performance domains
185 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 160 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
186 161
187 There are two API functions which provide the 162 There are two API functions which provide the access to the energy model:
188 em_cpu_get() which takes CPU id as an argument 163 em_cpu_get() which takes CPU id as an argument and em_pd_get() with device
189 pointer as an argument. It depends on the subs 164 pointer as an argument. It depends on the subsystem which interface it is
190 going to use, but in case of CPU devices both 165 going to use, but in case of CPU devices both functions return the same
191 performance domain. 166 performance domain.
192 167
193 Subsystems interested in the energy model of a 168 Subsystems interested in the energy model of a CPU can retrieve it using the
194 em_cpu_get() API. The energy model tables are 169 em_cpu_get() API. The energy model tables are allocated once upon creation of
195 the performance domains, and kept in memory un 170 the performance domains, and kept in memory untouched.
196 171
197 The energy consumed by a performance domain ca 172 The energy consumed by a performance domain can be estimated using the
198 em_cpu_energy() API. The estimation is perform 173 em_cpu_energy() API. The estimation is performed assuming that the schedutil
199 CPUfreq governor is in use in case of CPU devi 174 CPUfreq governor is in use in case of CPU device. Currently this calculation is
200 not provided for other type of devices. 175 not provided for other type of devices.
201 176
202 More details about the above APIs can be found 177 More details about the above APIs can be found in ``<linux/energy_model.h>``
203 or in Section 2.5 !! 178 or in Section 2.4
204 <<
205 179
206 2.4 Runtime modifications <<
207 ^^^^^^^^^^^^^^^^^^^^^^^^^ <<
208 180
209 Drivers willing to update the EM at runtime sh !! 181 2.4 Description details of this API
210 function to allocate a new instance of the mod <<
211 below:: <<
212 <<
213 struct em_perf_table __rcu *em_table_alloc(s <<
214 <<
215 This allows to allocate a structure which cont <<
216 also RCU and kref needed by the EM framework. <<
217 contains array 'struct em_perf_state state[]' <<
218 states in ascending order. That list must be p <<
219 which wants to update the EM. The list of freq <<
220 existing EM (created during boot). The content <<
221 must be populated by the driver as well. <<
222 <<
223 This is the API which does the EM update, usin <<
224 <<
225 int em_dev_update_perf_domain(struct device <<
226 struct em_perf_table _ <<
227 <<
228 Drivers must provide a pointer to the allocate <<
229 'struct em_perf_table'. That new EM will be sa <<
230 and will be visible to other sub-systems in th <<
231 The main design goal for this API is to be fas <<
232 or memory allocations at runtime. When pre-com <<
233 device driver, than it should be possible to s <<
234 performance overhead. <<
235 <<
236 In order to free the EM, provided earlier by t <<
237 is unloaded), there is a need to call the API: <<
238 <<
239 void em_table_free(struct em_perf_table __rc <<
240 <<
241 It will allow the EM framework to safely remov <<
242 no other sub-system using it, e.g. EAS. <<
243 <<
244 To use the power values in other sub-systems ( <<
245 a need to call API which protects the reader a <<
246 table data:: <<
247 <<
248 struct em_perf_state *em_perf_state_from_pd( <<
249 <<
250 It returns the 'struct em_perf_state' pointer <<
251 states in ascending order. <<
252 This function must be called in the RCU read l <<
253 rcu_read_lock()). When the EM table is not nee <<
254 call rcu_real_unlock(). In this way the EM saf <<
255 and protects the users. It also allows the EM <<
256 and free it. More details how to use it can be <<
257 example driver. <<
258 <<
259 There is dedicated API for device drivers to c <<
260 values:: <<
261 <<
262 int em_dev_compute_costs(struct device *dev, <<
263 int nr_states); <<
264 <<
265 These 'cost' values from EM are used in EAS. T <<
266 together with the number of entries and device <<
267 of the cost values is done properly the return <<
268 The function takes care for right setting of i <<
269 state as well. It updates em_perf_state::flags <<
270 Then such prepared new EM can be passed to the <<
271 function, which will allow to use it. <<
272 <<
273 More details about the above APIs can be found <<
274 or in Section 3.2 with an example code showing <<
275 updating mechanism in a device driver. <<
276 <<
277 <<
278 2.5 Description details of this API <<
279 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 182 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
280 .. kernel-doc:: include/linux/energy_model.h 183 .. kernel-doc:: include/linux/energy_model.h
281 :internal: 184 :internal:
282 185
283 .. kernel-doc:: kernel/power/energy_model.c 186 .. kernel-doc:: kernel/power/energy_model.c
284 :export: 187 :export:
285 188
286 189
287 3. Examples !! 190 3. Example driver
288 ----------- !! 191 -----------------
289 <<
290 3.1 Example driver with EM registration <<
291 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ <<
292 192
293 The CPUFreq framework supports dedicated callb 193 The CPUFreq framework supports dedicated callback for registering
294 the EM for a given CPU(s) 'policy' object: cpu 194 the EM for a given CPU(s) 'policy' object: cpufreq_driver::register_em().
295 That callback has to be implemented properly f 195 That callback has to be implemented properly for a given driver,
296 because the framework would call it at the rig 196 because the framework would call it at the right time during setup.
297 This section provides a simple example of a CP 197 This section provides a simple example of a CPUFreq driver registering a
298 performance domain in the Energy Model framewo 198 performance domain in the Energy Model framework using the (fake) 'foo'
299 protocol. The driver implements an est_power() 199 protocol. The driver implements an est_power() function to be provided to the
300 EM framework:: 200 EM framework::
301 201
302 -> drivers/cpufreq/foo_cpufreq.c 202 -> drivers/cpufreq/foo_cpufreq.c
303 203
304 01 static int est_power(struct device *de 204 01 static int est_power(struct device *dev, unsigned long *mW,
305 02 unsigned long *KHz) 205 02 unsigned long *KHz)
306 03 { 206 03 {
307 04 long freq, power; 207 04 long freq, power;
308 05 208 05
309 06 /* Use the 'foo' protocol to c 209 06 /* Use the 'foo' protocol to ceil the frequency */
310 07 freq = foo_get_freq_ceil(dev, 210 07 freq = foo_get_freq_ceil(dev, *KHz);
311 08 if (freq < 0); 211 08 if (freq < 0);
312 09 return freq; 212 09 return freq;
313 10 213 10
314 11 /* Estimate the power cost for 214 11 /* Estimate the power cost for the dev at the relevant freq. */
315 12 power = foo_estimate_power(dev 215 12 power = foo_estimate_power(dev, freq);
316 13 if (power < 0); 216 13 if (power < 0);
317 14 return power; 217 14 return power;
318 15 218 15
319 16 /* Return the values to the EM 219 16 /* Return the values to the EM framework */
320 17 *mW = power; 220 17 *mW = power;
321 18 *KHz = freq; 221 18 *KHz = freq;
322 19 222 19
323 20 return 0; 223 20 return 0;
324 21 } 224 21 }
325 22 225 22
326 23 static void foo_cpufreq_register_em(st 226 23 static void foo_cpufreq_register_em(struct cpufreq_policy *policy)
327 24 { 227 24 {
328 25 struct em_data_callback em_cb 228 25 struct em_data_callback em_cb = EM_DATA_CB(est_power);
329 26 struct device *cpu_dev; 229 26 struct device *cpu_dev;
330 27 int nr_opp; 230 27 int nr_opp;
331 28 231 28
332 29 cpu_dev = get_cpu_device(cpuma 232 29 cpu_dev = get_cpu_device(cpumask_first(policy->cpus));
333 30 233 30
334 31 /* Find the number of OPPs for 234 31 /* Find the number of OPPs for this policy */
335 32 nr_opp = foo_get_nr_opp(policy 235 32 nr_opp = foo_get_nr_opp(policy);
336 33 236 33
337 34 /* And register the new perfor 237 34 /* And register the new performance domain */
338 35 em_dev_register_perf_domain(cp 238 35 em_dev_register_perf_domain(cpu_dev, nr_opp, &em_cb, policy->cpus,
339 36 tr 239 36 true);
340 37 } 240 37 }
341 38 241 38
342 39 static struct cpufreq_driver foo_cpufr 242 39 static struct cpufreq_driver foo_cpufreq_driver = {
343 40 .register_em = foo_cpufreq_reg 243 40 .register_em = foo_cpufreq_register_em,
344 41 }; 244 41 };
345 <<
346 <<
347 3.2 Example driver with EM modification <<
348 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ <<
349 <<
350 This section provides a simple example of a th <<
351 The driver implements a foo_thermal_em_update( <<
352 up periodically to check the temperature and m <<
353 <<
354 -> drivers/soc/example/example_em_mod.c <<
355 <<
356 01 static void foo_get_new_em(struct foo_ <<
357 02 { <<
358 03 struct em_perf_table __rcu *em <<
359 04 struct em_perf_state *table, * <<
360 05 struct device *dev = ctx->dev; <<
361 06 struct em_perf_domain *pd; <<
362 07 unsigned long freq; <<
363 08 int i, ret; <<
364 09 <<
365 10 pd = em_pd_get(dev); <<
366 11 if (!pd) <<
367 12 return; <<
368 13 <<
369 14 em_table = em_table_alloc(pd); <<
370 15 if (!em_table) <<
371 16 return; <<
372 17 <<
373 18 new_table = em_table->state; <<
374 19 <<
375 20 rcu_read_lock(); <<
376 21 table = em_perf_state_from_pd( <<
377 22 for (i = 0; i < pd->nr_perf_st <<
378 23 freq = table[i].freque <<
379 24 foo_get_power_perf_val <<
380 25 } <<
381 26 rcu_read_unlock(); <<
382 27 <<
383 28 /* Calculate 'cost' values for <<
384 29 ret = em_dev_compute_costs(dev <<
385 30 if (ret) { <<
386 31 dev_warn(dev, "EM: com <<
387 32 em_free_table(em_table <<
388 33 return; <<
389 34 } <<
390 35 <<
391 36 ret = em_dev_update_perf_domai <<
392 37 if (ret) { <<
393 38 dev_warn(dev, "EM: upd <<
394 39 em_free_table(em_table <<
395 40 return; <<
396 41 } <<
397 42 <<
398 43 /* <<
399 44 * Since it's one-time-update <<
400 45 * The EM framework will later <<
401 46 */ <<
402 47 em_table_free(em_table); <<
403 48 } <<
404 49 <<
405 50 /* <<
406 51 * Function called periodically to che <<
407 52 * update the EM if needed <<
408 53 */ <<
409 54 static void foo_thermal_em_update(stru <<
410 55 { <<
411 56 struct device *dev = ctx->dev; <<
412 57 int cpu; <<
413 58 <<
414 59 ctx->temperature = foo_get_tem <<
415 60 if (ctx->temperature < FOO_EM_ <<
416 61 return; <<
417 62 <<
418 63 foo_get_new_em(ctx); <<
419 64 } <<
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.