1 .. SPDX-License-Identifier: GPL-2.0 2 .. include:: <isonum.txt> 3 4 ============================================== 5 ``amd-pstate`` CPU Performance Scaling Driver 6 ============================================== 7 8 :Copyright: |copy| 2021 Advanced Micro Devices 9 10 :Author: Huang Rui <ray.huang@amd.com> 11 12 13 Introduction 14 =================== 15 16 ``amd-pstate`` is the AMD CPU performance scal 17 new CPU frequency control mechanism on modern 18 Linux kernel. The new mechanism is based on Co 19 Performance Control (CPPC) which provides fine 20 than legacy ACPI hardware P-States. Current AM 21 the ACPI P-states driver to manage CPU frequen 22 only in 3 P-states. CPPC replaces the ACPI P-s 23 flexible, low-latency interface for the Linux 24 communicate the performance hints to hardware. 25 26 ``amd-pstate`` leverages the Linux kernel gove 27 ``ondemand``, etc. to manage the performance h 28 CPPC hardware functionality that internally fo 29 specification (for details refer to AMD64 Arch 30 Volume 2: System Programming [1]_). Currently, 31 frequency control function according to kernel 32 Zen2 and Zen3 processors, and we will implemen 33 in future after we verify them on the hardware 34 35 36 AMD CPPC Overview 37 ======================= 38 39 Collaborative Processor Performance Control (C 40 continuous, abstract, and unit-less performanc 41 not tied to a specific performance state / fre 42 standard [2]_ which software can specify appli 43 hints as a relative target to the infrastructu 44 provide the low latency register model (MSR) i 45 interpreter for performance adjustments. ``amd 46 ``struct cpufreq_driver`` instance, ``amd_psta 47 to manage each performance update behavior. :: 48 49 Highest Perf ------>+-----------------------+ 50 | | 51 | | 52 | | 53 | | 54 | | 55 Nominal Perf ------>+-----------------------+ 56 | | 57 | | 58 | | 59 | | 60 | | 61 | | 62 | | 63 | | 64 | | 65 | | 66 | | 67 | | 68 | | 69 | | 70 | | 71 | | 72 Lowest non- | | 73 linear perf ------>+-----------------------+ 74 | | 75 | | 76 | | 77 Lowest perf ------>+-----------------------+ 78 | | 79 | | 80 | | 81 0 ------>+-----------------------+ 82 83 AMD P-Sta 84 85 86 .. _perf_cap: 87 88 AMD CPPC Performance Capability 89 -------------------------------- 90 91 Highest Performance (RO) 92 ......................... 93 94 This is the absolute maximum performance an in 95 assuming ideal conditions. This performance le 96 for long durations and may only be achievable 97 are in a specific state; for example, it may r 98 an idle state. This would be equivalent to the 99 supported by the processor. 100 101 Nominal (Guaranteed) Performance (RO) 102 ...................................... 103 104 This is the maximum sustained performance leve 105 ideal operating conditions. In the absence of 106 thermal, etc.), this is the performance level 107 be able to maintain continuously. All cores/pr 108 able to sustain their nominal performance stat 109 110 Lowest non-linear Performance (RO) 111 ................................... 112 113 This is the lowest performance level at which 114 achieved, for example, due to the combined eff 115 scaling. Above this threshold, lower performan 116 more energy efficient than higher performance 117 effectively conveys the most efficient perform 118 119 Lowest Performance (RO) 120 ........................ 121 122 This is the absolute lowest performance level 123 performance level lower than the lowest nonlin 124 cause an efficiency penalty but should reduce 125 consumption of the processor. 126 127 AMD CPPC Performance Control 128 ------------------------------ 129 130 ``amd-pstate`` passes performance goals throug 131 register drives the behavior of the desired pe 132 133 Minimum requested performance (RW) 134 ................................... 135 136 ``amd-pstate`` specifies the minimum allowed p 137 138 Maximum requested performance (RW) 139 ................................... 140 141 ``amd-pstate`` specifies a limit the maximum p 142 to be supplied by the hardware. 143 144 Desired performance target (RW) 145 ................................... 146 147 ``amd-pstate`` specifies a desired target in t 148 a relative number. This can be expressed as pe 149 performance (infrastructure max). Below the no 150 level, desired performance expresses the avera 151 processor subject to hardware. Above the nomin 152 the processor must provide at least nominal pe 153 if current operating conditions allow. 154 155 Energy Performance Preference (EPP) (RW) 156 ......................................... 157 158 This attribute provides a hint to the hardware 159 toward performance (0x0) or energy efficiency 160 161 162 Key Governors Support 163 ======================= 164 165 ``amd-pstate`` can be used with all the (gener 166 by the ``scaling_available_governors`` policy 167 it is responsible for the configuration of pol 168 CPUs and provides the ``CPUFreq`` core (and th 169 to the policy objects) with accurate informati 170 operating frequencies supported by the hardwar 171 ``scaling_cur_freq`` information comes from th 172 173 ``amd-pstate`` mainly supports ``schedutil`` a 174 frequency control. It is to fine tune the proc 175 ``amd-pstate`` to the ``schedutil`` with CPU C 176 registers the adjust_perf callback to implemen 177 similar to CPPC. It is initialized by ``sugov_ 178 CPU's update_util_data pointer to assign ``sug 179 utilization update callback function in the CP 180 will call ``cpufreq_update_util`` and assigns 181 to the ``struct sugov_cpu`` that the utilizati 182 Then, ``amd-pstate`` updates the desired perfo 183 scheduler assigned. 184 185 .. _processor_support: 186 187 Processor Support 188 ======================= 189 190 The ``amd-pstate`` initialization will fail if 191 SBIOS does not exist in the detected processor 192 to check the existence of ``_CPC``. All Zen ba 193 ACPI hardware P-States function, so when ``amd 194 the kernel will fall back to initialize the `` 195 196 There are two types of hardware implementation 197 `Full MSR Support <perf_cap_>`_ and another is 198 <perf_cap_>`_. It can use the :c:macro:`X86_FE 199 indicate the different types. (For details, re 200 Reference (PPR) for AMD Family 19h Model 51h, 201 ``amd-pstate`` is to register different ``stat 202 hardware implementations. 203 204 Currently, some of the Zen2 and Zen3 processor 205 future, it will be supported on more and more 206 207 Full MSR Support 208 ----------------- 209 210 Some new Zen3 processors such as Cezanne provi 211 while the :c:macro:`X86_FEATURE_CPPC` CPU feat 212 ``amd-pstate`` can handle the MSR register to 213 function in ``CPUFreq`` that can reduce the la 214 interrupt context. The functions with a ``psta 215 operations on MSR registers. 216 217 Shared Memory Support 218 ---------------------- 219 220 If the :c:macro:`X86_FEATURE_CPPC` CPU feature 221 processor supports the shared memory solution. 222 uses the ``cppc_acpi`` helper methods to imple 223 that are defined on ``static_call``. The funct 224 represent the operations of ACPI CPPC helpers 225 226 227 AMD P-States and ACPI hardware P-States always 228 processor. But AMD P-States has the higher pri 229 with :c:macro:`MSR_AMD_CPPC_ENABLE` or ``cppc_ 230 to the request from AMD P-States. 231 232 233 User Space Interface in ``sysfs`` - Per-policy 234 ============================================== 235 236 ``amd-pstate`` exposes several global attribut 237 control its functionality at the system level. 238 ``/sys/devices/system/cpu/cpufreq/policyX/`` d 239 240 root@hr-test1:/home/ray# ls /sys/devices/syst 241 /sys/devices/system/cpu/cpufreq/policy0/amd_p 242 /sys/devices/system/cpu/cpufreq/policy0/amd_p 243 /sys/devices/system/cpu/cpufreq/policy0/amd_p 244 245 246 ``amd_pstate_highest_perf / amd_pstate_max_fre 247 248 Maximum CPPC performance and CPU frequency tha 249 set, in percent of the maximum supported CPPC 250 performance supported in `AMD CPPC Performance 251 In some ASICs, the highest CPPC performance is 252 table, so we need to expose it to sysfs. If bo 253 still supported, this maximum frequency will b 254 ``cpuinfo``. On systems that support preferred 255 different values for some cores than others an 256 advertised by the platform at bootup. 257 This attribute is read-only. 258 259 ``amd_pstate_lowest_nonlinear_freq`` 260 261 The lowest non-linear CPPC CPU frequency that 262 in percent of the maximum supported CPPC perfo 263 lowest non-linear performance in `AMD CPPC Per 264 <perf_cap_>`_.) 265 This attribute is read-only. 266 267 ``amd_pstate_hw_prefcore`` 268 269 Whether the platform supports the preferred co 270 enabled. This attribute is read-only. 271 272 ``amd_pstate_prefcore_ranking`` 273 274 The performance ranking of the core. This numb 275 larger numbers are preferred at the time of re 276 runtime based on platform conditions. This att 277 278 ``energy_performance_available_preferences`` 279 280 A list of all the supported EPP preferences th 281 ``energy_performance_preference`` on this syst 282 These profiles represent different hints that 283 to the low-level firmware about the user's des 284 tradeoff. ``default`` represents the epp valu 285 firmware. This attribute is read-only. 286 287 ``energy_performance_preference`` 288 289 The current energy performance preference can 290 and user can change current preference accordi 291 Please get all support profiles list from 292 ``energy_performance_available_preferences`` a 293 integer values defined between 0 to 255 when E 294 firmware, if EPP feature is disabled, driver w 295 This attribute is read-write. 296 297 ``boost`` 298 The `boost` sysfs attribute provides control o 299 performance boost, allowing users to manage th 300 of the CPU. This attribute can be used to enab 301 on individual CPUs. 302 303 When the boost feature is enabled, the CPU can 304 beyond the base frequency, providing enhanced 305 On the other hand, disabling the boost feature 306 base frequency, which may be desirable in cert 307 efficiency or manage temperature. 308 309 To manipulate the `boost` attribute, users can 310 boost or `1` to enable it, for the respective 311 `/sys/devices/system/cpu/cpuX/cpufreq/boost`, 312 313 Other performance and frequency values can be 314 ``/sys/devices/system/cpu/cpuX/acpi_cppc/``, s 315 316 317 ``amd-pstate`` vs ``acpi-cpufreq`` 318 ====================================== 319 320 On the majority of AMD platforms supported by 321 provided by the platform firmware are used for 322 only provide 3 P-states on AMD processors. 323 However, on modern AMD APU and CPU series, har 324 Processor Performance Control according to the 325 for AMD platforms. That is, fine-grained and c 326 instead of the legacy hardware P-states. ``amd 327 module which supports the new AMD P-States mec 328 platforms. The AMD P-States mechanism is the m 329 efficiency frequency management method on AMD 330 331 332 ``amd-pstate`` Driver Operation Modes 333 ====================================== 334 335 ``amd_pstate`` CPPC has 3 operation modes: aut 336 non-autonomous (passive) mode and guided auton 337 Active/passive/guided mode can be chosen by di 338 339 - In autonomous mode, platform ignores the des 340 and takes into account only the values set t 341 performance preference registers. 342 - In non-autonomous mode, platform gets desire 343 from OS directly through Desired Performance 344 - In guided-autonomous mode, platform sets ope 345 autonomously according to the current worklo 346 OS through min and max performance registers 347 348 Active Mode 349 ------------ 350 351 ``amd_pstate=active`` 352 353 This is the low-level firmware control mode wh 354 driver with ``amd_pstate=active`` passed to th 355 In this mode, ``amd_pstate_epp`` driver provid 356 wants to bias toward performance (0x0) or ener 357 then CPPC power algorithm will calculate the r 358 cores frequency according to the power supply 359 hardware conditions. 360 361 Passive Mode 362 ------------ 363 364 ``amd_pstate=passive`` 365 366 It will be enabled if the ``amd_pstate=passive 367 In this mode, ``amd_pstate`` driver software s 368 performance scale as a relative number. This c 369 performance (infrastructure max). Below the no 370 desired performance expresses the average perf 371 to the Performance Reduction Tolerance registe 372 processor must provide at least nominal perfor 373 operating conditions allow. 374 375 Guided Mode 376 ----------- 377 378 ``amd_pstate=guided`` 379 380 If ``amd_pstate=guided`` is passed to kernel c 381 is activated. In this mode, driver requests m 382 level and the platform autonomously selects a 383 and appropriate to the current workload. 384 385 ``amd-pstate`` Preferred Core 386 ================================= 387 388 The core frequency is subjected to the process 389 Not all cores are able to reach the maximum fr 390 infrastructure limits. Consequently, AMD has r 391 maximum frequency of a part. This means that a 392 maximum frequency. To find the best process sc 393 scenario, OS needs to know the core ordering i 394 highest performance capability register of the 395 396 ``amd-pstate`` preferred core enables the sche 397 cores that can achieve a higher frequency with 398 core rankings can dynamically change based on 399 thermals and ageing. 400 401 The priority metric will be initialized by the 402 driver will also determine whether or not ``am 403 supported by the platform. 404 405 ``amd-pstate`` driver will provide an initial 406 The platform uses the CPPC interfaces to commu 407 operating system and scheduler to make sure th 408 with highest performance firstly for schedulin 409 driver receives a message with the highest per 410 update the core ranking and set the cpu's prio 411 412 ``amd-pstate`` Preferred Core Switch 413 ===================================== 414 Kernel Parameters 415 ----------------- 416 417 ``amd-pstate`` peferred core`` has two states: 418 Enable/disable states can be chosen by differe 419 Default enable ``amd-pstate`` preferred core. 420 421 ``amd_prefcore=disable`` 422 423 For systems that support ``amd-pstate`` prefer 424 always be advertised by the platform. But OS c 425 kernel parameter ``amd_prefcore=disable``. 426 427 User Space Interface in ``sysfs`` - General 428 =========================================== 429 430 Global Attributes 431 ----------------- 432 433 ``amd-pstate`` exposes several global attribut 434 control its functionality at the system level. 435 ``/sys/devices/system/cpu/amd_pstate/`` direct 436 437 ``status`` 438 Operation mode of the driver: "active" 439 440 "active" 441 The driver is functional and i 442 443 "passive" 444 The driver is functional and i 445 446 "guided" 447 The driver is functional and i 448 449 "disable" 450 The driver is unregistered and 451 452 This attribute can be written to in or 453 operation mode or to unregister it. T 454 one of the possible values of it and, 455 these values to the sysfs file will ca 456 to the operation mode represented by t 457 unregistered in the "disable" case. 458 459 ``prefcore`` 460 Preferred core state of the driver: "e 461 462 "enabled" 463 Enable the ``amd-pstate`` pref 464 465 "disabled" 466 Disable the ``amd-pstate`` pre 467 468 469 This attribute is read-only to check t 470 by the kernel parameter. 471 472 ``cpupower`` tool support for ``amd-pstate`` 473 ============================================== 474 475 ``amd-pstate`` is supported by the ``cpupower` 476 frequency information. Development is in progr 477 operations for the new ``amd-pstate`` module w 478 479 root@hr-test1:/home/ray# cpupower frequency-i 480 analyzing CPU 0: 481 driver: amd-pstate 482 CPUs which run at the same hardware frequen 483 CPUs which need to have their frequency coo 484 maximum transition latency: 131 us 485 hardware limits: 400 MHz - 4.68 GHz 486 available cpufreq governors: ondemand conse 487 current policy: frequency should be within 488 The governor "schedutil" ma 489 within this range. 490 current CPU frequency: Unable to call hardw 491 current CPU frequency: 4.02 GHz (asserted b 492 boost state support: 493 Supported: yes 494 Active: yes 495 AMD PSTATE Highest Performance: 166. Maxi 496 AMD PSTATE Nominal Performance: 117. Nomi 497 AMD PSTATE Lowest Non-linear Performance: 498 AMD PSTATE Lowest Performance: 15. Lowest 499 500 501 Diagnostics and Tuning 502 ======================= 503 504 Trace Events 505 -------------- 506 507 There are two static trace events that can be 508 diagnostics. One of them is the ``cpu_frequenc 509 by ``CPUFreq``, and the other one is the ``amd 510 specific to ``amd-pstate``. The following seq 511 be used to enable them and see their output (i 512 configured to support event tracing). :: 513 514 root@hr-test1:/home/ray# cd /sys/kernel/traci 515 root@hr-test1:/sys/kernel/tracing# echo 1 > e 516 root@hr-test1:/sys/kernel/tracing# cat trace 517 # tracer: nop 518 # 519 # entries-in-buffer/entries-written: 47827/42 520 # 521 # _-----=> irq 522 # / _----=> nee 523 # | / _---=> har 524 # || / _--=> pre 525 # ||| / dela 526 # TASK-PID CPU# |||| TIMESTA 527 # | | | |||| | 528 <idle>-0 [015] dN... 4995.979 529 <idle>-0 [007] d.h.. 4995.979 530 cat-2161 [000] d.... 4995.980 531 sshd-2125 [004] d.s.. 4995.980 532 <idle>-0 [007] d.s.. 4995.980 533 <idle>-0 [003] d.s.. 4995.980 534 <idle>-0 [011] d.s.. 4995.980 535 536 The ``cpu_frequency`` trace event will be trig 537 governor (for the policies it is attached to), 538 policies with other scaling governors). 539 540 541 Tracer Tool 542 ------------- 543 544 ``amd_pstate_tracer.py`` can record and parse 545 generate performance plots. This utility can b 546 performance of ``amd-pstate`` driver. The trac 547 pstate tracer. 548 549 Tracer tool located in ``linux/tools/power/x86 550 used in two ways. If trace file is available, 551 with command :: 552 553 ./amd_pstate_trace.py [-c cpus] -t <trace_fil 554 555 Or generate trace file with root privilege, th 556 557 sudo ./amd_pstate_trace.py [-c cpus] -n <test 558 559 The test result can be found in ``results/test 560 about part of the output. :: 561 562 common_cpu common_secs common_usecs min_pe 563 CPU_005 712 116384 39 564 CPU_006 712 116408 39 565 566 Unit Tests for amd-pstate 567 ------------------------- 568 569 ``amd-pstate-ut`` is a test module for testing 570 571 * It can help all users to verify their proce 572 573 * Kernel can have a basic function test to av 574 575 * We can introduce more functional or perform 576 577 1. Test case descriptions 578 579 1). Basic tests 580 581 Test prerequisite and basic functions 582 583 +---------+--------------------------- 584 | Index | Functions 585 +=========+=========================== 586 | 1 | amd_pstate_ut_acpi_cpc_val 587 | | 588 | | 589 +---------+--------------------------- 590 | 2 | amd_pstate_ut_check_enable 591 | | 592 | | 593 | | 594 | | 595 | | 596 +---------+--------------------------- 597 | 3 | amd_pstate_ut_check_perf 598 | | 599 +---------+--------------------------- 600 | 4 | amd_pstate_ut_check_freq 601 | | 602 | | 603 | | 604 | | 605 +---------+--------------------------- 606 607 2). Tbench test 608 609 Test and monitor the cpu changes when 610 These changes include desire performan 611 The specified governor is ondemand or 612 Tbench can also be tested on the ``acp 613 614 3). Gitsource test 615 616 Test and monitor the cpu changes when 617 These changes include desire performan 618 The specified governor is ondemand or 619 Gitsource can also be tested on the `` 620 621 #. How to execute the tests 622 623 We use test module in the kselftest framewo 624 We create ``amd-pstate-ut`` module and tie 625 details refer to Linux Kernel Selftests [4] 626 627 1). Build 628 629 + open the :c:macro:`CONFIG_X86_AMD_PS 630 + set the :c:macro:`CONFIG_X86_AMD_PST 631 + make project 632 + make selftest :: 633 634 $ cd linux 635 $ make -C tools/testing/selftests 636 637 + make perf :: 638 639 $ cd tools/perf/ 640 $ make 641 642 643 2). Installation & Steps :: 644 645 $ make -C tools/testing/selftests inst 646 $ cp tools/perf/perf /usr/bin/perf 647 $ sudo ./kselftest/run_kselftest.sh -c 648 649 3). Specified test case :: 650 651 $ cd ~/kselftest/amd-pstate 652 $ sudo ./run.sh -t basic 653 $ sudo ./run.sh -t tbench 654 $ sudo ./run.sh -t tbench -m acpi-cpuf 655 $ sudo ./run.sh -t gitsource 656 $ sudo ./run.sh -t gitsource -m acpi-c 657 $ ./run.sh --help 658 ./run.sh: illegal option -- - 659 Usage: ./run.sh [OPTION...] 660 [-h <help>] 661 [-o <output-file-for-dump>] 662 [-c <all: All testing, 663 basic: Basic testing, 664 tbench: Tbench testing, 665 gitsource: Gitsource test 666 [-t <tbench time limit>] 667 [-p <tbench process number>] 668 [-l <loop times for tbench>] 669 [-i <amd tracer interval>] 670 [-m <comparative test: acpi-cp 671 672 673 4). Results 674 675 + basic 676 677 When you finish test, you will get th 678 679 $ dmesg | grep "amd_pstate_ut" | tee 680 [12977.570663] amd_pstate_ut: 1 a 681 [12977.570673] amd_pstate_ut: 2 a 682 [12977.571207] amd_pstate_ut: 3 a 683 [12977.571212] amd_pstate_ut: 4 a 684 685 + tbench 686 687 When you finish test, you will get se 688 The selftest.tbench.csv file contains 689 The png images shows the performance, 690 Open selftest.tbench.csv : 691 692 +------------------------------------ 693 + Governor 694 +------------------------------------ 695 + Unit 696 +==================================== 697 + amd-pstate-ondemand 698 +------------------------------------ 699 + amd-pstate-ondemand 700 +------------------------------------ 701 + amd-pstate-ondemand 702 +------------------------------------ 703 + amd-pstate-ondemand 704 +------------------------------------ 705 + amd-pstate-schedutil 706 +------------------------------------ 707 + amd-pstate-schedutil 708 +------------------------------------ 709 + amd-pstate-schedutil 710 +------------------------------------ 711 + amd-pstate-schedutil 712 +------------------------------------ 713 + acpi-cpufreq-ondemand 714 +------------------------------------ 715 + acpi-cpufreq-ondemand 716 +------------------------------------ 717 + acpi-cpufreq-ondemand 718 +------------------------------------ 719 + acpi-cpufreq-ondemand 720 +------------------------------------ 721 + acpi-cpufreq-schedutil 722 +------------------------------------ 723 + acpi-cpufreq-schedutil 724 +------------------------------------ 725 + acpi-cpufreq-schedutil 726 +------------------------------------ 727 + acpi-cpufreq-schedutil 728 +------------------------------------ 729 + acpi-cpufreq-ondemand VS acpi-cpufr 730 +------------------------------------ 731 + amd-pstate-ondemand VS amd-pstate-s 732 +------------------------------------ 733 + acpi-cpufreq-ondemand VS amd-pstate 734 +------------------------------------ 735 + acpi-cpufreq-schedutil VS amd-pstat 736 +------------------------------------ 737 738 + gitsource 739 740 When you finish test, you will get se 741 The selftest.gitsource.csv file conta 742 The png images shows the performance, 743 Open selftest.gitsource.csv : 744 745 +------------------------------------ 746 + Governor 747 +------------------------------------ 748 + Unit 749 +==================================== 750 + amd-pstate-ondemand 751 +------------------------------------ 752 + amd-pstate-ondemand 753 +------------------------------------ 754 + amd-pstate-ondemand 755 +------------------------------------ 756 + amd-pstate-ondemand 757 +------------------------------------ 758 + amd-pstate-schedutil 759 +------------------------------------ 760 + amd-pstate-schedutil 761 +------------------------------------ 762 + amd-pstate-schedutil 763 +------------------------------------ 764 + amd-pstate-schedutil 765 +------------------------------------ 766 + acpi-cpufreq-ondemand 767 +------------------------------------ 768 + acpi-cpufreq-ondemand 769 +------------------------------------ 770 + acpi-cpufreq-ondemand 771 +------------------------------------ 772 + acpi-cpufreq-ondemand 773 +------------------------------------ 774 + acpi-cpufreq-schedutil 775 +------------------------------------ 776 + acpi-cpufreq-schedutil 777 +------------------------------------ 778 + acpi-cpufreq-schedutil 779 +------------------------------------ 780 + acpi-cpufreq-schedutil 781 +------------------------------------ 782 + acpi-cpufreq-ondemand VS acpi-cpufr 783 +------------------------------------ 784 + amd-pstate-ondemand VS amd-pstate-s 785 +------------------------------------ 786 + acpi-cpufreq-ondemand VS amd-pstate 787 +------------------------------------ 788 + acpi-cpufreq-schedutil VS amd-pstat 789 +------------------------------------ 790 791 Reference 792 =========== 793 794 .. [1] AMD64 Architecture Programmer's Manual 795 https://www.amd.com/system/files/TechDo 796 797 .. [2] Advanced Configuration and Power Interf 798 https://uefi.org/sites/default/files/re 799 800 .. [3] Processor Programming Reference (PPR) f 801 https://www.amd.com/system/files/TechDo 802 803 .. [4] Linux Kernel Selftests, 804 https://www.kernel.org/doc/html/latest/
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.