1 .. SPDX-License-Identifier: GPL-2.0 2 3 ============ 4 x86 Topology 5 ============ 6 7 This documents and clarifies the main aspects of x86 topology modelling and 8 representation in the kernel. Update/change when doing changes to the 9 respective code. 10 11 The architecture-agnostic topology definitions are in 12 Documentation/admin-guide/cputopology.rst. This file holds x86-specific 13 differences/specialities which must not necessarily apply to the generic 14 definitions. Thus, the way to read up on Linux topology on x86 is to start 15 with the generic one and look at this one in parallel for the x86 specifics. 16 17 Needless to say, code should use the generic functions - this file is *only* 18 here to *document* the inner workings of x86 topology. 19 20 Started by Thomas Gleixner <tglx@linutronix.de> and Borislav Petkov <bp@alien8.de>. 21 22 The main aim of the topology facilities is to present adequate interfaces to 23 code which needs to know/query/use the structure of the running system wrt 24 threads, cores, packages, etc. 25 26 The kernel does not care about the concept of physical sockets because a 27 socket has no relevance to software. It's an electromechanical component. In 28 the past a socket always contained a single package (see below), but with the 29 advent of Multi Chip Modules (MCM) a socket can hold more than one package. So 30 there might be still references to sockets in the code, but they are of 31 historical nature and should be cleaned up. 32 33 The topology of a system is described in the units of: 34 35 - packages 36 - cores 37 - threads 38 39 Package 40 ======= 41 Packages contain a number of cores plus shared resources, e.g. DRAM 42 controller, shared caches etc. 43 44 Modern systems may also use the term 'Die' for package. 45 46 AMD nomenclature for package is 'Node'. 47 48 Package-related topology information in the kernel: 49 50 - topology_num_threads_per_package() 51 52 The number of threads in a package. 53 54 - topology_num_cores_per_package() 55 56 The number of cores in a package. 57 58 - topology_max_dies_per_package() 59 60 The maximum number of dies in a package. 61 62 - cpuinfo_x86.topo.die_id: 63 64 The physical ID of the die. 65 66 - cpuinfo_x86.topo.pkg_id: 67 68 The physical ID of the package. This information is retrieved via CPUID 69 and deduced from the APIC IDs of the cores in the package. 70 71 Modern systems use this value for the socket. There may be multiple 72 packages within a socket. This value may differ from topo.die_id. 73 74 - cpuinfo_x86.topo.logical_pkg_id: 75 76 The logical ID of the package. As we do not trust BIOSes to enumerate the 77 packages in a consistent way, we introduced the concept of logical package 78 ID so we can sanely calculate the number of maximum possible packages in 79 the system and have the packages enumerated linearly. 80 81 - topology_max_packages(): 82 83 The maximum possible number of packages in the system. Helpful for per 84 package facilities to preallocate per package information. 85 86 - cpuinfo_x86.topo.llc_id: 87 88 - On Intel, the first APIC ID of the list of CPUs sharing the Last Level 89 Cache 90 91 - On AMD, the Node ID or Core Complex ID containing the Last Level 92 Cache. In general, it is a number identifying an LLC uniquely on the 93 system. 94 95 Cores 96 ===== 97 A core consists of 1 or more threads. It does not matter whether the threads 98 are SMT- or CMT-type threads. 99 100 AMDs nomenclature for a CMT core is "Compute Unit". The kernel always uses 101 "core". 102 103 Threads 104 ======= 105 A thread is a single scheduling unit. It's the equivalent to a logical Linux 106 CPU. 107 108 AMDs nomenclature for CMT threads is "Compute Unit Core". The kernel always 109 uses "thread". 110 111 Thread-related topology information in the kernel: 112 113 - topology_core_cpumask(): 114 115 The cpumask contains all online threads in the package to which a thread 116 belongs. 117 118 The number of online threads is also printed in /proc/cpuinfo "siblings." 119 120 - topology_sibling_cpumask(): 121 122 The cpumask contains all online threads in the core to which a thread 123 belongs. 124 125 - topology_logical_package_id(): 126 127 The logical package ID to which a thread belongs. 128 129 - topology_physical_package_id(): 130 131 The physical package ID to which a thread belongs. 132 133 - topology_core_id(); 134 135 The ID of the core to which a thread belongs. It is also printed in /proc/cpuinfo 136 "core_id." 137 138 139 140 System topology examples 141 ======================== 142 143 .. note:: 144 The alternative Linux CPU enumeration depends on how the BIOS enumerates the 145 threads. Many BIOSes enumerate all threads 0 first and then all threads 1. 146 That has the "advantage" that the logical Linux CPU numbers of threads 0 stay 147 the same whether threads are enabled or not. That's merely an implementation 148 detail and has no practical impact. 149 150 1) Single Package, Single Core:: 151 152 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 153 154 2) Single Package, Dual Core 155 156 a) One thread per core:: 157 158 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 159 -> [core 1] -> [thread 0] -> Linux CPU 1 160 161 b) Two threads per core:: 162 163 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 164 -> [thread 1] -> Linux CPU 1 165 -> [core 1] -> [thread 0] -> Linux CPU 2 166 -> [thread 1] -> Linux CPU 3 167 168 Alternative enumeration:: 169 170 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 171 -> [thread 1] -> Linux CPU 2 172 -> [core 1] -> [thread 0] -> Linux CPU 1 173 -> [thread 1] -> Linux CPU 3 174 175 AMD nomenclature for CMT systems:: 176 177 [node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0 178 -> [Compute Unit Core 1] -> Linux CPU 1 179 -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2 180 -> [Compute Unit Core 1] -> Linux CPU 3 181 182 4) Dual Package, Dual Core 183 184 a) One thread per core:: 185 186 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 187 -> [core 1] -> [thread 0] -> Linux CPU 1 188 189 [package 1] -> [core 0] -> [thread 0] -> Linux CPU 2 190 -> [core 1] -> [thread 0] -> Linux CPU 3 191 192 b) Two threads per core:: 193 194 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 195 -> [thread 1] -> Linux CPU 1 196 -> [core 1] -> [thread 0] -> Linux CPU 2 197 -> [thread 1] -> Linux CPU 3 198 199 [package 1] -> [core 0] -> [thread 0] -> Linux CPU 4 200 -> [thread 1] -> Linux CPU 5 201 -> [core 1] -> [thread 0] -> Linux CPU 6 202 -> [thread 1] -> Linux CPU 7 203 204 Alternative enumeration:: 205 206 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 207 -> [thread 1] -> Linux CPU 4 208 -> [core 1] -> [thread 0] -> Linux CPU 1 209 -> [thread 1] -> Linux CPU 5 210 211 [package 1] -> [core 0] -> [thread 0] -> Linux CPU 2 212 -> [thread 1] -> Linux CPU 6 213 -> [core 1] -> [thread 0] -> Linux CPU 3 214 -> [thread 1] -> Linux CPU 7 215 216 AMD nomenclature for CMT systems:: 217 218 [node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0 219 -> [Compute Unit Core 1] -> Linux CPU 1 220 -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2 221 -> [Compute Unit Core 1] -> Linux CPU 3 222 223 [node 1] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 4 224 -> [Compute Unit Core 1] -> Linux CPU 5 225 -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 6 226 -> [Compute Unit Core 1] -> Linux CPU 7
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.