1 ============================================== 1 ======================================================
2 Confidential Computing in Linux for x86 virtua 2 Confidential Computing in Linux for x86 virtualization
3 ============================================== 3 ======================================================
4 4
5 .. contents:: :local: 5 .. contents:: :local:
6 6
7 By: Elena Reshetova <elena.reshetova@intel.com> !! 7 By: Elena Reshetova <elena.reshetova@intel.com> and Carlos Bilbao <carlos.bilbao@amd.com>
8 8
9 Motivation 9 Motivation
10 ========== 10 ==========
11 11
12 Kernel developers working on confidential comp 12 Kernel developers working on confidential computing for virtualized
13 environments in x86 operate under a set of ass 13 environments in x86 operate under a set of assumptions regarding the Linux
14 kernel threat model that differ from the tradi 14 kernel threat model that differ from the traditional view. Historically,
15 the Linux threat model acknowledges attackers 15 the Linux threat model acknowledges attackers residing in userspace, as
16 well as a limited set of external attackers th 16 well as a limited set of external attackers that are able to interact with
17 the kernel through various networking or limit 17 the kernel through various networking or limited HW-specific exposed
18 interfaces (USB, thunderbolt). The goal of thi 18 interfaces (USB, thunderbolt). The goal of this document is to explain
19 additional attack vectors that arise in the co 19 additional attack vectors that arise in the confidential computing space
20 and discuss the proposed protection mechanisms 20 and discuss the proposed protection mechanisms for the Linux kernel.
21 21
22 Overview and terminology 22 Overview and terminology
23 ======================== 23 ========================
24 24
25 Confidential Computing (CoCo) is a broad term 25 Confidential Computing (CoCo) is a broad term covering a wide range of
26 security technologies that aim to protect the 26 security technologies that aim to protect the confidentiality and integrity
27 of data in use (vs. data at rest or data in tr 27 of data in use (vs. data at rest or data in transit). At its core, CoCo
28 solutions provide a Trusted Execution Environm 28 solutions provide a Trusted Execution Environment (TEE), where secure data
29 processing can be performed and, as a result, 29 processing can be performed and, as a result, they are typically further
30 classified into different subtypes depending o 30 classified into different subtypes depending on the SW that is intended
31 to be run in TEE. This document focuses on a s 31 to be run in TEE. This document focuses on a subclass of CoCo technologies
32 that are targeting virtualized environments an 32 that are targeting virtualized environments and allow running Virtual
33 Machines (VM) inside TEE. From now on in this 33 Machines (VM) inside TEE. From now on in this document will be referring
34 to this subclass of CoCo as 'Confidential Comp 34 to this subclass of CoCo as 'Confidential Computing (CoCo) for the
35 virtualized environments (VE)'. 35 virtualized environments (VE)'.
36 36
37 CoCo, in the virtualization context, refers to 37 CoCo, in the virtualization context, refers to a set of HW and/or SW
38 technologies that allow for stronger security 38 technologies that allow for stronger security guarantees for the SW running
39 inside a CoCo VM. Namely, confidential computi 39 inside a CoCo VM. Namely, confidential computing allows its users to
40 confirm the trustworthiness of all SW pieces t 40 confirm the trustworthiness of all SW pieces to include in its reduced
41 Trusted Computing Base (TCB) given its ability 41 Trusted Computing Base (TCB) given its ability to attest the state of these
42 trusted components. 42 trusted components.
43 43
44 While the concrete implementation details diff 44 While the concrete implementation details differ between technologies, all
45 available mechanisms aim to provide increased 45 available mechanisms aim to provide increased confidentiality and
46 integrity for the VM's guest memory and execut 46 integrity for the VM's guest memory and execution state (vCPU registers),
47 more tightly controlled guest interrupt inject 47 more tightly controlled guest interrupt injection, as well as some
48 additional mechanisms to control guest-host pa 48 additional mechanisms to control guest-host page mapping. More details on
49 the x86-specific solutions can be found in 49 the x86-specific solutions can be found in
50 :doc:`Intel Trust Domain Extensions (TDX) </ar 50 :doc:`Intel Trust Domain Extensions (TDX) </arch/x86/tdx>` and
51 `AMD Memory Encryption <https://www.amd.com/sy 51 `AMD Memory Encryption <https://www.amd.com/system/files/techdocs/sev-snp-strengthening-vm-isolation-with-integrity-protection-and-more.pdf>`_.
52 52
53 The basic CoCo guest layout includes the host, 53 The basic CoCo guest layout includes the host, guest, the interfaces that
54 communicate guest and host, a platform capable 54 communicate guest and host, a platform capable of supporting CoCo VMs, and
55 a trusted intermediary between the guest VM an 55 a trusted intermediary between the guest VM and the underlying platform
56 that acts as a security manager. The host-side 56 that acts as a security manager. The host-side virtual machine monitor
57 (VMM) typically consists of a subset of tradit 57 (VMM) typically consists of a subset of traditional VMM features and
58 is still in charge of the guest lifecycle, i.e 58 is still in charge of the guest lifecycle, i.e. create or destroy a CoCo
59 VM, manage its access to system resources, etc 59 VM, manage its access to system resources, etc. However, since it
60 typically stays out of CoCo VM TCB, its access 60 typically stays out of CoCo VM TCB, its access is limited to preserve the
61 security objectives. 61 security objectives.
62 62
63 In the following diagram, the "<--->" lines re 63 In the following diagram, the "<--->" lines represent bi-directional
64 communication channels or interfaces between t 64 communication channels or interfaces between the CoCo security manager and
65 the rest of the components (data flow for gues 65 the rest of the components (data flow for guest, host, hardware) ::
66 66
67 +-------------------+ +-------------- 67 +-------------------+ +-----------------------+
68 | CoCo guest VM |<---->| 68 | CoCo guest VM |<---->| |
69 +-------------------+ | 69 +-------------------+ | |
70 | Interfaces | | CoCo security 70 | Interfaces | | CoCo security manager |
71 +-------------------+ | 71 +-------------------+ | |
72 | Host VMM |<---->| 72 | Host VMM |<---->| |
73 +-------------------+ | 73 +-------------------+ | |
74 | 74 | |
75 +--------------------+ | 75 +--------------------+ | |
76 | CoCo platform |<--->| 76 | CoCo platform |<--->| |
77 +--------------------+ +-------------- 77 +--------------------+ +-----------------------+
78 78
79 The specific details of the CoCo security mana 79 The specific details of the CoCo security manager vastly diverge between
80 technologies. For example, in some cases, it w 80 technologies. For example, in some cases, it will be implemented in HW
81 while in others it may be pure SW. 81 while in others it may be pure SW.
82 82
83 Existing Linux kernel threat model 83 Existing Linux kernel threat model
84 ================================== 84 ==================================
85 85
86 The overall components of the current Linux ke 86 The overall components of the current Linux kernel threat model are::
87 87
88 +-----------------------+ +--------- 88 +-----------------------+ +-------------------+
89 | |<---->| Userspac 89 | |<---->| Userspace |
90 | | +--------- 90 | | +-------------------+
91 | External attack | | Inter 91 | External attack | | Interfaces |
92 | vectors | +--------- 92 | vectors | +-------------------+
93 | |<---->| Linux Ke 93 | |<---->| Linux Kernel |
94 | | +--------- 94 | | +-------------------+
95 +-----------------------+ +--------- 95 +-----------------------+ +-------------------+
96 | Bootload 96 | Bootloader/BIOS |
97 +--------- 97 +-------------------+
98 +--------- 98 +-------------------+
99 | HW platf 99 | HW platform |
100 +--------- 100 +-------------------+
101 101
102 There is also communication between the bootlo 102 There is also communication between the bootloader and the kernel during
103 the boot process, but this diagram does not re 103 the boot process, but this diagram does not represent it explicitly. The
104 "Interfaces" box represents the various interf 104 "Interfaces" box represents the various interfaces that allow
105 communication between kernel and userspace. Th 105 communication between kernel and userspace. This includes system calls,
106 kernel APIs, device drivers, etc. 106 kernel APIs, device drivers, etc.
107 107
108 The existing Linux kernel threat model typical 108 The existing Linux kernel threat model typically assumes execution on a
109 trusted HW platform with all of the firmware a 109 trusted HW platform with all of the firmware and bootloaders included on
110 its TCB. The primary attacker resides in the u 110 its TCB. The primary attacker resides in the userspace, and all of the data
111 coming from there is generally considered untr 111 coming from there is generally considered untrusted, unless userspace is
112 privileged enough to perform trusted actions. 112 privileged enough to perform trusted actions. In addition, external
113 attackers are typically considered, including 113 attackers are typically considered, including those with access to enabled
114 external networks (e.g. Ethernet, Wireless, Bl 114 external networks (e.g. Ethernet, Wireless, Bluetooth), exposed hardware
115 interfaces (e.g. USB, Thunderbolt), and the ab 115 interfaces (e.g. USB, Thunderbolt), and the ability to modify the contents
116 of disks offline. 116 of disks offline.
117 117
118 Regarding external attack vectors, it is inter 118 Regarding external attack vectors, it is interesting to note that in most
119 cases external attackers will try to exploit v 119 cases external attackers will try to exploit vulnerabilities in userspace
120 first, but that it is possible for an attacker 120 first, but that it is possible for an attacker to directly target the
121 kernel; particularly if the host has physical 121 kernel; particularly if the host has physical access. Examples of direct
122 kernel attacks include the vulnerabilities CVE 122 kernel attacks include the vulnerabilities CVE-2019-19524, CVE-2022-0435
123 and CVE-2020-24490. 123 and CVE-2020-24490.
124 124
125 Confidential Computing threat model and its se 125 Confidential Computing threat model and its security objectives
126 ============================================== 126 ===============================================================
127 127
128 Confidential Computing adds a new type of atta 128 Confidential Computing adds a new type of attacker to the above list: a
129 potentially misbehaving host (which can also i 129 potentially misbehaving host (which can also include some part of a
130 traditional VMM or all of it), which is typica 130 traditional VMM or all of it), which is typically placed outside of the
131 CoCo VM TCB due to its large SW attack surface 131 CoCo VM TCB due to its large SW attack surface. It is important to note
132 that this doesn’t imply that the host or VMM 132 that this doesn’t imply that the host or VMM are intentionally
133 malicious, but that there exists a security va 133 malicious, but that there exists a security value in having a small CoCo
134 VM TCB. This new type of adversary may be view 134 VM TCB. This new type of adversary may be viewed as a more powerful type
135 of external attacker, as it resides locally on 135 of external attacker, as it resides locally on the same physical machine
136 (in contrast to a remote network attacker) and 136 (in contrast to a remote network attacker) and has control over the guest
137 kernel communication with most of the HW:: 137 kernel communication with most of the HW::
138 138
139 +------------ 139 +------------------------+
140 | CoCo gue 140 | CoCo guest VM |
141 +-----------------------+ | +--------- 141 +-----------------------+ | +-------------------+ |
142 | |<--->| | Userspac 142 | |<--->| | Userspace | |
143 | | | +--------- 143 | | | +-------------------+ |
144 | External attack | | | Inter 144 | External attack | | | Interfaces | |
145 | vectors | | +--------- 145 | vectors | | +-------------------+ |
146 | |<--->| | Linux Ke 146 | |<--->| | Linux Kernel | |
147 | | | +--------- 147 | | | +-------------------+ |
148 +-----------------------+ | +--------- 148 +-----------------------+ | +-------------------+ |
149 | | Bootload 149 | | Bootloader/BIOS | |
150 +-----------------------+ | +--------- 150 +-----------------------+ | +-------------------+ |
151 | |<--->+------------ 151 | |<--->+------------------------+
152 | | | Interf 152 | | | Interfaces |
153 | | +------------ 153 | | +------------------------+
154 | CoCo security |<--->| Host/Host-s 154 | CoCo security |<--->| Host/Host-side VMM |
155 | manager | +------------ 155 | manager | +------------------------+
156 | | +------------ 156 | | +------------------------+
157 | |<--->| CoCo plat 157 | |<--->| CoCo platform |
158 +-----------------------+ +------------ 158 +-----------------------+ +------------------------+
159 159
160 While traditionally the host has unlimited acc 160 While traditionally the host has unlimited access to guest data and can
161 leverage this access to attack the guest, the 161 leverage this access to attack the guest, the CoCo systems mitigate such
162 attacks by adding security features like guest 162 attacks by adding security features like guest data confidentiality and
163 integrity protection. This threat model assume 163 integrity protection. This threat model assumes that those features are
164 available and intact. 164 available and intact.
165 165
166 The **Linux kernel CoCo VM security objectives 166 The **Linux kernel CoCo VM security objectives** can be summarized as follows:
167 167
168 1. Preserve the confidentiality and integrity 168 1. Preserve the confidentiality and integrity of CoCo guest's private
169 memory and registers. 169 memory and registers.
170 170
171 2. Prevent privileged escalation from a host i 171 2. Prevent privileged escalation from a host into a CoCo guest Linux kernel.
172 While it is true that the host (and host-side 172 While it is true that the host (and host-side VMM) requires some level of
173 privilege to create, destroy, or pause the gue 173 privilege to create, destroy, or pause the guest, part of the goal of
174 preventing privileged escalation is to ensure 174 preventing privileged escalation is to ensure that these operations do not
175 provide a pathway for attackers to gain access 175 provide a pathway for attackers to gain access to the guest's kernel.
176 176
177 The above security objectives result in two pr 177 The above security objectives result in two primary **Linux kernel CoCo
178 VM assets**: 178 VM assets**:
179 179
180 1. Guest kernel execution context. 180 1. Guest kernel execution context.
181 2. Guest kernel private memory. 181 2. Guest kernel private memory.
182 182
183 The host retains full control over the CoCo gu 183 The host retains full control over the CoCo guest resources, and can deny
184 access to them at any time. Examples of resour 184 access to them at any time. Examples of resources include CPU time, memory
185 that the guest can consume, network bandwidth, 185 that the guest can consume, network bandwidth, etc. Because of this, the
186 host Denial of Service (DoS) attacks against C 186 host Denial of Service (DoS) attacks against CoCo guests are beyond the
187 scope of this threat model. 187 scope of this threat model.
188 188
189 The **Linux CoCo VM attack surface** is any in 189 The **Linux CoCo VM attack surface** is any interface exposed from a CoCo
190 guest Linux kernel towards an untrusted host t 190 guest Linux kernel towards an untrusted host that is not covered by the
191 CoCo technology SW/HW protection. This include 191 CoCo technology SW/HW protection. This includes any possible
192 side-channels, as well as transient execution 192 side-channels, as well as transient execution side channels. Examples of
193 explicit (not side-channel) interfaces include 193 explicit (not side-channel) interfaces include accesses to port I/O, MMIO
194 and DMA interfaces, access to PCI configuratio 194 and DMA interfaces, access to PCI configuration space, VMM-specific
195 hypercalls (towards Host-side VMM), access to 195 hypercalls (towards Host-side VMM), access to shared memory pages,
196 interrupts allowed to be injected into the gue 196 interrupts allowed to be injected into the guest kernel by the host, as
197 well as CoCo technology-specific hypercalls, i 197 well as CoCo technology-specific hypercalls, if present. Additionally, the
198 host in a CoCo system typically controls the p 198 host in a CoCo system typically controls the process of creating a CoCo
199 guest: it has a method to load into a guest th 199 guest: it has a method to load into a guest the firmware and bootloader
200 images, the kernel image together with the ker 200 images, the kernel image together with the kernel command line. All of this
201 data should also be considered untrusted until 201 data should also be considered untrusted until its integrity and
202 authenticity is established via attestation. 202 authenticity is established via attestation.
203 203
204 The table below shows a threat matrix for the 204 The table below shows a threat matrix for the CoCo guest Linux kernel but
205 does not discuss potential mitigation strategi 205 does not discuss potential mitigation strategies. The matrix refers to
206 CoCo-specific versions of the guest, host and 206 CoCo-specific versions of the guest, host and platform.
207 207
208 .. list-table:: CoCo Linux guest kernel threat 208 .. list-table:: CoCo Linux guest kernel threat matrix
209 :widths: auto 209 :widths: auto
210 :align: center 210 :align: center
211 :header-rows: 1 211 :header-rows: 1
212 212
213 * - Threat name 213 * - Threat name
214 - Threat description 214 - Threat description
215 215
216 * - Guest malicious configuration 216 * - Guest malicious configuration
217 - A misbehaving host modifies one of the 217 - A misbehaving host modifies one of the following guest's
218 configuration: 218 configuration:
219 219
220 1. Guest firmware or bootloader 220 1. Guest firmware or bootloader
221 221
222 2. Guest kernel or module binaries 222 2. Guest kernel or module binaries
223 223
224 3. Guest command line parameters 224 3. Guest command line parameters
225 225
226 This allows the host to break the integ 226 This allows the host to break the integrity of the code running
227 inside a CoCo guest, and violates the C 227 inside a CoCo guest, and violates the CoCo security objectives.
228 228
229 * - CoCo guest data attacks 229 * - CoCo guest data attacks
230 - A misbehaving host retains full control 230 - A misbehaving host retains full control of the CoCo guest's data
231 in-transit between the guest and the ho 231 in-transit between the guest and the host-managed physical or
232 virtual devices. This allows any attack 232 virtual devices. This allows any attack against confidentiality,
233 integrity or freshness of such data. 233 integrity or freshness of such data.
234 234
235 * - Malformed runtime input 235 * - Malformed runtime input
236 - A misbehaving host injects malformed in 236 - A misbehaving host injects malformed input via any communication
237 interface used by the guest's kernel co 237 interface used by the guest's kernel code. If the code is not
238 prepared to handle this input correctly 238 prepared to handle this input correctly, this can result in a host
239 --> guest kernel privilege escalation. 239 --> guest kernel privilege escalation. This includes traditional
240 side-channel and/or transient execution 240 side-channel and/or transient execution attack vectors.
241 241
242 * - Malicious runtime input 242 * - Malicious runtime input
243 - A misbehaving host injects a specific i 243 - A misbehaving host injects a specific input value via any
244 communication interface used by the gue 244 communication interface used by the guest's kernel code. The
245 difference with the previous attack vec 245 difference with the previous attack vector (malformed runtime input)
246 is that this input is not malformed, bu 246 is that this input is not malformed, but its value is crafted to
247 impact the guest's kernel security. Exa 247 impact the guest's kernel security. Examples of such inputs include
248 providing a malicious time to the guest 248 providing a malicious time to the guest or the entropy to the guest
249 random number generator. Additionally, 249 random number generator. Additionally, the timing of such events can
250 be an attack vector on its own, if it r 250 be an attack vector on its own, if it results in a particular guest
251 kernel action (i.e. processing of a hos 251 kernel action (i.e. processing of a host-injected interrupt).
252 resistant to supplied host input. 252 resistant to supplied host input.
253 253
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.