1 .. SPDX-License-Identifier: GPL-2.0 1 .. SPDX-License-Identifier: GPL-2.0
2 2
3 ==================== 3 ====================
4 APEI Error INJection 4 APEI Error INJection
5 ==================== 5 ====================
6 6
7 EINJ provides a hardware error injection mecha 7 EINJ provides a hardware error injection mechanism. It is very useful
8 for debugging and testing APEI and RAS feature 8 for debugging and testing APEI and RAS features in general.
9 9
10 You need to check whether your BIOS supports E 10 You need to check whether your BIOS supports EINJ first. For that, look
11 for early boot messages similar to this one:: 11 for early boot messages similar to this one::
12 12
13 ACPI: EINJ 0x000000007370A000 000150 (v01 IN 13 ACPI: EINJ 0x000000007370A000 000150 (v01 INTEL 00000001 INTL 00000001)
14 14
15 which shows that the BIOS is exposing an EINJ 15 which shows that the BIOS is exposing an EINJ table - it is the
16 mechanism through which the injection is done. 16 mechanism through which the injection is done.
17 17
18 Alternatively, look in /sys/firmware/acpi/tabl 18 Alternatively, look in /sys/firmware/acpi/tables for an "EINJ" file,
19 which is a different representation of the sam 19 which is a different representation of the same thing.
20 20
21 It doesn't necessarily mean that EINJ is not s 21 It doesn't necessarily mean that EINJ is not supported if those above
22 don't exist: before you give up, go into BIOS 22 don't exist: before you give up, go into BIOS setup to see if the BIOS
23 has an option to enable error injection. Look 23 has an option to enable error injection. Look for something called WHEA
24 or similar. Often, you need to enable an ACPI5 24 or similar. Often, you need to enable an ACPI5 support option prior, in
25 order to see the APEI,EINJ,... functionality s 25 order to see the APEI,EINJ,... functionality supported and exposed by
26 the BIOS menu. 26 the BIOS menu.
27 27
28 To use EINJ, make sure the following are optio 28 To use EINJ, make sure the following are options enabled in your kernel
29 configuration:: 29 configuration::
30 30
31 CONFIG_DEBUG_FS 31 CONFIG_DEBUG_FS
32 CONFIG_ACPI_APEI 32 CONFIG_ACPI_APEI
33 CONFIG_ACPI_APEI_EINJ 33 CONFIG_ACPI_APEI_EINJ
34 34
35 ...and to (optionally) enable CXL protocol err 35 ...and to (optionally) enable CXL protocol error injection set::
36 36
37 CONFIG_ACPI_APEI_EINJ_CXL 37 CONFIG_ACPI_APEI_EINJ_CXL
38 38
39 The EINJ user interface is in <debugfs mount p 39 The EINJ user interface is in <debugfs mount point>/apei/einj.
40 40
41 The following files belong to it: 41 The following files belong to it:
42 42
43 - available_error_type 43 - available_error_type
44 44
45 This file shows which error types are suppor 45 This file shows which error types are supported:
46 46
47 ================ ========================== 47 ================ ===================================
48 Error Type Value Error Description 48 Error Type Value Error Description
49 ================ ========================== 49 ================ ===================================
50 0x00000001 Processor Correctable 50 0x00000001 Processor Correctable
51 0x00000002 Processor Uncorrectable no 51 0x00000002 Processor Uncorrectable non-fatal
52 0x00000004 Processor Uncorrectable fa 52 0x00000004 Processor Uncorrectable fatal
53 0x00000008 Memory Correctable 53 0x00000008 Memory Correctable
54 0x00000010 Memory Uncorrectable non-f 54 0x00000010 Memory Uncorrectable non-fatal
55 0x00000020 Memory Uncorrectable fatal 55 0x00000020 Memory Uncorrectable fatal
56 0x00000040 PCI Express Correctable 56 0x00000040 PCI Express Correctable
57 0x00000080 PCI Express Uncorrectable 57 0x00000080 PCI Express Uncorrectable non-fatal
58 0x00000100 PCI Express Uncorrectable 58 0x00000100 PCI Express Uncorrectable fatal
59 0x00000200 Platform Correctable 59 0x00000200 Platform Correctable
60 0x00000400 Platform Uncorrectable non 60 0x00000400 Platform Uncorrectable non-fatal
61 0x00000800 Platform Uncorrectable fat 61 0x00000800 Platform Uncorrectable fatal
62 ================ ========================== 62 ================ ===================================
63 63
64 The format of the file contents are as above 64 The format of the file contents are as above, except present are only
65 the available error types. 65 the available error types.
66 66
67 - error_type 67 - error_type
68 68
69 Set the value of the error type being inject 69 Set the value of the error type being injected. Possible error types
70 are defined in the file available_error_type 70 are defined in the file available_error_type above.
71 71
72 - error_inject 72 - error_inject
73 73
74 Write any integer to this file to trigger th 74 Write any integer to this file to trigger the error injection. Make
75 sure you have specified all necessary error 75 sure you have specified all necessary error parameters, i.e. this
76 write should be the last step when injecting 76 write should be the last step when injecting errors.
77 77
78 - flags 78 - flags
79 79
80 Present for kernel versions 3.13 and above. 80 Present for kernel versions 3.13 and above. Used to specify which
81 of param{1..4} are valid and should be used 81 of param{1..4} are valid and should be used by the firmware during
82 injection. Value is a bitmask as specified i 82 injection. Value is a bitmask as specified in ACPI5.0 spec for the
83 SET_ERROR_TYPE_WITH_ADDRESS data structure: 83 SET_ERROR_TYPE_WITH_ADDRESS data structure:
84 84
85 Bit 0 85 Bit 0
86 Processor APIC field valid (see param3 b 86 Processor APIC field valid (see param3 below).
87 Bit 1 87 Bit 1
88 Memory address and mask valid (param1 an 88 Memory address and mask valid (param1 and param2).
89 Bit 2 89 Bit 2
90 PCIe (seg,bus,dev,fn) valid (see param4 90 PCIe (seg,bus,dev,fn) valid (see param4 below).
91 91
92 If set to zero, legacy behavior is mimicked 92 If set to zero, legacy behavior is mimicked where the type of
93 injection specifies just one bit set, and pa 93 injection specifies just one bit set, and param1 is multiplexed.
94 94
95 - param1 95 - param1
96 96
97 This file is used to set the first error par 97 This file is used to set the first error parameter value. Its effect
98 depends on the error type specified in error 98 depends on the error type specified in error_type. For example, if
99 error type is memory related type, the param 99 error type is memory related type, the param1 should be a valid
100 physical memory address. [Unless "flag" is s 100 physical memory address. [Unless "flag" is set - see above]
101 101
102 - param2 102 - param2
103 103
104 Same use as param1 above. For example, if er 104 Same use as param1 above. For example, if error type is of memory
105 related type, then param2 should be a physic 105 related type, then param2 should be a physical memory address mask.
106 Linux requires page or narrower granularity, 106 Linux requires page or narrower granularity, say, 0xfffffffffffff000.
107 107
108 - param3 108 - param3
109 109
110 Used when the 0x1 bit is set in "flags" to s 110 Used when the 0x1 bit is set in "flags" to specify the APIC id
111 111
112 - param4 112 - param4
113 Used when the 0x4 bit is set in "flags" to s 113 Used when the 0x4 bit is set in "flags" to specify target PCIe device
114 114
115 - notrigger 115 - notrigger
116 116
117 The error injection mechanism is a two-step 117 The error injection mechanism is a two-step process. First inject the
118 error, then perform some actions to trigger 118 error, then perform some actions to trigger it. Setting "notrigger"
119 to 1 skips the trigger phase, which *may* al 119 to 1 skips the trigger phase, which *may* allow the user to cause the
120 error in some other context by a simple acce 120 error in some other context by a simple access to the CPU, memory
121 location, or device that is the target of th 121 location, or device that is the target of the error injection. Whether
122 this actually works depends on what operatio 122 this actually works depends on what operations the BIOS actually
123 includes in the trigger phase. 123 includes in the trigger phase.
124 124
125 CXL error types are supported from ACPI 6.5 on 125 CXL error types are supported from ACPI 6.5 onwards (given a CXL port
126 is present). The EINJ user interface for CXL e 126 is present). The EINJ user interface for CXL error types is at
127 <debugfs mount point>/cxl. The following files 127 <debugfs mount point>/cxl. The following files belong to it:
128 128
129 - einj_types: 129 - einj_types:
130 130
131 Provides the same functionality as available 131 Provides the same functionality as available_error_types above, but
132 for CXL error types 132 for CXL error types
133 133
134 - $dport_dev/einj_inject: 134 - $dport_dev/einj_inject:
135 135
136 Injects a CXL error type into the CXL port r 136 Injects a CXL error type into the CXL port represented by $dport_dev,
137 where $dport_dev is the name of the CXL port 137 where $dport_dev is the name of the CXL port (usually a PCIe device name).
138 Error injections targeting a CXL 2.0+ port c 138 Error injections targeting a CXL 2.0+ port can use the legacy interface
139 under <debugfs mount point>/apei/einj, while 139 under <debugfs mount point>/apei/einj, while CXL 1.1/1.0 port injections
140 must use this file. 140 must use this file.
141 141
142 142
143 BIOS versions based on the ACPI 4.0 specificat 143 BIOS versions based on the ACPI 4.0 specification have limited options
144 in controlling where the errors are injected. 144 in controlling where the errors are injected. Your BIOS may support an
145 extension (enabled with the param_extension=1 145 extension (enabled with the param_extension=1 module parameter, or boot
146 command line einj.param_extension=1). This all 146 command line einj.param_extension=1). This allows the address and mask
147 for memory injections to be specified by the p 147 for memory injections to be specified by the param1 and param2 files in
148 apei/einj. 148 apei/einj.
149 149
150 BIOS versions based on the ACPI 5.0 specificat 150 BIOS versions based on the ACPI 5.0 specification have more control over
151 the target of the injection. For processor-rel 151 the target of the injection. For processor-related errors (type 0x1, 0x2
152 and 0x4), you can set flags to 0x3 (param3 for 152 and 0x4), you can set flags to 0x3 (param3 for bit 0, and param1 and
153 param2 for bit 1) so that you have more inform 153 param2 for bit 1) so that you have more information added to the error
154 signature being injected. The actual data pass 154 signature being injected. The actual data passed is this::
155 155
156 memory_address = param1; 156 memory_address = param1;
157 memory_address_range = param2; 157 memory_address_range = param2;
158 apicid = param3; 158 apicid = param3;
159 pcie_sbdf = param4; 159 pcie_sbdf = param4;
160 160
161 For memory errors (type 0x8, 0x10 and 0x20) th 161 For memory errors (type 0x8, 0x10 and 0x20) the address is set using
162 param1 with a mask in param2 (0x0 is equivalen 162 param1 with a mask in param2 (0x0 is equivalent to all ones). For PCI
163 express errors (type 0x40, 0x80 and 0x100) the 163 express errors (type 0x40, 0x80 and 0x100) the segment, bus, device and
164 function are specified using param1:: 164 function are specified using param1::
165 165
166 31 24 23 16 15 11 10 8 166 31 24 23 16 15 11 10 8 7 0
167 +------------------------------------- 167 +-------------------------------------------------+
168 | segment | bus | device | function 168 | segment | bus | device | function | reserved |
169 +------------------------------------- 169 +-------------------------------------------------+
170 170
171 Anyway, you get the idea, if there's doubt jus 171 Anyway, you get the idea, if there's doubt just take a look at the code
172 in drivers/acpi/apei/einj.c. 172 in drivers/acpi/apei/einj.c.
173 173
174 An ACPI 5.0 BIOS may also allow vendor-specifi 174 An ACPI 5.0 BIOS may also allow vendor-specific errors to be injected.
175 In this case a file named vendor will contain 175 In this case a file named vendor will contain identifying information
176 from the BIOS that hopefully will allow an app 176 from the BIOS that hopefully will allow an application wishing to use
177 the vendor-specific extension to tell that the 177 the vendor-specific extension to tell that they are running on a BIOS
178 that supports it. All vendor extensions have t 178 that supports it. All vendor extensions have the 0x80000000 bit set in
179 error_type. A file vendor_flags controls the i 179 error_type. A file vendor_flags controls the interpretation of param1
180 and param2 (1 = PROCESSOR, 2 = MEMORY, 4 = PCI 180 and param2 (1 = PROCESSOR, 2 = MEMORY, 4 = PCI). See your BIOS vendor
181 documentation for details (and expect changes 181 documentation for details (and expect changes to this API if vendors
182 creativity in using this feature expands beyon 182 creativity in using this feature expands beyond our expectations).
183 183
184 184
185 An error injection example:: 185 An error injection example::
186 186
187 # cd /sys/kernel/debug/apei/einj 187 # cd /sys/kernel/debug/apei/einj
188 # cat available_error_type # See 188 # cat available_error_type # See which errors can be injected
189 0x00000002 Processor Uncorrectable non-fa 189 0x00000002 Processor Uncorrectable non-fatal
190 0x00000008 Memory Correctable 190 0x00000008 Memory Correctable
191 0x00000010 Memory Uncorrectable non-fatal 191 0x00000010 Memory Uncorrectable non-fatal
192 # echo 0x12345000 > param1 # Set 192 # echo 0x12345000 > param1 # Set memory address for injection
193 # echo 0xfffffffffffff000 > param2 193 # echo 0xfffffffffffff000 > param2 # Mask - anywhere in this page
194 # echo 0x8 > error_type 194 # echo 0x8 > error_type # Choose correctable memory error
195 # echo 1 > error_inject 195 # echo 1 > error_inject # Inject now
196 196
197 You should see something like this in dmesg:: 197 You should see something like this in dmesg::
198 198
199 [22715.830801] EDAC sbridge MC3: HANDLING MC 199 [22715.830801] EDAC sbridge MC3: HANDLING MCE MEMORY ERROR
200 [22715.834759] EDAC sbridge MC3: CPU 0: Mach 200 [22715.834759] EDAC sbridge MC3: CPU 0: Machine Check Event: 0 Bank 7: 8c00004000010090
201 [22715.834759] EDAC sbridge MC3: TSC 0 201 [22715.834759] EDAC sbridge MC3: TSC 0
202 [22715.834759] EDAC sbridge MC3: ADDR 123450 202 [22715.834759] EDAC sbridge MC3: ADDR 12345000 EDAC sbridge MC3: MISC 144780c86
203 [22715.834759] EDAC sbridge MC3: PROCESSOR 0 203 [22715.834759] EDAC sbridge MC3: PROCESSOR 0:306e7 TIME 1422553404 SOCKET 0 APIC 0
204 [22716.616173] EDAC MC3: 1 CE memory read er 204 [22716.616173] EDAC MC3: 1 CE memory read error on CPU_SrcID#0_Channel#0_DIMM#0 (channel:0 slot:0 page:0x12345 offset:0x0 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0090 socket:0 channel_mask:1 rank:0)
205 205
206 A CXL error injection example with $dport_dev= 206 A CXL error injection example with $dport_dev=0000:e0:01.1::
207 207
208 # cd /sys/kernel/debug/cxl/ 208 # cd /sys/kernel/debug/cxl/
209 # ls 209 # ls
210 0000:e0:01.1 0000:0c:00.0 210 0000:e0:01.1 0000:0c:00.0
211 # cat einj_types # See whic 211 # cat einj_types # See which errors can be injected
212 0x00008000 CXL.mem Protocol Correctab 212 0x00008000 CXL.mem Protocol Correctable
213 0x00010000 CXL.mem Protocol Uncorrect 213 0x00010000 CXL.mem Protocol Uncorrectable non-fatal
214 0x00020000 CXL.mem Protocol Uncorrect 214 0x00020000 CXL.mem Protocol Uncorrectable fatal
215 # cd 0000:e0:01.1 # Navigate 215 # cd 0000:e0:01.1 # Navigate to dport to inject into
216 # echo 0x8000 > einj_inject # Inject e 216 # echo 0x8000 > einj_inject # Inject error
217 217
218 Special notes for injection into SGX enclaves: 218 Special notes for injection into SGX enclaves:
219 219
220 There may be a separate BIOS setup option to e 220 There may be a separate BIOS setup option to enable SGX injection.
221 221
222 The injection process consists of setting some 222 The injection process consists of setting some special memory controller
223 trigger that will inject the error on the next 223 trigger that will inject the error on the next write to the target
224 address. But the h/w prevents any software out 224 address. But the h/w prevents any software outside of an SGX enclave
225 from accessing enclave pages (even BIOS SMM mo 225 from accessing enclave pages (even BIOS SMM mode).
226 226
227 The following sequence can be used: 227 The following sequence can be used:
228 1) Determine physical address of enclave pag 228 1) Determine physical address of enclave page
229 2) Use "notrigger=1" mode to inject (this wi 229 2) Use "notrigger=1" mode to inject (this will setup
230 the injection address, but will not actua 230 the injection address, but will not actually inject)
231 3) Enter the enclave 231 3) Enter the enclave
232 4) Store data to the virtual address matchin 232 4) Store data to the virtual address matching physical address from step 1
233 5) Execute CLFLUSH for that virtual address 233 5) Execute CLFLUSH for that virtual address
234 6) Spin delay for 250ms 234 6) Spin delay for 250ms
235 7) Read from the virtual address. This will 235 7) Read from the virtual address. This will trigger the error
236 236
237 For more information about EINJ, please refer 237 For more information about EINJ, please refer to ACPI specification
238 version 4.0, section 17.5 and ACPI 5.0, sectio 238 version 4.0, section 17.5 and ACPI 5.0, section 18.6.
Linux® is a registered trademark of Linus Torvalds in the United States and other countries.
TOMOYO® is a registered trademark of NTT DATA CORPORATION.