Orange Pi5 kernel

^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   1) .. SPDX-License-Identifier: GPL-2.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   2) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   3) ====================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   4) APEI Error INJection
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   5) ====================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   6) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   7) EINJ provides a hardware error injection mechanism. It is very useful
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   8) for debugging and testing APEI and RAS features in general.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   9) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  10) You need to check whether your BIOS supports EINJ first. For that, look
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  11) for early boot messages similar to this one::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  12) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  13)   ACPI: EINJ 0x000000007370A000 000150 (v01 INTEL           00000001 INTL 00000001)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  14) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  15) which shows that the BIOS is exposing an EINJ table - it is the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  16) mechanism through which the injection is done.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  17) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  18) Alternatively, look in /sys/firmware/acpi/tables for an "EINJ" file,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  19) which is a different representation of the same thing.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  20) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  21) It doesn't necessarily mean that EINJ is not supported if those above
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  22) don't exist: before you give up, go into BIOS setup to see if the BIOS
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  23) has an option to enable error injection. Look for something called WHEA
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  24) or similar. Often, you need to enable an ACPI5 support option prior, in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  25) order to see the APEI,EINJ,... functionality supported and exposed by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  26) the BIOS menu.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  27) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  28) To use EINJ, make sure the following are options enabled in your kernel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  29) configuration::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  30) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  31)   CONFIG_DEBUG_FS
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  32)   CONFIG_ACPI_APEI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  33)   CONFIG_ACPI_APEI_EINJ
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  34) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  35) The EINJ user interface is in <debugfs mount point>/apei/einj.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  36) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  37) The following files belong to it:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  38) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  39) - available_error_type
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  40) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  41)   This file shows which error types are supported:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  42) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  43)   ================  ===================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  44)   Error Type Value	Error Description
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  45)   ================  ===================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  46)   0x00000001        Processor Correctable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  47)   0x00000002        Processor Uncorrectable non-fatal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  48)   0x00000004        Processor Uncorrectable fatal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  49)   0x00000008        Memory Correctable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  50)   0x00000010        Memory Uncorrectable non-fatal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  51)   0x00000020        Memory Uncorrectable fatal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  52)   0x00000040        PCI Express Correctable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  53)   0x00000080        PCI Express Uncorrectable fatal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  54)   0x00000100        PCI Express Uncorrectable non-fatal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  55)   0x00000200        Platform Correctable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  56)   0x00000400        Platform Uncorrectable non-fatal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  57)   0x00000800        Platform Uncorrectable fatal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  58)   ================  ===================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  59) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  60)   The format of the file contents are as above, except present are only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  61)   the available error types.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  62) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  63) - error_type
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  64) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  65)   Set the value of the error type being injected. Possible error types
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  66)   are defined in the file available_error_type above.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  67) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  68) - error_inject
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  69) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  70)   Write any integer to this file to trigger the error injection. Make
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  71)   sure you have specified all necessary error parameters, i.e. this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  72)   write should be the last step when injecting errors.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  73) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  74) - flags
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  75) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  76)   Present for kernel versions 3.13 and above. Used to specify which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  77)   of param{1..4} are valid and should be used by the firmware during
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  78)   injection. Value is a bitmask as specified in ACPI5.0 spec for the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  79)   SET_ERROR_TYPE_WITH_ADDRESS data structure:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  80) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  81)     Bit 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  82)       Processor APIC field valid (see param3 below).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  83)     Bit 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  84)       Memory address and mask valid (param1 and param2).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  85)     Bit 2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  86)       PCIe (seg,bus,dev,fn) valid (see param4 below).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  87) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  88)   If set to zero, legacy behavior is mimicked where the type of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  89)   injection specifies just one bit set, and param1 is multiplexed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  90) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  91) - param1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  92) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  93)   This file is used to set the first error parameter value. Its effect
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  94)   depends on the error type specified in error_type. For example, if
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  95)   error type is memory related type, the param1 should be a valid
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  96)   physical memory address. [Unless "flag" is set - see above]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  97) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  98) - param2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  99) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100)   Same use as param1 above. For example, if error type is of memory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101)   related type, then param2 should be a physical memory address mask.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)   Linux requires page or narrower granularity, say, 0xfffffffffffff000.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) - param3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106)   Used when the 0x1 bit is set in "flags" to specify the APIC id
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) - param4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109)   Used when the 0x4 bit is set in "flags" to specify target PCIe device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) - notrigger
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113)   The error injection mechanism is a two-step process. First inject the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114)   error, then perform some actions to trigger it. Setting "notrigger"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115)   to 1 skips the trigger phase, which *may* allow the user to cause the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116)   error in some other context by a simple access to the CPU, memory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117)   location, or device that is the target of the error injection. Whether
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118)   this actually works depends on what operations the BIOS actually
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119)   includes in the trigger phase.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) BIOS versions based on the ACPI 4.0 specification have limited options
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) in controlling where the errors are injected. Your BIOS may support an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) extension (enabled with the param_extension=1 module parameter, or boot
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) command line einj.param_extension=1). This allows the address and mask
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) for memory injections to be specified by the param1 and param2 files in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) apei/einj.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) BIOS versions based on the ACPI 5.0 specification have more control over
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) the target of the injection. For processor-related errors (type 0x1, 0x2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) and 0x4), you can set flags to 0x3 (param3 for bit 0, and param1 and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) param2 for bit 1) so that you have more information added to the error
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) signature being injected. The actual data passed is this::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) 	memory_address = param1;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) 	memory_address_range = param2;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) 	apicid = param3;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) 	pcie_sbdf = param4;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) For memory errors (type 0x8, 0x10 and 0x20) the address is set using
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) param1 with a mask in param2 (0x0 is equivalent to all ones). For PCI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) express errors (type 0x40, 0x80 and 0x100) the segment, bus, device and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) function are specified using param1::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144)          31     24 23    16 15    11 10      8  7        0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) 	+-------------------------------------------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) 	| segment |   bus  | device | function | reserved |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) 	+-------------------------------------------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) Anyway, you get the idea, if there's doubt just take a look at the code
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) in drivers/acpi/apei/einj.c.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) An ACPI 5.0 BIOS may also allow vendor-specific errors to be injected.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) In this case a file named vendor will contain identifying information
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) from the BIOS that hopefully will allow an application wishing to use
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) the vendor-specific extension to tell that they are running on a BIOS
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) that supports it. All vendor extensions have the 0x80000000 bit set in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) error_type. A file vendor_flags controls the interpretation of param1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) and param2 (1 = PROCESSOR, 2 = MEMORY, 4 = PCI). See your BIOS vendor
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) documentation for details (and expect changes to this API if vendors
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) creativity in using this feature expands beyond our expectations).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) An error injection example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165)   # cd /sys/kernel/debug/apei/einj
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166)   # cat available_error_type		# See which errors can be injected
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167)   0x00000002	Processor Uncorrectable non-fatal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168)   0x00000008	Memory Correctable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)   0x00000010	Memory Uncorrectable non-fatal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170)   # echo 0x12345000 > param1		# Set memory address for injection
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171)   # echo $((-1 << 12)) > param2		# Mask 0xfffffffffffff000 - anywhere in this page
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172)   # echo 0x8 > error_type			# Choose correctable memory error
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173)   # echo 1 > error_inject			# Inject now
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) You should see something like this in dmesg::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177)   [22715.830801] EDAC sbridge MC3: HANDLING MCE MEMORY ERROR
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178)   [22715.834759] EDAC sbridge MC3: CPU 0: Machine Check Event: 0 Bank 7: 8c00004000010090
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179)   [22715.834759] EDAC sbridge MC3: TSC 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180)   [22715.834759] EDAC sbridge MC3: ADDR 12345000 EDAC sbridge MC3: MISC 144780c86
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181)   [22715.834759] EDAC sbridge MC3: PROCESSOR 0:306e7 TIME 1422553404 SOCKET 0 APIC 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182)   [22716.616173] EDAC MC3: 1 CE memory read error on CPU_SrcID#0_Channel#0_DIMM#0 (channel:0 slot:0 page:0x12345 offset:0x0 grain:32 syndrome:0x0 -  area:DRAM err_code:0001:0090 socket:0 channel_mask:1 rank:0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) For more information about EINJ, please refer to ACPI specification
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) version 4.0, section 17.5 and ACPI 5.0, section 18.6.