^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) Ramoops oops/panic logger
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) =========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4) Sergiu Iordache <sergiu@chromium.org>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) Updated: 10 Feb 2021
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) Introduction
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) ------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) Ramoops is an oops/panic logger that writes its logs to RAM before the system
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) crashes. It works by logging oopses and panics in a circular buffer. Ramoops
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13) needs a system with persistent RAM so that the content of that area can
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) survive after a restart.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) Ramoops concepts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) ----------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) Ramoops uses a predefined memory area to store the dump. The start and size
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) and type of the memory area are set using three variables:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) * ``mem_address`` for the start
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) * ``mem_size`` for the size. The memory size will be rounded down to a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) power of two.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) * ``mem_type`` to specifiy if the memory type (default is pgprot_writecombine).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) Typically the default value of ``mem_type=0`` should be used as that sets the pstore
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) mapping to pgprot_writecombine. Setting ``mem_type=1`` attempts to use
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) ``pgprot_noncached``, which only works on some platforms. This is because pstore
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30) depends on atomic operations. At least on ARM, pgprot_noncached causes the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) memory to be mapped strongly ordered, and atomic operations on strongly ordered
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) memory are implementation defined, and won't work on many ARMs such as omaps.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) Setting ``mem_type=2`` attempts to treat the memory region as normal memory,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) which enables full cache on it. This can improve the performance.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) The memory area is divided into ``record_size`` chunks (also rounded down to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) power of two) and each kmesg dump writes a ``record_size`` chunk of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) information.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) Limiting which kinds of kmsg dumps are stored can be controlled via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) the ``max_reason`` value, as defined in include/linux/kmsg_dump.h's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) ``enum kmsg_dump_reason``. For example, to store both Oopses and Panics,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) ``max_reason`` should be set to 2 (KMSG_DUMP_OOPS), to store only Panics
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) ``max_reason`` should be set to 1 (KMSG_DUMP_PANIC). Setting this to 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) (KMSG_DUMP_UNDEF), means the reason filtering will be controlled by the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46) ``printk.always_kmsg_dump`` boot param: if unset, it'll be KMSG_DUMP_OOPS,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) otherwise KMSG_DUMP_MAX.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) The module uses a counter to record multiple dumps but the counter gets reset
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) on restart (i.e. new dumps after the restart will overwrite old ones).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) Ramoops also supports software ECC protection of persistent memory regions.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) This might be useful when a hardware reset was used to bring the machine back
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) to life (i.e. a watchdog triggered). In such cases, RAM may be somewhat
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) corrupt, but usually it is restorable.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) Setting the parameters
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) ----------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) Setting the ramoops parameters can be done in several different manners:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) A. Use the module parameters (which have the names of the variables described
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63) as before). For quick debugging, you can also reserve parts of memory during
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) boot and then use the reserved memory for ramoops. For example, assuming a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) machine with > 128 MB of memory, the following kernel command line will tell
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) the kernel to use only the first 128 MB of memory, and place ECC-protected
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) ramoops region at 128 MB boundary::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) mem=128M ramoops.mem_address=0x8000000 ramoops.ecc=1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) B. Use Device Tree bindings, as described in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) ``Documentation/devicetree/bindings/reserved-memory/ramoops.txt``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73) For example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) reserved-memory {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) #address-cells = <2>;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) #size-cells = <2>;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) ranges;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) ramoops@8f000000 {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) compatible = "ramoops";
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) reg = <0 0x8f000000 0 0x100000>;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) record-size = <0x4000>;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) console-size = <0x4000>;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) };
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) };
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) C. Use a platform device and set the platform data. The parameters can then
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) be set through that platform data. An example of doing that is:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) .. code-block:: c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) #include <linux/pstore_ram.h>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) [...]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) static struct ramoops_platform_data ramoops_data = {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) .mem_size = <...>,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) .mem_address = <...>,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) .mem_type = <...>,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) .record_size = <...>,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) .max_reason = <...>,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) .ecc = <...>,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) };
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) static struct platform_device ramoops_dev = {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) .name = "ramoops",
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) .dev = {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) .platform_data = &ramoops_data,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) },
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) };
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) [... inside a function ...]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) int ret;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) ret = platform_device_register(&ramoops_dev);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) if (ret) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) printk(KERN_ERR "unable to register platform device\n");
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) return ret;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) You can specify either RAM memory or peripheral devices' memory. However, when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) specifying RAM, be sure to reserve the memory by issuing memblock_reserve()
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) very early in the architecture code, e.g.::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) #include <linux/memblock.h>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) memblock_reserve(ramoops_data.mem_address, ramoops_data.mem_size);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) Dump format
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) -----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) The data dump begins with a header, currently defined as ``====`` followed by a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) timestamp and a new line. The dump then continues with the actual data.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) Reading the data
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) ----------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) The dump data can be read from the pstore filesystem. The format for these
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) files is ``dmesg-ramoops-N``, where N is the record number in memory. To delete
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) a stored record from RAM, simply unlink the respective pstore file.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) Persistent function tracing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) ---------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) Persistent function tracing might be useful for debugging software or hardware
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) related hangs. The functions call chain log is stored in a ``ftrace-ramoops``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) file. Here is an example of usage::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) # mount -t debugfs debugfs /sys/kernel/debug/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) # echo 1 > /sys/kernel/debug/pstore/record_ftrace
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) # reboot -f
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) [...]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) # mount -t pstore pstore /mnt/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) # tail /mnt/ftrace-ramoops
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) 0 ffffffff8101ea64 ffffffff8101bcda native_apic_mem_read <- disconnect_bsp_APIC+0x6a/0xc0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) 0 ffffffff8101ea44 ffffffff8101bcf6 native_apic_mem_write <- disconnect_bsp_APIC+0x86/0xc0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) 0 ffffffff81020084 ffffffff8101a4b5 hpet_disable <- native_machine_shutdown+0x75/0x90
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) 0 ffffffff81005f94 ffffffff8101a4bb iommu_shutdown_noop <- native_machine_shutdown+0x7b/0x90
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) 0 ffffffff8101a6a1 ffffffff8101a437 native_machine_emergency_restart <- native_machine_restart+0x37/0x40
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) 0 ffffffff811f9876 ffffffff8101a73a acpi_reboot <- native_machine_emergency_restart+0xaa/0x1e0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) 0 ffffffff8101a514 ffffffff8101a772 mach_reboot_fixups <- native_machine_emergency_restart+0xe2/0x1e0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162) 0 ffffffff811d9c54 ffffffff8101a7a0 __const_udelay <- native_machine_emergency_restart+0x110/0x1e0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) 0 ffffffff811d9c34 ffffffff811d9c80 __delay <- __const_udelay+0x30/0x40
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) 0 ffffffff811d9d14 ffffffff811d9c3f delay_tsc <- __delay+0xf/0x20
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags