^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) Compile-time stack metadata validation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) ======================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) Overview
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) --------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) The kernel CONFIG_STACK_VALIDATION option enables a host tool named
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) objtool which runs at compile time. It has a "check" subcommand which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10) analyzes every .o file and ensures the validity of its stack metadata.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) It enforces a set of rules on asm code and C inline assembly code so
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) that stack traces can be reliable.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) For each function, it recursively follows all possible code paths and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) validates the correct frame pointer state at each instruction.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) It also follows code paths involving special sections, like
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) .altinstructions, __jump_table, and __ex_table, which can add
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) alternative execution paths to a given instruction (or set of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) instructions). Similarly, it knows how to follow switch statements, for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) which gcc sometimes uses jump tables.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) (Objtool also has an 'orc generate' subcommand which generates debuginfo
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) for the ORC unwinder. See Documentation/x86/orc-unwinder.rst in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) kernel tree for more details.)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) Why do we need stack metadata validation?
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) -----------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) Here are some of the benefits of validating stack metadata:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) a) More reliable stack traces for frame pointer enabled kernels
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) Frame pointers are used for debugging purposes. They allow runtime
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) code and debug tools to be able to walk the stack to determine the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) chain of function call sites that led to the currently executing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) code.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) For some architectures, frame pointers are enabled by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) CONFIG_FRAME_POINTER. For some other architectures they may be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) required by the ABI (sometimes referred to as "backchain pointers").
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) For C code, gcc automatically generates instructions for setting up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) frame pointers when the -fno-omit-frame-pointer option is used.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) But for asm code, the frame setup instructions have to be written by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) hand, which most people don't do. So the end result is that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) CONFIG_FRAME_POINTER is honored for C code but not for most asm code.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) For stack traces based on frame pointers to be reliable, all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) functions which call other functions must first create a stack frame
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) and update the frame pointer. If a first function doesn't properly
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) create a stack frame before calling a second function, the *caller*
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) of the first function will be skipped on the stack trace.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) For example, consider the following example backtrace with frame
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) pointers enabled:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) [<ffffffff81812584>] dump_stack+0x4b/0x63
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) [<ffffffff812d6dc2>] cmdline_proc_show+0x12/0x30
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) [<ffffffff8127f568>] seq_read+0x108/0x3e0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63) [<ffffffff812cce62>] proc_reg_read+0x42/0x70
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) [<ffffffff81256197>] __vfs_read+0x37/0x100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) [<ffffffff81256b16>] vfs_read+0x86/0x130
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) [<ffffffff81257898>] SyS_read+0x58/0xd0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) [<ffffffff8181c1f2>] entry_SYSCALL_64_fastpath+0x12/0x76
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) It correctly shows that the caller of cmdline_proc_show() is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) seq_read().
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) If we remove the frame pointer logic from cmdline_proc_show() by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73) replacing the frame pointer related instructions with nops, here's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) what it looks like instead:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) [<ffffffff81812584>] dump_stack+0x4b/0x63
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) [<ffffffff812d6dc2>] cmdline_proc_show+0x12/0x30
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) [<ffffffff812cce62>] proc_reg_read+0x42/0x70
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79) [<ffffffff81256197>] __vfs_read+0x37/0x100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) [<ffffffff81256b16>] vfs_read+0x86/0x130
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) [<ffffffff81257898>] SyS_read+0x58/0xd0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) [<ffffffff8181c1f2>] entry_SYSCALL_64_fastpath+0x12/0x76
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) Notice that cmdline_proc_show()'s caller, seq_read(), has been
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) skipped. Instead the stack trace seems to show that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) cmdline_proc_show() was called by proc_reg_read().
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) The benefit of objtool here is that because it ensures that *all*
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) functions honor CONFIG_FRAME_POINTER, no functions will ever[*] be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) skipped on a stack trace.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92) [*] unless an interrupt or exception has occurred at the very
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) beginning of a function before the stack frame has been created,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) or at the very end of the function after the stack frame has been
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) destroyed. This is an inherent limitation of frame pointers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) b) ORC (Oops Rewind Capability) unwind table generation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) An alternative to frame pointers and DWARF, ORC unwind data can be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) used to walk the stack. Unlike frame pointers, ORC data is out of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) band. So it doesn't affect runtime performance and it can be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) reliable even when interrupts or exceptions are involved.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) For more details, see Documentation/x86/orc-unwinder.rst.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) c) Higher live patching compatibility rate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) Livepatch has an optional "consistency model", which is needed for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) more complex patches. In order for the consistency model to work,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) stack traces need to be reliable (or an unreliable condition needs to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) be detectable). Objtool makes that possible.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) For more details, see the livepatch documentation in the Linux kernel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) source tree at Documentation/livepatch/livepatch.rst.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) Rules
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) -----
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) To achieve the validation, objtool enforces the following rules:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) 1. Each callable function must be annotated as such with the ELF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) function type. In asm code, this is typically done using the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) ENTRY/ENDPROC macros. If objtool finds a return instruction
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) outside of a function, it flags an error since that usually indicates
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) callable code which should be annotated accordingly.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) This rule is needed so that objtool can properly identify each
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) callable function in order to analyze its stack metadata.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) 2. Conversely, each section of code which is *not* callable should *not*
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) be annotated as an ELF function. The ENDPROC macro shouldn't be used
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) in this case.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) This rule is needed so that objtool can ignore non-callable code.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) Such code doesn't have to follow any of the other rules.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) 3. Each callable function which calls another function must have the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) correct frame pointer logic, if required by CONFIG_FRAME_POINTER or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) the architecture's back chain rules. This can by done in asm code
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) with the FRAME_BEGIN/FRAME_END macros.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) This rule ensures that frame pointer based stack traces will work as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) designed. If function A doesn't create a stack frame before calling
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) function B, the _caller_ of function A will be skipped on the stack
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) trace.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) 4. Dynamic jumps and jumps to undefined symbols are only allowed if:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) a) the jump is part of a switch statement; or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) b) the jump matches sibling call semantics and the frame pointer has
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) the same value it had on function entry.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) This rule is needed so that objtool can reliably analyze all of a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) function's code paths. If a function jumps to code in another file,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) and it's not a sibling call, objtool has no way to follow the jump
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) because it only analyzes a single file at a time.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) 5. A callable function may not execute kernel entry/exit instructions.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) The only code which needs such instructions is kernel entry code,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) which shouldn't be be in callable functions anyway.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) This rule is just a sanity check to ensure that callable functions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) return normally.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167) Objtool warnings
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) ----------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) For asm files, if you're getting an error which doesn't make sense,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) first make sure that the affected code follows the above rules.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) For C files, the common culprits are inline asm statements and calls to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) "noreturn" functions. See below for more details.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) Another possible cause for errors in C code is if the Makefile removes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) -fno-omit-frame-pointer or adds -fomit-frame-pointer to the gcc options.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) Here are some examples of common warnings reported by objtool, what
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) they mean, and suggestions for how to fix them.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) 1. file.o: warning: objtool: func()+0x128: call without frame pointer save/setup
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) The func() function made a function call without first saving and/or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) updating the frame pointer, and CONFIG_FRAME_POINTER is enabled.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) If the error is for an asm file, and func() is indeed a callable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) function, add proper frame pointer logic using the FRAME_BEGIN and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) FRAME_END macros. Otherwise, if it's not a callable function, remove
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) its ELF function annotation by changing ENDPROC to END, and instead
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192) use the manual unwind hint macros in asm/unwind_hints.h.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) If it's a GCC-compiled .c file, the error may be because the function
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) uses an inline asm() statement which has a "call" instruction. An
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) asm() statement with a call instruction must declare the use of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) stack pointer in its output operand. On x86_64, this means adding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) the ASM_CALL_CONSTRAINT as an output constraint:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) asm volatile("call func" : ASM_CALL_CONSTRAINT);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) Otherwise the stack frame may not get created before the call.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205) 2. file.o: warning: objtool: .text+0x53: unreachable instruction
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207) Objtool couldn't find a code path to reach the instruction.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) If the error is for an asm file, and the instruction is inside (or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210) reachable from) a callable function, the function should be annotated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211) with the ENTRY/ENDPROC macros (ENDPROC is the important one).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) Otherwise, the code should probably be annotated with the unwind hint
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213) macros in asm/unwind_hints.h so objtool and the unwinder can know the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214) stack state associated with the code.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216) If you're 100% sure the code won't affect stack traces, or if you're
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217) a just a bad person, you can tell objtool to ignore it. See the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218) "Adding exceptions" section below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220) If it's not actually in a callable function (e.g. kernel entry code),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) change ENDPROC to END.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224) 4. file.o: warning: objtool: func(): can't find starting instruction
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226) file.o: warning: objtool: func()+0x11dd: can't decode instruction
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228) Does the file have data in a text section? If so, that can confuse
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229) objtool's instruction decoder. Move the data to a more appropriate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230) section like .data or .rodata.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233) 5. file.o: warning: objtool: func()+0x6: unsupported instruction in callable function
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235) This is a kernel entry/exit instruction like sysenter or iret. Such
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) instructions aren't allowed in a callable function, and are most
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237) likely part of the kernel entry code. They should usually not have
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238) the callable function annotation (ENDPROC) and should always be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) annotated with the unwind hint macros in asm/unwind_hints.h.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242) 6. file.o: warning: objtool: func()+0x26: sibling call from callable instruction with modified stack frame
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) This is a dynamic jump or a jump to an undefined symbol. Objtool
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) assumed it's a sibling call and detected that the frame pointer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246) wasn't first restored to its original state.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248) If it's not really a sibling call, you may need to move the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249) destination code to the local file.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251) If the instruction is not actually in a callable function (e.g.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252) kernel entry code), change ENDPROC to END and annotate manually with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253) the unwind hint macros in asm/unwind_hints.h.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256) 7. file: warning: objtool: func()+0x5c: stack state mismatch
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258) The instruction's frame pointer state is inconsistent, depending on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259) which execution path was taken to reach the instruction.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261) Make sure that, when CONFIG_FRAME_POINTER is enabled, the function
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262) pushes and sets up the frame pointer (for x86_64, this means rbp) at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263) the beginning of the function and pops it at the end of the function.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 264) Also make sure that no other code in the function touches the frame
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 265) pointer.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 266)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 267) Another possibility is that the code has some asm or inline asm which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 268) does some unusual things to the stack or the frame pointer. In such
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 269) cases it's probably appropriate to use the unwind hint macros in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 270) asm/unwind_hints.h.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 271)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 272)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 273) 8. file.o: warning: objtool: funcA() falls through to next function funcB()
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 274)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 275) This means that funcA() doesn't end with a return instruction or an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 276) unconditional jump, and that objtool has determined that the function
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 277) can fall through into the next function. There could be different
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 278) reasons for this:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 279)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 280) 1) funcA()'s last instruction is a call to a "noreturn" function like
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 281) panic(). In this case the noreturn function needs to be added to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 282) objtool's hard-coded global_noreturns array. Feel free to bug the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 283) objtool maintainer, or you can submit a patch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 284)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 285) 2) funcA() uses the unreachable() annotation in a section of code
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 286) that is actually reachable.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 287)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 288) 3) If funcA() calls an inline function, the object code for funcA()
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 289) might be corrupt due to a gcc bug. For more details, see:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 290) https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70646
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 291)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 292) 9. file.o: warning: objtool: funcA() call to funcB() with UACCESS enabled
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 293)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 294) This means that an unexpected call to a non-whitelisted function exists
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 295) outside of arch-specific guards.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 296) X86: SMAP (stac/clac): __uaccess_begin()/__uaccess_end()
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 297) ARM: PAN: uaccess_enable()/uaccess_disable()
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 298)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 299) These functions should be called to denote a minimal critical section around
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 300) access to __user variables. See also: https://lwn.net/Articles/517475/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 301)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 302) The intention of the warning is to prevent calls to funcB() from eventually
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 303) calling schedule(), potentially leaking the AC flags state, and not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 304) restoring them correctly.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 305)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 306) It also helps verify that there are no unexpected calls to funcB() which may
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 307) access user space pages with protections against doing so disabled.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 308)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 309) To fix, either:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 310) 1) remove explicit calls to funcB() from funcA().
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 311) 2) add the correct guards before and after calls to low level functions like
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 312) __get_user_size()/__put_user_size().
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 313) 3) add funcB to uaccess_safe_builtin whitelist in tools/objtool/check.c, if
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 314) funcB obviously does not call schedule(), and is marked notrace (since
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 315) function tracing inserts additional calls, which is not obvious from the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 316) sources).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 317)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 318) 10. file.o: warning: func()+0x5c: alternative modifies stack
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 319)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 320) This means that an alternative includes instructions that modify the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 321) stack. The problem is that there is only one ORC unwind table, this means
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 322) that the ORC unwind entries must be valid for each of the alternatives.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 323) The easiest way to enforce this is to ensure alternatives do not contain
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 324) any ORC entries, which in turn implies the above constraint.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 325)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 326) 11. file.o: warning: unannotated intra-function call
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 327)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 328) This warning means that a direct call is done to a destination which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 329) is not at the beginning of a function. If this is a legit call, you
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 330) can remove this warning by putting the ANNOTATE_INTRA_FUNCTION_CALL
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 331) directive right before the call.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 332)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 333)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 334) If the error doesn't seem to make sense, it could be a bug in objtool.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 335) Feel free to ask the objtool maintainer for help.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 336)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 337)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 338) Adding exceptions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 339) -----------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 340)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 341) If you _really_ need objtool to ignore something, and are 100% sure
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 342) that it won't affect kernel stack traces, you can tell objtool to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 343) ignore it:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 344)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 345) - To skip validation of a function, use the STACK_FRAME_NON_STANDARD
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 346) macro.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 347)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 348) - To skip validation of a file, add
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 349)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 350) OBJECT_FILES_NON_STANDARD_filename.o := y
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 351)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 352) to the Makefile.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 353)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 354) - To skip validation of a directory, add
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 355)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 356) OBJECT_FILES_NON_STANDARD := y
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 357)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 358) to the Makefile.
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags