^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) Tainted kernels
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4) The kernel will mark itself as 'tainted' when something occurs that might be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) relevant later when investigating problems. Don't worry too much about this,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) most of the time it's not a problem to run a tainted kernel; the information is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7) mainly of interest once someone wants to investigate some problem, as its real
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) cause might be the event that got the kernel tainted. That's why bug reports
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) from tainted kernels will often be ignored by developers, hence try to reproduce
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10) problems with an untainted kernel.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) Note the kernel will remain tainted even after you undo what caused the taint
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13) (i.e. unload a proprietary kernel module), to indicate the kernel remains not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) trustworthy. That's also why the kernel will print the tainted state when it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) notices an internal problem (a 'kernel bug'), a recoverable error
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) ('kernel oops') or a non-recoverable error ('kernel panic') and writes debug
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) information about this to the logs ``dmesg`` outputs. It's also possible to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) check the tainted state at runtime through a file in ``/proc/``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) Tainted flag in bugs, oops or panics messages
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) You find the tainted state near the top in a line starting with 'CPU:'; if or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) why the kernel was tainted is shown after the Process ID ('PID:') and a shortened
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) name of the command ('Comm:') that triggered the event::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) Oops: 0002 [#1] SMP PTI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30) CPU: 0 PID: 4424 Comm: insmod Tainted: P W O 4.20.0-0.rc6.fc30 #1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) RIP: 0010:my_oops_init+0x13/0x1000 [kpanic]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) [...]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) You'll find a 'Not tainted: ' there if the kernel was not tainted at the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) time of the event; if it was, then it will print 'Tainted: ' and characters
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) either letters or blanks. In above example it looks like this::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) Tainted: P W O
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) The meaning of those characters is explained in the table below. In this case
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) the kernel got tainted earlier because a proprietary Module (``P``) was loaded,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) a warning occurred (``W``), and an externally-built module was loaded (``O``).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) To decode other letters use the table below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) Decoding tainted state at runtime
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) At runtime, you can query the tainted state by reading
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) ``cat /proc/sys/kernel/tainted``. If that returns ``0``, the kernel is not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) tainted; any other number indicates the reasons why it is. The easiest way to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) decode that number is the script ``tools/debugging/kernel-chktaint``, which your
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) distribution might ship as part of a package called ``linux-tools`` or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) ``kernel-tools``; if it doesn't you can download the script from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) `git.kernel.org <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/tools/debugging/kernel-chktaint>`_
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) and execute it with ``sh kernel-chktaint``, which would print something like
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) this on the machine that had the statements in the logs that were quoted earlier::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) Kernel is Tainted for following reasons:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) * Proprietary module was loaded (#0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) * Kernel issued warning (#9)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63) * Externally-built ('out-of-tree') module was loaded (#12)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) See Documentation/admin-guide/tainted-kernels.rst in the Linux kernel or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) https://www.kernel.org/doc/html/latest/admin-guide/tainted-kernels.html for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) a more details explanation of the various taint flags.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) Raw taint value as int/string: 4609/'P W O '
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) You can try to decode the number yourself. That's easy if there was only one
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) reason that got your kernel tainted, as in this case you can find the number
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) with the table below. If there were multiple reasons you need to decode the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) number, as it is a bitfield, where each bit indicates the absence or presence of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73) a particular type of taint. It's best to leave that to the aforementioned
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) script, but if you need something quick you can use this shell command to check
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) which bits are set::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) $ for i in $(seq 18); do echo $(($i-1)) $(($(cat /proc/sys/kernel/tainted)>>($i-1)&1));done
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79) Table for decoding tainted state
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) === === ====== ========================================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) Bit Log Number Reason that got the kernel tainted
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) === === ====== ========================================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) 0 G/P 1 proprietary module was loaded
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) 1 _/F 2 module was force loaded
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87) 2 _/S 4 SMP kernel oops on an officially SMP incapable processor
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) 3 _/R 8 module was force unloaded
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) 4 _/M 16 processor reported a Machine Check Exception (MCE)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) 5 _/B 32 bad page referenced or some unexpected page flags
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) 6 _/U 64 taint requested by userspace application
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92) 7 _/D 128 kernel died recently, i.e. there was an OOPS or BUG
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) 8 _/A 256 ACPI table overridden by user
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) 9 _/W 512 kernel issued warning
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) 10 _/C 1024 staging driver was loaded
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) 11 _/I 2048 workaround for bug in platform firmware applied
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) 12 _/O 4096 externally-built ("out-of-tree") module was loaded
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) 13 _/E 8192 unsigned module was loaded
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) 14 _/L 16384 soft lockup occurred
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) 15 _/K 32768 kernel has been live patched
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) 16 _/X 65536 auxiliary taint, defined for and used by distros
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) 17 _/T 131072 kernel was built with the struct randomization plugin
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) === === ====== ========================================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) Note: The character ``_`` is representing a blank in this table to make reading
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) easier.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) More detailed explanation for tainting
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) 0) ``G`` if all modules loaded have a GPL or compatible license, ``P`` if
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) any proprietary module has been loaded. Modules without a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) MODULE_LICENSE or with a MODULE_LICENSE that is not recognised by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) insmod as GPL compatible are assumed to be proprietary.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) 1) ``F`` if any module was force loaded by ``insmod -f``, ``' '`` if all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) modules were loaded normally.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) 2) ``S`` if the oops occurred on an SMP kernel running on hardware that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) hasn't been certified as safe to run multiprocessor.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) Currently this occurs only on various Athlons that are not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) SMP capable.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) 3) ``R`` if a module was force unloaded by ``rmmod -f``, ``' '`` if all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) modules were unloaded normally.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) 4) ``M`` if any processor has reported a Machine Check Exception,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) ``' '`` if no Machine Check Exceptions have occurred.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) 5) ``B`` If a page-release function has found a bad page reference or some
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) unexpected page flags. This indicates a hardware problem or a kernel bug;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) there should be other information in the log indicating why this tainting
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) occurred.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) 6) ``U`` if a user or user application specifically requested that the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) Tainted flag be set, ``' '`` otherwise.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) 7) ``D`` if the kernel has died recently, i.e. there was an OOPS or BUG.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) 8) ``A`` if an ACPI table has been overridden.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) 9) ``W`` if a warning has previously been issued by the kernel.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) (Though some warnings may set more specific taint flags.)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) 10) ``C`` if a staging driver has been loaded.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) 11) ``I`` if the kernel is working around a severe bug in the platform
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) firmware (BIOS or similar).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) 12) ``O`` if an externally-built ("out-of-tree") module has been loaded.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) 13) ``E`` if an unsigned module has been loaded in a kernel supporting
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) module signature.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) 14) ``L`` if a soft lockup has previously occurred on the system.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) 15) ``K`` if the kernel has been live patched.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) 16) ``X`` Auxiliary taint, defined for and used by Linux distributors.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) 17) ``T`` Kernel was build with the randstruct plugin, which can intentionally
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162) produce extremely unusual kernel structure layouts (even performance
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) pathological ones), which is important to know when debugging. Set at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) build time.