^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) ==========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) Kprobe-based Event Tracing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) ==========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) :Author: Masami Hiramatsu
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7) Overview
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) --------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) These events are similar to tracepoint based events. Instead of Tracepoint,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10) this is based on kprobes (kprobe and kretprobe). So it can probe wherever
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) kprobes can probe (this means, all functions except those with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) __kprobes/nokprobe_inline annotation and those marked NOKPROBE_SYMBOL).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13) Unlike the Tracepoint based event, this can be added and removed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) dynamically, on the fly.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) To enable this feature, build your kernel with CONFIG_KPROBE_EVENTS=y.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) Similar to the events tracer, this doesn't need to be activated via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) current_tracer. Instead of that, add probe points via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) /sys/kernel/debug/tracing/kprobe_events, and enable it via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) /sys/kernel/debug/tracing/events/kprobes/<EVENT>/enable.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) You can also use /sys/kernel/debug/tracing/dynamic_events instead of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) kprobe_events. That interface will provide unified access to other
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) dynamic events too.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) Synopsis of kprobe_events
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) -------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) p[:[GRP/]EVENT] [MOD:]SYM[+offs]|MEMADDR [FETCHARGS] : Set a probe
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) r[MAXACTIVE][:[GRP/]EVENT] [MOD:]SYM[+0] [FETCHARGS] : Set a return probe
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) p:[GRP/]EVENT] [MOD:]SYM[+0]%return [FETCHARGS] : Set a return probe
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) -:[GRP/]EVENT : Clear a probe
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) GRP : Group name. If omitted, use "kprobes" for it.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) EVENT : Event name. If omitted, the event name is generated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) based on SYM+offs or MEMADDR.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) MOD : Module name which has given SYM.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) SYM[+offs] : Symbol+offset where the probe is inserted.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) SYM%return : Return address of the symbol
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) MEMADDR : Address where the probe is inserted.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) MAXACTIVE : Maximum number of instances of the specified function that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) can be probed simultaneously, or 0 for the default value
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) as defined in Documentation/trace/kprobes.rst section 1.3.1.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) FETCHARGS : Arguments. Each probe can have up to 128 args.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) %REG : Fetch register REG
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) @ADDR : Fetch memory at ADDR (ADDR should be in kernel)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) @SYM[+|-offs] : Fetch memory at SYM +|- offs (SYM should be a data symbol)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) $stackN : Fetch Nth entry of stack (N >= 0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) $stack : Fetch stack address.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) $argN : Fetch the Nth function argument. (N >= 1) (\*1)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) $retval : Fetch return value.(\*2)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) $comm : Fetch current task comm.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) +|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*3)(\*4)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) \IMM : Store an immediate value to the argument.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) NAME=FETCHARG : Set NAME as the argument name of FETCHARG.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59) FETCHARG:TYPE : Set TYPE as the type of FETCHARG. Currently, basic types
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) (u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) (x8/x16/x32/x64), "string", "ustring" and bitfield
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) are supported.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) (\*1) only for the probe on function entry (offs == 0).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) (\*2) only for return probe.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) (\*3) this is useful for fetching a field of data structures.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) (\*4) "u" means user-space dereference. See :ref:`user_mem_access`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) Types
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) -----
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) Several types are supported for fetch-args. Kprobe tracer will access memory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) by given type. Prefix 's' and 'u' means those types are signed and unsigned
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73) respectively. 'x' prefix implies it is unsigned. Traced arguments are shown
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) in decimal ('s' and 'u') or hexadecimal ('x'). Without type casting, 'x32'
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) or 'x64' is used depends on the architecture (e.g. x86-32 uses x32, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) x86-64 uses x64).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) These value types can be an array. To record array data, you can add '[N]'
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) (where N is a fixed number, less than 64) to the base type.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79) E.g. 'x16[4]' means an array of x16 (2bytes hex) with 4 elements.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) Note that the array can be applied to memory type fetchargs, you can not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) apply it to registers/stack-entries etc. (for example, '$stack1:x8[8]' is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) wrong, but '+8($stack):x8[8]' is OK.)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) String type is a special type, which fetches a "null-terminated" string from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) kernel space. This means it will fail and store NULL if the string container
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) has been paged out. "ustring" type is an alternative of string for user-space.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) See :ref:`user_mem_access` for more info..
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87) The string array type is a bit different from other types. For other base
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) types, <base-type>[1] is equal to <base-type> (e.g. +0(%di):x32[1] is same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) as +0(%di):x32.) But string[1] is not equal to string. The string type itself
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) represents "char array", but string array type represents "char * array".
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) So, for example, +0(%di):string[1] is equal to +0(+0(%di)):string.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92) Bitfield is another special type, which takes 3 parameters, bit-width, bit-
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) offset, and container-size (usually 32). The syntax is::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) b<bit-width>@<bit-offset>/<container-size>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) Symbol type('symbol') is an alias of u32 or u64 type (depends on BITS_PER_LONG)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) which shows given pointer in "symbol+offset" style.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) For $comm, the default type is "string"; any other type is invalid.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) .. _user_mem_access:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) User Memory Access
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) ------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) Kprobe events supports user-space memory access. For that purpose, you can use
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) either user-space dereference syntax or 'ustring' type.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) The user-space dereference syntax allows you to access a field of a data
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) structure in user-space. This is done by adding the "u" prefix to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) dereference syntax. For example, +u4(%si) means it will read memory from the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) address in the register %si offset by 4, and the memory is expected to be in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) user-space. You can use this for strings too, e.g. +u0(%si):string will read
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) a string from the address in the register %si that is expected to be in user-
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) space. 'ustring' is a shortcut way of performing the same task. That is,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) +0(%si):ustring is equivalent to +u0(%si):string.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) Note that kprobe-event provides the user-memory access syntax but it doesn't
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) use it transparently. This means if you use normal dereference or string type
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) for user memory, it might fail, and may always fail on some archs. The user
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) has to carefully check if the target data is in kernel or user space.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) Per-Probe Event Filtering
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) -------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) Per-probe event filtering feature allows you to set different filter on each
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) probe and gives you what arguments will be shown in trace buffer. If an event
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) name is specified right after 'p:' or 'r:' in kprobe_events, it adds an event
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) under tracing/events/kprobes/<EVENT>, at the directory you can see 'id',
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) 'enable', 'format', 'filter' and 'trigger'.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) enable:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) You can enable/disable the probe by writing 1 or 0 on it.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) format:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) This shows the format of this probe event.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) filter:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) You can write filtering rules of this event.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) id:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) This shows the id of this probe event.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) trigger:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) This allows to install trigger commands which are executed when the event is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) hit (for details, see Documentation/trace/events.rst, section 6).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) Event Profiling
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) You can check the total number of probe hits and probe miss-hits via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) /sys/kernel/debug/tracing/kprobe_profile.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) The first column is event name, the second is the number of probe hits,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) the third is the number of probe miss-hits.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) Kernel Boot Parameter
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) ---------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) You can add and enable new kprobe events when booting up the kernel by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) "kprobe_event=" parameter. The parameter accepts a semicolon-delimited
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) kprobe events, which format is similar to the kprobe_events.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) The difference is that the probe definition parameters are comma-delimited
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) instead of space. For example, adding myprobe event on do_sys_open like below
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) p:myprobe do_sys_open dfd=%ax filename=%dx flags=%cx mode=+4($stack)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) should be below for kernel boot parameter (just replace spaces with comma)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165) p:myprobe,do_sys_open,dfd=%ax,filename=%dx,flags=%cx,mode=+4($stack)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) Usage examples
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169) --------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) To add a probe as a new event, write a new definition to kprobe_events
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) as below::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) echo 'p:myprobe do_sys_open dfd=%ax filename=%dx flags=%cx mode=+4($stack)' > /sys/kernel/debug/tracing/kprobe_events
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) This sets a kprobe on the top of do_sys_open() function with recording
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) 1st to 4th arguments as "myprobe" event. Note, which register/stack entry is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) assigned to each function argument depends on arch-specific ABI. If you unsure
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) the ABI, please try to use probe subcommand of perf-tools (you can find it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) under tools/perf/).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) As this example shows, users can choose more familiar names for each arguments.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) echo 'r:myretprobe do_sys_open $retval' >> /sys/kernel/debug/tracing/kprobe_events
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) This sets a kretprobe on the return point of do_sys_open() function with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) recording return value as "myretprobe" event.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) You can see the format of these events via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) /sys/kernel/debug/tracing/events/kprobes/<EVENT>/format.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) cat /sys/kernel/debug/tracing/events/kprobes/myprobe/format
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192) name: myprobe
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) ID: 780
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) format:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) field:unsigned short common_type; offset:0; size:2; signed:0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) field:unsigned char common_flags; offset:2; size:1; signed:0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) field:unsigned char common_preempt_count; offset:3; size:1;signed:0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) field:int common_pid; offset:4; size:4; signed:1;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) field:unsigned long __probe_ip; offset:12; size:4; signed:0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201) field:int __probe_nargs; offset:16; size:4; signed:1;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) field:unsigned long dfd; offset:20; size:4; signed:0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) field:unsigned long filename; offset:24; size:4; signed:0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) field:unsigned long flags; offset:28; size:4; signed:0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205) field:unsigned long mode; offset:32; size:4; signed:0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208) print fmt: "(%lx) dfd=%lx filename=%lx flags=%lx mode=%lx", REC->__probe_ip,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) REC->dfd, REC->filename, REC->flags, REC->mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211) You can see that the event has 4 arguments as in the expressions you specified.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214) echo > /sys/kernel/debug/tracing/kprobe_events
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216) This clears all probe points.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218) Or,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) echo -:myprobe >> kprobe_events
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223) This clears probe points selectively.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) Right after definition, each event is disabled by default. For tracing these
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226) events, you need to enable it.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229) echo 1 > /sys/kernel/debug/tracing/events/kprobes/myprobe/enable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230) echo 1 > /sys/kernel/debug/tracing/events/kprobes/myretprobe/enable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232) Use the following command to start tracing in an interval.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235) # echo 1 > tracing_on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) Open something...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237) # echo 0 > tracing_on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) And you can see the traced information via /sys/kernel/debug/tracing/trace.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242) cat /sys/kernel/debug/tracing/trace
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243) # tracer: nop
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) #
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) # TASK-PID CPU# TIMESTAMP FUNCTION
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246) # | | | | |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247) <...>-1447 [001] 1038282.286875: myprobe: (do_sys_open+0x0/0xd6) dfd=3 filename=7fffd1ec4440 flags=8000 mode=0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248) <...>-1447 [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) $retval=fffffffffffffffe
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249) <...>-1447 [001] 1038282.286885: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=40413c flags=8000 mode=1b6
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250) <...>-1447 [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $retval=3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251) <...>-1447 [001] 1038282.286969: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=4041c6 flags=98800 mode=10
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252) <...>-1447 [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $retval=3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255) Each line shows when the kernel hits an event, and <- SYMBOL means kernel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256) returns from SYMBOL(e.g. "sys_open+0x1b/0x1d <- do_sys_open" means kernel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257) returns from do_sys_open to sys_open+0x1b).