Orange Pi5 kernel

Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards

3 Commits   0 Branches   0 Tags
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   1) ===============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   2) Lock Statistics
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   3) ===============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   4) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   5) What
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   6) ====
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   7) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   8) As the name suggests, it provides statistics on locks.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   9) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  10) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  11) Why
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  12) ===
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  13) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  14) Because things like lock contention can severely impact performance.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  15) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  16) How
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  17) ===
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  18) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  19) Lockdep already has hooks in the lock functions and maps lock instances to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  20) lock classes. We build on that (see Documentation/locking/lockdep-design.rst).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  21) The graph below shows the relation between the lock functions and the various
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  22) hooks therein::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  23) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  24)         __acquire
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  25)             |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  26)            lock _____
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  27)             |        \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  28)             |    __contended
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  29)             |         |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  30)             |       <wait>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  31)             | _______/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  32)             |/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  33)             |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  34)        __acquired
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  35)             |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  36)             .
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  37)           <hold>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  38)             .
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  39)             |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  40)        __release
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  41)             |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  42)          unlock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  43) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  44)   lock, unlock	- the regular lock functions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  45)   __*		- the hooks
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  46)   <> 		- states
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  47) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  48) With these hooks we provide the following statistics:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  49) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  50)  con-bounces
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  51) 	- number of lock contention that involved x-cpu data
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  52)  contentions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  53) 	- number of lock acquisitions that had to wait
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  54)  wait time
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  55)      min
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  56) 	- shortest (non-0) time we ever had to wait for a lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  57)      max
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  58) 	- longest time we ever had to wait for a lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  59)      total
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  60) 	- total time we spend waiting on this lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  61)      avg
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  62) 	- average time spent waiting on this lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  63)  acq-bounces
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  64) 	- number of lock acquisitions that involved x-cpu data
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  65)  acquisitions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  66) 	- number of times we took the lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  67)  hold time
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  68)      min
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  69) 	- shortest (non-0) time we ever held the lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  70)      max
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  71) 	- longest time we ever held the lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  72)      total
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  73) 	- total time this lock was held
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  74)      avg
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  75) 	- average time this lock was held
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  76) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  77) These numbers are gathered per lock class, per read/write state (when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  78) applicable).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  79) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  80) It also tracks 4 contention points per class. A contention point is a call site
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  81) that had to wait on lock acquisition.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  82) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  83) Configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  84) -------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  85) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  86) Lock statistics are enabled via CONFIG_LOCK_STAT.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  87) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  88) Usage
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  89) -----
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  90) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  91) Enable collection of statistics::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  92) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  93) 	# echo 1 >/proc/sys/kernel/lock_stat
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  94) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  95) Disable collection of statistics::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  96) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  97) 	# echo 0 >/proc/sys/kernel/lock_stat
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  98) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  99) Look at the current lock statistics::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101)   ( line numbers not part of actual output, done for clarity in the explanation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)     below )
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104)   # less /proc/lock_stat
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106)   01 lock_stat version 0.4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107)   02-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108)   03                              class name    con-bounces    contentions   waittime-min   waittime-max waittime-total   waittime-avg    acq-bounces   acquisitions   holdtime-min   holdtime-max holdtime-total   holdtime-avg
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109)   04-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110)   05
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111)   06                         &mm->mmap_sem-W:            46             84           0.26         939.10       16371.53         194.90          47291        2922365           0.16     2220301.69 17464026916.32        5975.99
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112)   07                         &mm->mmap_sem-R:            37            100           1.31      299502.61      325629.52        3256.30         212344       34316685           0.10        7744.91    95016910.20           2.77
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113)   08                         ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114)   09                           &mm->mmap_sem              1          [<ffffffff811502a7>] khugepaged_scan_mm_slot+0x57/0x280
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115)   10                           &mm->mmap_sem             96          [<ffffffff815351c4>] __do_page_fault+0x1d4/0x510
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116)   11                           &mm->mmap_sem             34          [<ffffffff81113d77>] vm_mmap_pgoff+0x87/0xd0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117)   12                           &mm->mmap_sem             17          [<ffffffff81127e71>] vm_munmap+0x41/0x80
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118)   13                         ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119)   14                           &mm->mmap_sem              1          [<ffffffff81046fda>] dup_mmap+0x2a/0x3f0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120)   15                           &mm->mmap_sem             60          [<ffffffff81129e29>] SyS_mprotect+0xe9/0x250
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121)   16                           &mm->mmap_sem             41          [<ffffffff815351c4>] __do_page_fault+0x1d4/0x510
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122)   17                           &mm->mmap_sem             68          [<ffffffff81113d77>] vm_mmap_pgoff+0x87/0xd0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123)   18
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124)   19.............................................................................................................................................................................................................................
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125)   20
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126)   21                         unix_table_lock:           110            112           0.21          49.24         163.91           1.46          21094          66312           0.12         624.42       31589.81           0.48
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127)   22                         ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128)   23                         unix_table_lock             45          [<ffffffff8150ad8e>] unix_create1+0x16e/0x1b0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129)   24                         unix_table_lock             47          [<ffffffff8150b111>] unix_release_sock+0x31/0x250
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130)   25                         unix_table_lock             15          [<ffffffff8150ca37>] unix_find_other+0x117/0x230
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131)   26                         unix_table_lock              5          [<ffffffff8150a09f>] unix_autobind+0x11f/0x1b0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132)   27                         ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133)   28                         unix_table_lock             39          [<ffffffff8150b111>] unix_release_sock+0x31/0x250
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134)   29                         unix_table_lock             49          [<ffffffff8150ad8e>] unix_create1+0x16e/0x1b0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135)   30                         unix_table_lock             20          [<ffffffff8150ca37>] unix_find_other+0x117/0x230
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136)   31                         unix_table_lock              4          [<ffffffff8150a09f>] unix_autobind+0x11f/0x1b0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) This excerpt shows the first two lock class statistics. Line 01 shows the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) output version - each time the format changes this will be updated. Line 02-04
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) show the header with column descriptions. Lines 05-18 and 20-31 show the actual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) statistics. These statistics come in two parts; the actual stats separated by a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) short separator (line 08, 13) from the contention points.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) Lines 09-12 show the first 4 recorded contention points (the code
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) which tries to get the lock) and lines 14-17 show the first 4 recorded
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) contended points (the lock holder). It is possible that the max
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) con-bounces point is missing in the statistics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) The first lock (05-18) is a read/write lock, and shows two lines above the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) short separator. The contention points don't match the column descriptors,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) they have two: contentions and [<IP>] symbol. The second set of contention
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) points are the points we're contending with.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) The integer part of the time values is in us.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) Dealing with nested locks, subclasses may appear::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159)   32...........................................................................................................................................................................................................................
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160)   33
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161)   34                               &rq->lock:       13128          13128           0.43         190.53      103881.26           7.91          97454        3453404           0.00         401.11    13224683.11           3.82
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)   35                               ---------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163)   36                               &rq->lock          645          [<ffffffff8103bfc4>] task_rq_lock+0x43/0x75
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164)   37                               &rq->lock          297          [<ffffffff8104ba65>] try_to_wake_up+0x127/0x25a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165)   38                               &rq->lock          360          [<ffffffff8103c4c5>] select_task_rq_fair+0x1f0/0x74a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166)   39                               &rq->lock          428          [<ffffffff81045f98>] scheduler_tick+0x46/0x1fb
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167)   40                               ---------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168)   41                               &rq->lock           77          [<ffffffff8103bfc4>] task_rq_lock+0x43/0x75
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)   42                               &rq->lock          174          [<ffffffff8104ba65>] try_to_wake_up+0x127/0x25a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170)   43                               &rq->lock         4715          [<ffffffff8103ed4b>] double_rq_lock+0x42/0x54
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171)   44                               &rq->lock          893          [<ffffffff81340524>] schedule+0x157/0x7b8
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172)   45
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173)   46...........................................................................................................................................................................................................................
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174)   47
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175)   48                             &rq->lock/1:        1526          11488           0.33         388.73      136294.31          11.86          21461          38404           0.00          37.93      109388.53           2.84
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176)   49                             -----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177)   50                             &rq->lock/1        11526          [<ffffffff8103ed58>] double_rq_lock+0x4f/0x54
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178)   51                             -----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179)   52                             &rq->lock/1         5645          [<ffffffff8103ed4b>] double_rq_lock+0x42/0x54
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180)   53                             &rq->lock/1         1224          [<ffffffff81340524>] schedule+0x157/0x7b8
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181)   54                             &rq->lock/1         4336          [<ffffffff8103ed58>] double_rq_lock+0x4f/0x54
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182)   55                             &rq->lock/1          181          [<ffffffff8104ba65>] try_to_wake_up+0x127/0x25a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) Line 48 shows statistics for the second subclass (/1) of &rq->lock class
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) (subclass starts from 0), since in this case, as line 50 suggests,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) double_rq_lock actually acquires a nested lock of two spinlocks.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) View the top contending locks::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190)   # grep : /proc/lock_stat | head
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) 			clockevents_lock:       2926159        2947636           0.15       46882.81  1784540466.34         605.41        3381345        3879161           0.00        2260.97    53178395.68          13.71
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192) 		     tick_broadcast_lock:        346460         346717           0.18        2257.43    39364622.71         113.54        3642919        4242696           0.00        2263.79    49173646.60          11.59
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) 		  &mapping->i_mmap_mutex:        203896         203899           3.36      645530.05 31767507988.39      155800.21        3361776        8893984           0.17        2254.15    14110121.02           1.59
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) 			       &rq->lock:        135014         136909           0.18         606.09      842160.68           6.15        1540728       10436146           0.00         728.72    17606683.41           1.69
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) 	       &(&zone->lru_lock)->rlock:         93000          94934           0.16          59.18      188253.78           1.98        1199912        3809894           0.15         391.40     3559518.81           0.93
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) 			 tasklist_lock-W:         40667          41130           0.23        1189.42      428980.51          10.43         270278         510106           0.16         653.51     3939674.91           7.72
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) 			 tasklist_lock-R:         21298          21305           0.20        1310.05      215511.12          10.12         186204         241258           0.14        1162.33     1179779.23           4.89
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) 			      rcu_node_1:         47656          49022           0.16         635.41      193616.41           3.95         844888        1865423           0.00         764.26     1656226.96           0.89
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199)        &(&dentry->d_lockref.lock)->rlock:         39791          40179           0.15        1302.08       88851.96           2.21        2790851       12527025           0.10        1910.75     3379714.27           0.27
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) 			      rcu_node_0:         29203          30064           0.16         786.55     1555573.00          51.74          88963         244254           0.00         398.87      428872.51           1.76
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) Clear the statistics::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204)   # echo 0 > /proc/lock_stat