^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) ===============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) Lock Statistics
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) ===============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) What
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) ====
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) As the name suggests, it provides statistics on locks.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) Why
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) ===
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) Because things like lock contention can severely impact performance.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) How
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) ===
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) Lockdep already has hooks in the lock functions and maps lock instances to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) lock classes. We build on that (see Documentation/locking/lockdep-design.rst).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) The graph below shows the relation between the lock functions and the various
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) hooks therein::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) __acquire
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) lock _____
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) | \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) | __contended
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) | |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30) | <wait>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) | _______/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) |/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) __acquired
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) .
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) <hold>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) .
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) __release
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) unlock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) lock, unlock - the regular lock functions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) __* - the hooks
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46) <> - states
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) With these hooks we provide the following statistics:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) con-bounces
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) - number of lock contention that involved x-cpu data
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) contentions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) - number of lock acquisitions that had to wait
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) wait time
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) min
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) - shortest (non-0) time we ever had to wait for a lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) max
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) - longest time we ever had to wait for a lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59) total
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) - total time we spend waiting on this lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) avg
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) - average time spent waiting on this lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63) acq-bounces
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) - number of lock acquisitions that involved x-cpu data
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) acquisitions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) - number of times we took the lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) hold time
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68) min
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) - shortest (non-0) time we ever held the lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) max
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) - longest time we ever held the lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) total
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73) - total time this lock was held
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) avg
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) - average time this lock was held
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) These numbers are gathered per lock class, per read/write state (when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) applicable).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) It also tracks 4 contention points per class. A contention point is a call site
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) that had to wait on lock acquisition.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) Configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) -------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) Lock statistics are enabled via CONFIG_LOCK_STAT.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) Usage
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) -----
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) Enable collection of statistics::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) # echo 1 >/proc/sys/kernel/lock_stat
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) Disable collection of statistics::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) # echo 0 >/proc/sys/kernel/lock_stat
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) Look at the current lock statistics::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) ( line numbers not part of actual output, done for clarity in the explanation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) below )
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) # less /proc/lock_stat
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) 01 lock_stat version 0.4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) 02-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) 03 class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) 04-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) 05
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) 06 &mm->mmap_sem-W: 46 84 0.26 939.10 16371.53 194.90 47291 2922365 0.16 2220301.69 17464026916.32 5975.99
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) 07 &mm->mmap_sem-R: 37 100 1.31 299502.61 325629.52 3256.30 212344 34316685 0.10 7744.91 95016910.20 2.77
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) 08 ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) 09 &mm->mmap_sem 1 [<ffffffff811502a7>] khugepaged_scan_mm_slot+0x57/0x280
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) 10 &mm->mmap_sem 96 [<ffffffff815351c4>] __do_page_fault+0x1d4/0x510
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) 11 &mm->mmap_sem 34 [<ffffffff81113d77>] vm_mmap_pgoff+0x87/0xd0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) 12 &mm->mmap_sem 17 [<ffffffff81127e71>] vm_munmap+0x41/0x80
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) 13 ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) 14 &mm->mmap_sem 1 [<ffffffff81046fda>] dup_mmap+0x2a/0x3f0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) 15 &mm->mmap_sem 60 [<ffffffff81129e29>] SyS_mprotect+0xe9/0x250
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) 16 &mm->mmap_sem 41 [<ffffffff815351c4>] __do_page_fault+0x1d4/0x510
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) 17 &mm->mmap_sem 68 [<ffffffff81113d77>] vm_mmap_pgoff+0x87/0xd0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) 18
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) 19.............................................................................................................................................................................................................................
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) 20
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) 21 unix_table_lock: 110 112 0.21 49.24 163.91 1.46 21094 66312 0.12 624.42 31589.81 0.48
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) 22 ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) 23 unix_table_lock 45 [<ffffffff8150ad8e>] unix_create1+0x16e/0x1b0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) 24 unix_table_lock 47 [<ffffffff8150b111>] unix_release_sock+0x31/0x250
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) 25 unix_table_lock 15 [<ffffffff8150ca37>] unix_find_other+0x117/0x230
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) 26 unix_table_lock 5 [<ffffffff8150a09f>] unix_autobind+0x11f/0x1b0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) 27 ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) 28 unix_table_lock 39 [<ffffffff8150b111>] unix_release_sock+0x31/0x250
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) 29 unix_table_lock 49 [<ffffffff8150ad8e>] unix_create1+0x16e/0x1b0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) 30 unix_table_lock 20 [<ffffffff8150ca37>] unix_find_other+0x117/0x230
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) 31 unix_table_lock 4 [<ffffffff8150a09f>] unix_autobind+0x11f/0x1b0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) This excerpt shows the first two lock class statistics. Line 01 shows the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) output version - each time the format changes this will be updated. Line 02-04
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) show the header with column descriptions. Lines 05-18 and 20-31 show the actual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) statistics. These statistics come in two parts; the actual stats separated by a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) short separator (line 08, 13) from the contention points.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) Lines 09-12 show the first 4 recorded contention points (the code
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) which tries to get the lock) and lines 14-17 show the first 4 recorded
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) contended points (the lock holder). It is possible that the max
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) con-bounces point is missing in the statistics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) The first lock (05-18) is a read/write lock, and shows two lines above the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) short separator. The contention points don't match the column descriptors,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) they have two: contentions and [<IP>] symbol. The second set of contention
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) points are the points we're contending with.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) The integer part of the time values is in us.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) Dealing with nested locks, subclasses may appear::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) 32...........................................................................................................................................................................................................................
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) 33
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) 34 &rq->lock: 13128 13128 0.43 190.53 103881.26 7.91 97454 3453404 0.00 401.11 13224683.11 3.82
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162) 35 ---------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) 36 &rq->lock 645 [<ffffffff8103bfc4>] task_rq_lock+0x43/0x75
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) 37 &rq->lock 297 [<ffffffff8104ba65>] try_to_wake_up+0x127/0x25a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165) 38 &rq->lock 360 [<ffffffff8103c4c5>] select_task_rq_fair+0x1f0/0x74a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) 39 &rq->lock 428 [<ffffffff81045f98>] scheduler_tick+0x46/0x1fb
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167) 40 ---------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) 41 &rq->lock 77 [<ffffffff8103bfc4>] task_rq_lock+0x43/0x75
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169) 42 &rq->lock 174 [<ffffffff8104ba65>] try_to_wake_up+0x127/0x25a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) 43 &rq->lock 4715 [<ffffffff8103ed4b>] double_rq_lock+0x42/0x54
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) 44 &rq->lock 893 [<ffffffff81340524>] schedule+0x157/0x7b8
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) 45
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) 46...........................................................................................................................................................................................................................
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) 47
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) 48 &rq->lock/1: 1526 11488 0.33 388.73 136294.31 11.86 21461 38404 0.00 37.93 109388.53 2.84
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) 49 -----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) 50 &rq->lock/1 11526 [<ffffffff8103ed58>] double_rq_lock+0x4f/0x54
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) 51 -----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) 52 &rq->lock/1 5645 [<ffffffff8103ed4b>] double_rq_lock+0x42/0x54
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) 53 &rq->lock/1 1224 [<ffffffff81340524>] schedule+0x157/0x7b8
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) 54 &rq->lock/1 4336 [<ffffffff8103ed58>] double_rq_lock+0x4f/0x54
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) 55 &rq->lock/1 181 [<ffffffff8104ba65>] try_to_wake_up+0x127/0x25a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) Line 48 shows statistics for the second subclass (/1) of &rq->lock class
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) (subclass starts from 0), since in this case, as line 50 suggests,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) double_rq_lock actually acquires a nested lock of two spinlocks.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) View the top contending locks::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) # grep : /proc/lock_stat | head
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) clockevents_lock: 2926159 2947636 0.15 46882.81 1784540466.34 605.41 3381345 3879161 0.00 2260.97 53178395.68 13.71
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192) tick_broadcast_lock: 346460 346717 0.18 2257.43 39364622.71 113.54 3642919 4242696 0.00 2263.79 49173646.60 11.59
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) &mapping->i_mmap_mutex: 203896 203899 3.36 645530.05 31767507988.39 155800.21 3361776 8893984 0.17 2254.15 14110121.02 1.59
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) &rq->lock: 135014 136909 0.18 606.09 842160.68 6.15 1540728 10436146 0.00 728.72 17606683.41 1.69
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) &(&zone->lru_lock)->rlock: 93000 94934 0.16 59.18 188253.78 1.98 1199912 3809894 0.15 391.40 3559518.81 0.93
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) tasklist_lock-W: 40667 41130 0.23 1189.42 428980.51 10.43 270278 510106 0.16 653.51 3939674.91 7.72
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) tasklist_lock-R: 21298 21305 0.20 1310.05 215511.12 10.12 186204 241258 0.14 1162.33 1179779.23 4.89
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) rcu_node_1: 47656 49022 0.16 635.41 193616.41 3.95 844888 1865423 0.00 764.26 1656226.96 0.89
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) &(&dentry->d_lockref.lock)->rlock: 39791 40179 0.15 1302.08 88851.96 2.21 2790851 12527025 0.10 1910.75 3379714.27 0.27
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) rcu_node_0: 29203 30064 0.16 786.55 1555573.00 51.74 88963 244254 0.00 398.87 428872.51 1.76
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) Clear the statistics::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) # echo 0 > /proc/lock_stat