^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) .. SPDX-License-Identifier: GPL-2.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) ========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4) VCPU Dispatch Statistics
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) ========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7) For Shared Processor LPARs, the POWER Hypervisor maintains a relatively
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) static mapping of the LPAR processors (vcpus) to physical processor
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) chips (representing the "home" node) and tries to always dispatch vcpus
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10) on their associated physical processor chip. However, under certain
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) scenarios, vcpus may be dispatched on a different processor chip (away
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) from its home node).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) /proc/powerpc/vcpudispatch_stats can be used to obtain statistics
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) related to the vcpu dispatch behavior. Writing '1' to this file enables
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) collecting the statistics, while writing '0' disables the statistics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) By default, the DTLB log for each vcpu is processed 50 times a second so
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) as not to miss any entries. This processing frequency can be changed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) through /proc/powerpc/vcpudispatch_stats_freq.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) The statistics themselves are available by reading the procfs file
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) /proc/powerpc/vcpudispatch_stats. Each line in the output corresponds to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) a vcpu as represented by the first field, followed by 8 numbers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) The first number corresponds to:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) 1. total vcpu dispatches since the beginning of statistics collection
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) The next 4 numbers represent vcpu dispatch dispersions:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) 2. number of times this vcpu was dispatched on the same processor as last
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) time
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) 3. number of times this vcpu was dispatched on a different processor core
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) as last time, but within the same chip
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) 4. number of times this vcpu was dispatched on a different chip
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) 5. number of times this vcpu was dispatches on a different socket/drawer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) (next numa boundary)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) The final 3 numbers represent statistics in relation to the home node of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) the vcpu:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) 6. number of times this vcpu was dispatched in its home node (chip)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) 7. number of times this vcpu was dispatched in a different node
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) 8. number of times this vcpu was dispatched in a node further away (numa
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) distance)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) An example output::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) $ sudo cat /proc/powerpc/vcpudispatch_stats
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) cpu0 6839 4126 2683 30 0 6821 18 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) cpu1 2515 1274 1229 12 0 2509 6 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) cpu2 2317 1198 1109 10 0 2312 5 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) cpu3 2259 1165 1088 6 0 2256 3 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) cpu4 2205 1143 1056 6 0 2202 3 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) cpu5 2165 1121 1038 6 0 2162 3 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) cpu6 2183 1127 1050 6 0 2180 3 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) cpu7 2193 1133 1052 8 0 2187 6 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) cpu8 2165 1115 1032 18 0 2156 9 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59) cpu9 2301 1252 1033 16 0 2293 8 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) cpu10 2197 1138 1041 18 0 2187 10 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) cpu11 2273 1185 1062 26 0 2260 13 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) cpu12 2186 1125 1043 18 0 2177 9 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63) cpu13 2161 1115 1030 16 0 2153 8 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) cpu14 2206 1153 1033 20 0 2196 10 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) cpu15 2163 1115 1032 16 0 2155 8 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) In the output above, for vcpu0, there have been 6839 dispatches since
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68) statistics were enabled. 4126 of those dispatches were on the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) physical cpu as the last time. 2683 were on a different core, but within
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) the same chip, while 30 dispatches were on a different chip compared to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) its last dispatch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73) Also, out of the total of 6839 dispatches, we see that there have been
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) 6821 dispatches on the vcpu's home node, while 18 dispatches were
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) outside its home node, on a neighbouring chip.