^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) =============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) DM statistics
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) =============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) Device Mapper supports the collection of I/O statistics on user-defined
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) regions of a DM device. If no regions are defined no statistics are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7) collected so there isn't any performance impact. Only bio-based DM
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) devices are currently supported.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10) Each user-defined region specifies a starting sector, length and step.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) Individual statistics will be collected for each step-sized area within
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) the range specified.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) The I/O statistics counters for each step-sized area of a region are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) in the same format as `/sys/block/*/stat` or `/proc/diskstats` (see:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) Documentation/admin-guide/iostats.rst). But two extra counters (12 and 13) are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) provided: total time spent reading and writing. When the histogram
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) argument is used, the 14th parameter is reported that represents the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) histogram of latencies. All these counters may be accessed by sending
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) the @stats_print message to the appropriate DM device via dmsetup.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) The reported times are in milliseconds and the granularity depends on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) the kernel ticks. When the option precise_timestamps is used, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) reported times are in nanoseconds.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) Each region has a corresponding unique identifier, which we call a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) region_id, that is assigned when the region is created. The region_id
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) must be supplied when querying statistics about the region, deleting the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) region, etc. Unique region_ids enable multiple userspace programs to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30) request and process statistics for the same DM device without stepping
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) on each other's data.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) The creation of DM statistics will allocate memory via kmalloc or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) fallback to using vmalloc space. At most, 1/4 of the overall system
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) memory may be allocated by DM statistics. The admin can see how much
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) memory is used by reading:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) /sys/module/dm_mod/parameters/stats_current_allocated_bytes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) Messages
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) ========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) @stats_create <range> <step> [<number_of_optional_arguments> <optional_arguments>...] [<program_id> [<aux_data>]]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) Create a new region and return the region_id.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46) <range>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) "-"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) whole device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) "<start_sector>+<length>"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) a range of <length> 512-byte sectors
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) starting with <start_sector>.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) <step>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) "<area_size>"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) the range is subdivided into areas each containing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) <area_size> sectors.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) "/<number_of_areas>"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) the range is subdivided into the specified
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59) number of areas.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) <number_of_optional_arguments>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) The number of optional arguments
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) <optional_arguments>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) The following optional arguments are supported:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) precise_timestamps
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68) use precise timer with nanosecond resolution
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) instead of the "jiffies" variable. When this argument is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) used, the resulting times are in nanoseconds instead of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) milliseconds. Precise timestamps are a little bit slower
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) to obtain than jiffies-based timestamps.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73) histogram:n1,n2,n3,n4,...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) collect histogram of latencies. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) numbers n1, n2, etc are times that represent the boundaries
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) of the histogram. If precise_timestamps is not used, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) times are in milliseconds, otherwise they are in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) nanoseconds. For each range, the kernel will report the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79) number of requests that completed within this range. For
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) example, if we use "histogram:10,20,30", the kernel will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) report four numbers a:b:c:d. a is the number of requests
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) that took 0-10 ms to complete, b is the number of requests
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) that took 10-20 ms to complete, c is the number of requests
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) that took 20-30 ms to complete and d is the number of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) requests that took more than 30 ms to complete.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87) <program_id>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) An optional parameter. A name that uniquely identifies
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) the userspace owner of the range. This groups ranges together
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) so that userspace programs can identify the ranges they
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) created and ignore those created by others.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92) The kernel returns this string back in the output of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) @stats_list message, but it doesn't use it for anything else.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) If we omit the number of optional arguments, program id must not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) be a number, otherwise it would be interpreted as the number of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) optional arguments.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) <aux_data>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) An optional parameter. A word that provides auxiliary data
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) that is useful to the client program that created the range.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) The kernel returns this string back in the output of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) @stats_list message, but it doesn't use this value for anything.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) @stats_delete <region_id>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) Delete the region with the specified id.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) <region_id>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) region_id returned from @stats_create
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) @stats_clear <region_id>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) Clear all the counters except the in-flight i/o counters.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) <region_id>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) region_id returned from @stats_create
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) @stats_list [<program_id>]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) List all regions registered with @stats_create.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) <program_id>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) An optional parameter.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) If this parameter is specified, only matching regions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) are returned.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) If it is not specified, all regions are returned.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) Output format:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) <region_id>: <start_sector>+<length> <step> <program_id> <aux_data>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) precise_timestamps histogram:n1,n2,n3,...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) The strings "precise_timestamps" and "histogram" are printed only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) if they were specified when creating the region.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) @stats_print <region_id> [<starting_line> <number_of_lines>]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) Print counters for each step-sized area of a region.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) <region_id>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) region_id returned from @stats_create
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) <starting_line>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) The index of the starting line in the output.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) If omitted, all lines are returned.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) <number_of_lines>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) The number of lines to include in the output.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) If omitted, all lines are returned.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) Output format for each step-sized area of a region:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) <start_sector>+<length>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) counters
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) The first 11 counters have the same meaning as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) `/sys/block/*/stat or /proc/diskstats`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) Please refer to Documentation/admin-guide/iostats.rst for details.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) 1. the number of reads completed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) 2. the number of reads merged
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) 3. the number of sectors read
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) 4. the number of milliseconds spent reading
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) 5. the number of writes completed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) 6. the number of writes merged
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162) 7. the number of sectors written
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) 8. the number of milliseconds spent writing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) 9. the number of I/Os currently in progress
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165) 10. the number of milliseconds spent doing I/Os
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) 11. the weighted number of milliseconds spent doing I/Os
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) Additional counters:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) 12. the total time spent reading in milliseconds
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) 13. the total time spent writing in milliseconds
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) @stats_print_clear <region_id> [<starting_line> <number_of_lines>]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) Atomically print and then clear all the counters except the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) in-flight i/o counters. Useful when the client consuming the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) statistics does not want to lose any statistics (those updated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) between printing and clearing).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) <region_id>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) region_id returned from @stats_create
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) <starting_line>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) The index of the starting line in the output.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) If omitted, all lines are printed and then cleared.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) <number_of_lines>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) The number of lines to process.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) If omitted, all lines are printed and then cleared.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) @stats_set_aux <region_id> <aux_data>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) Store auxiliary data aux_data for the specified region.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) <region_id>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) region_id returned from @stats_create
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) <aux_data>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) The string that identifies data which is useful to the client
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) program that created the range. The kernel returns this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) string back in the output of @stats_list message, but it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) doesn't use this value for anything.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) Examples
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) ========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205) Subdivide the DM device 'vol' into 100 pieces and start collecting
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206) statistics on them::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208) dmsetup message vol 0 @stats_create - /100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210) Set the auxiliary data string to "foo bar baz" (the escape for each
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211) space must also be escaped, otherwise the shell will consume them)::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213) dmsetup message vol 0 @stats_set_aux 0 foo\\ bar\\ baz
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215) List the statistics::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217) dmsetup message vol 0 @stats_list
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) Print the statistics::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) dmsetup message vol 0 @stats_print 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223) Delete the statistics::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) dmsetup message vol 0 @stats_delete 0