^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) .. SPDX-License-Identifier: GPL-2.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) ====================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4) Interface statistics
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) ====================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7) Overview
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) ========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10) This document is a guide to Linux network interface statistics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) There are three main sources of interface statistics in Linux:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) - standard interface statistics based on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) - protocol-specific statistics; and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) - driver-defined statistics available via ethtool.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) Standard interface statistics
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) -----------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) There are multiple interfaces to reach the standard statistics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) Most commonly used is the `ip` command from `iproute2`::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) $ ip -s -s link show dev ens4u1u1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) 6: ens4u1u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) link/ether 48:2a:e3:4c:b1:d1 brd ff:ff:ff:ff:ff:ff
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) RX: bytes packets errors dropped overrun mcast
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) 74327665117 69016965 0 0 0 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30) RX errors: length crc frame fifo missed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) 0 0 0 0 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) TX: bytes packets errors dropped carrier collsns
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) 21405556176 44608960 0 0 0 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) TX errors: aborted fifo window heartbeat transns
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) 0 0 0 0 128
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) altname enp58s0u1u1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) Note that `-s` has been specified twice to see all members of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) If `-s` is specified once the detailed errors won't be shown.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) `ip` supports JSON formatting via the `-j` option.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) Protocol-specific statistics
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) ----------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) Some of the interfaces used for configuring devices are also able
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) to report related statistics. For example ethtool interface used
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) to configure pause frames can report corresponding hardware counters::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) $ ethtool --include-statistics -a eth0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) Pause parameters for eth0:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) Autonegotiate: on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) RX: on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) TX: on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) Statistics:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) tx_pause_frames: 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) rx_pause_frames: 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) Driver-defined statistics
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) -------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63) Driver-defined ethtool statistics can be dumped using `ethtool -S $ifc`, e.g.::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) $ ethtool -S ens4u1u1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) NIC statistics:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) tx_single_collisions: 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68) tx_multi_collisions: 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) uAPIs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) =====
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73) procfs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) ------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) The historical `/proc/net/dev` text interface gives access to the list
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) of interfaces as well as their statistics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79) Note that even though this interface is using
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) internally it combines some of the fields.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) sysfs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) -----
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) Each device directory in sysfs contains a `statistics` directory (e.g.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87) `/sys/class/net/lo/statistics/`) with files corresponding to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) members of :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) This simple interface is convenient especially in constrained/embedded
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) environments without access to tools. However, it's inefficient when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92) reading multiple stats as it internally performs a full dump of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) and reports only the stat corresponding to the accessed file.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) Sysfs files are documented in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) `Documentation/ABI/testing/sysfs-class-net-statistics`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) netlink
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) -------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) `rtnetlink` (`NETLINK_ROUTE`) is the preferred method of accessing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` stats.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) Statistics are reported both in the responses to link information
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) requests (`RTM_GETLINK`) and statistic requests (`RTM_GETSTATS`,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) when `IFLA_STATS_LINK_64` bit is set in the `.filter_mask` of the request).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) ethtool
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) -------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) Ethtool IOCTL interface allows drivers to report implementation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) specific statistics. Historically it has also been used to report
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) statistics for which other APIs did not exist, like per-device-queue
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) statistics, or standard-based statistics (e.g. RFC 2863).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) Statistics and their string identifiers are retrieved separately.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) Identifiers via `ETHTOOL_GSTRINGS` with `string_set` set to `ETH_SS_STATS`,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) and values via `ETHTOOL_GSTATS`. User space should use `ETHTOOL_GDRVINFO`
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) to retrieve the number of statistics (`.n_stats`).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) ethtool-netlink
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) Ethtool netlink is a replacement for the older IOCTL interface.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) Protocol-related statistics can be requested in get commands by setting
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) the `ETHTOOL_FLAG_STATS` flag in `ETHTOOL_A_HEADER_FLAGS`. Currently
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) statistics are supported in the following commands:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) - `ETHTOOL_MSG_PAUSE_GET`
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) debugfs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) -------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) Some drivers expose extra statistics via `debugfs`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) struct rtnl_link_stats64
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) ========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) .. kernel-doc:: include/uapi/linux/if_link.h
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) :identifiers: rtnl_link_stats64
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) Notes for driver authors
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) ========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) Drivers should report all statistics which have a matching member in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` exclusively
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) via `.ndo_get_stats64`. Reporting such standard stats via ethtool
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) or debugfs will not be accepted.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) Drivers must ensure best possible compliance with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) Please note for example that detailed error statistics must be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) added into the general `rx_error` / `tx_error` counters.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) The `.ndo_get_stats64` callback can not sleep because of accesses
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) via `/proc/net/dev`. If driver may sleep when retrieving the statistics
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) from the device it should do so periodically asynchronously and only return
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) a recent copy from `.ndo_get_stats64`. Ethtool interrupt coalescing interface
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162) allows setting the frequency of refreshing statistics, if needed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) Retrieving ethtool statistics is a multi-syscall process, drivers are advised
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165) to keep the number of statistics constant to avoid race conditions with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) user space trying to read them.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) Statistics must persist across routine operations like bringing the interface
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169) down and up.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) Kernel-internal data structures
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) -------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) The following structures are internal to the kernel, their members are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) translated to netlink attributes when dumped. Drivers must not overwrite
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) the statistics they don't report with 0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) - ethtool_pause_stats()