^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) .. SPDX-License-Identifier: GPL-2.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) ===================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4) Linux Ethernet Bonding Driver HOWTO
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) ===================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7) Latest update: 27 April 2011
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) Initial release: Thomas Davis <tadavis at lbl.gov>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) Corrections, HA extensions: 2000/10/03-15:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13) - Willy Tarreau <willy at meta-x.org>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) - Constantine Gavrilov <const-g at xpert.com>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) - Chad N. Tindel <ctindel at ieee dot org>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) - Janice Girouard <girouard at us dot ibm dot com>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) - Jay Vosburgh <fubar at us dot ibm dot com>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) Reorganized and updated Feb 2005 by Jay Vosburgh
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) Added Sysfs information: 2006/04/24
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) - Mitch Williams <mitch.a.williams at intel.com>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) Introduction
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) ============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) The Linux bonding driver provides a method for aggregating
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) multiple network interfaces into a single logical "bonded" interface.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) The behavior of the bonded interfaces depends upon the mode; generally
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30) speaking, modes provide either hot standby or load balancing services.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) Additionally, link integrity monitoring may be performed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) The bonding driver originally came from Donald Becker's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) beowulf patches for kernel 2.0. It has changed quite a bit since, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) the original tools from extreme-linux and beowulf sites will not work
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) with this version of the driver.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) For new versions of the driver, updated userspace tools, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) who to ask for help, please follow the links at the end of this file.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) .. Table of Contents
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) 1. Bonding Driver Installation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) 2. Bonding Driver Options
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) 3. Configuring Bonding Devices
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) 3.1 Configuration with Sysconfig Support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) 3.1.1 Using DHCP with Sysconfig
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) 3.1.2 Configuring Multiple Bonds with Sysconfig
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) 3.2 Configuration with Initscripts Support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) 3.2.1 Using DHCP with Initscripts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) 3.2.2 Configuring Multiple Bonds with Initscripts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) 3.3 Configuring Bonding Manually with Ifenslave
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) 3.3.1 Configuring Multiple Bonds Manually
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) 3.4 Configuring Bonding Manually via Sysfs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) 3.5 Configuration with Interfaces Support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) 3.6 Overriding Configuration for Special Cases
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59) 3.7 Configuring LACP for 802.3ad mode in a more secure way
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) 4. Querying Bonding Configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) 4.1 Bonding Configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63) 4.2 Network Configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) 5. Switch Configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) 6. 802.1q VLAN Support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) 7. Link Monitoring
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) 7.1 ARP Monitor Operation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) 7.2 Configuring Multiple ARP Targets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) 7.3 MII Monitor Operation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) 8. Potential Trouble Sources
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) 8.1 Adventures in Routing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) 8.2 Ethernet Device Renaming
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) 8.3 Painfully Slow Or No Failed Link Detection By Miimon
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79) 9. SNMP agents
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) 10. Promiscuous mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) 11. Configuring Bonding for High Availability
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) 11.1 High Availability in a Single Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) 11.2 High Availability in a Multiple Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) 11.2.1 HA Bonding Mode Selection for Multiple Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87) 11.2.2 HA Link Monitoring for Multiple Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) 12. Configuring Bonding for Maximum Throughput
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) 12.1 Maximum Throughput in a Single Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) 12.1.1 MT Bonding Mode Selection for Single Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92) 12.1.2 MT Link Monitoring for Single Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) 12.2 Maximum Throughput in a Multiple Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) 12.2.1 MT Bonding Mode Selection for Multiple Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) 12.2.2 MT Link Monitoring for Multiple Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) 13. Switch Behavior Issues
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) 13.1 Link Establishment and Failover Delays
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) 13.2 Duplicated Incoming Packets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) 14. Hardware Specific Considerations
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) 14.1 IBM BladeCenter
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) 15. Frequently Asked Questions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) 16. Resources and Links
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) 1. Bonding Driver Installation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) ==============================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) Most popular distro kernels ship with the bonding driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) already available as a module. If your distro does not, or you
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) have need to compile bonding from source (e.g., configuring and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) installing a mainline kernel from kernel.org), you'll need to perform
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) the following steps:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) 1.1 Configure and build the kernel with bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) -----------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) The current version of the bonding driver is available in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) drivers/net/bonding subdirectory of the most recent kernel source
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) (which is available on http://kernel.org). Most users "rolling their
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) own" will want to use the most recent kernel from kernel.org.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) Configure kernel with "make menuconfig" (or "make xconfig" or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) "make config"), then select "Bonding driver support" in the "Network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) device support" section. It is recommended that you configure the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) driver as module since it is currently the only way to pass parameters
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) to the driver or configure more than one bonding device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) Build and install the new kernel and modules.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) 1.2 Bonding Control Utility
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) ---------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) It is recommended to configure bonding via iproute2 (netlink)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) or sysfs, the old ifenslave control utility is obsolete.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) 2. Bonding Driver Options
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) =========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) Options for the bonding driver are supplied as parameters to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) bonding module at load time, or are specified via sysfs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) Module options may be given as command line arguments to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) insmod or modprobe command, but are usually specified in either the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) ``/etc/modprobe.d/*.conf`` configuration files, or in a distro-specific
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) configuration file (some of which are detailed in the next section).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) Details on bonding support for sysfs is provided in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) "Configuring Bonding Manually via Sysfs" section, below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) The available bonding driver parameters are listed below. If a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) parameter is not specified the default value is used. When initially
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) configuring a bond, it is recommended "tail -f /var/log/messages" be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) run in a separate window to watch for bonding driver error messages.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) It is critical that either the miimon or arp_interval and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) arp_ip_target parameters be specified, otherwise serious network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) degradation will occur during link failures. Very few devices do not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162) support at least miimon, so there is really no reason not to use it.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) Options with textual values will accept either the text name
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165) or, for backwards compatibility, the option value. E.g.,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) "mode=802.3ad" and "mode=4" set the same mode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) The parameters are as follows:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) active_slave
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) Specifies the new active slave for modes that support it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) (active-backup, balance-alb and balance-tlb). Possible values
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) are the name of any currently enslaved interface, or an empty
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) string. If a name is given, the slave and its link must be up in order
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) to be selected as the new active slave. If an empty string is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) specified, the current active slave is cleared, and a new active
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) slave is selected automatically.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) Note that this is only available through the sysfs interface. No module
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) parameter by this name exists.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) The normal value of this option is the name of the currently
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) active slave, or the empty string if there is no active slave or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) the current mode does not use an active slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) ad_actor_sys_prio
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) In an AD system, this specifies the system priority. The allowed range
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) is 1 - 65535. If the value is not specified, it takes 65535 as the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) default value.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) This parameter has effect only in 802.3ad mode and is available through
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) SysFs interface.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) ad_actor_system
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) In an AD system, this specifies the mac-address for the actor in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) protocol packet exchanges (LACPDUs). The value cannot be a multicast
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) address. If the all-zeroes MAC is specified, bonding will internally
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201) use the MAC of the bond itself. It is preferred to have the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) local-admin bit set for this mac but driver does not enforce it. If
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) the value is not given then system defaults to using the masters'
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) mac address as actors' system address.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206) This parameter has effect only in 802.3ad mode and is available through
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207) SysFs interface.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) ad_select
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211) Specifies the 802.3ad aggregation selection logic to use. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) possible values and their effects are:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214) stable or 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216) The active aggregator is chosen by largest aggregate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217) bandwidth.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) Reselection of the active aggregator occurs only when all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220) slaves of the active aggregator are down or the active
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) aggregator has no slaves.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223) This is the default value.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) bandwidth or 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227) The active aggregator is chosen by largest aggregate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228) bandwidth. Reselection occurs if:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230) - A slave is added to or removed from the bond
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232) - Any slave's link state changes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234) - Any slave's 802.3ad association state changes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) - The bond's administrative state changes to up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238) count or 2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240) The active aggregator is chosen by the largest number of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241) ports (slaves). Reselection occurs as described under the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242) "bandwidth" setting, above.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) The bandwidth and count selection policies permit failover of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) 802.3ad aggregations when partial failure of the active aggregator
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246) occurs. This keeps the aggregator with the highest availability
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247) (either in bandwidth or in number of ports) active at all times.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249) This option was added in bonding version 3.4.0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251) ad_user_port_key
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253) In an AD system, the port-key has three parts as shown below -
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255) ===== ============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256) Bits Use
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257) ===== ============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258) 00 Duplex
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259) 01-05 Speed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260) 06-15 User-defined
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261) ===== ============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263) This defines the upper 10 bits of the port key. The values can be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 264) from 0 - 1023. If not given, the system defaults to 0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 265)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 266) This parameter has effect only in 802.3ad mode and is available through
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 267) SysFs interface.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 268)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 269) all_slaves_active
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 270)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 271) Specifies that duplicate frames (received on inactive ports) should be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 272) dropped (0) or delivered (1).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 273)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 274) Normally, bonding will drop duplicate frames (received on inactive
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 275) ports), which is desirable for most users. But there are some times
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 276) it is nice to allow duplicate frames to be delivered.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 277)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 278) The default value is 0 (drop duplicate frames received on inactive
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 279) ports).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 280)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 281) arp_interval
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 282)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 283) Specifies the ARP link monitoring frequency in milliseconds.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 284)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 285) The ARP monitor works by periodically checking the slave
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 286) devices to determine whether they have sent or received
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 287) traffic recently (the precise criteria depends upon the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 288) bonding mode, and the state of the slave). Regular traffic is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 289) generated via ARP probes issued for the addresses specified by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 290) the arp_ip_target option.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 291)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 292) This behavior can be modified by the arp_validate option,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 293) below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 294)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 295) If ARP monitoring is used in an etherchannel compatible mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 296) (modes 0 and 2), the switch should be configured in a mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 297) that evenly distributes packets across all links. If the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 298) switch is configured to distribute the packets in an XOR
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 299) fashion, all replies from the ARP targets will be received on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 300) the same link which could cause the other team members to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 301) fail. ARP monitoring should not be used in conjunction with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 302) miimon. A value of 0 disables ARP monitoring. The default
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 303) value is 0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 304)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 305) arp_ip_target
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 306)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 307) Specifies the IP addresses to use as ARP monitoring peers when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 308) arp_interval is > 0. These are the targets of the ARP request
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 309) sent to determine the health of the link to the targets.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 310) Specify these values in ddd.ddd.ddd.ddd format. Multiple IP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 311) addresses must be separated by a comma. At least one IP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 312) address must be given for ARP monitoring to function. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 313) maximum number of targets that can be specified is 16. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 314) default value is no IP addresses.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 315)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 316) arp_validate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 317)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 318) Specifies whether or not ARP probes and replies should be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 319) validated in any mode that supports arp monitoring, or whether
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 320) non-ARP traffic should be filtered (disregarded) for link
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 321) monitoring purposes.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 322)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 323) Possible values are:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 324)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 325) none or 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 326)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 327) No validation or filtering is performed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 328)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 329) active or 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 330)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 331) Validation is performed only for the active slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 332)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 333) backup or 2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 334)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 335) Validation is performed only for backup slaves.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 336)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 337) all or 3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 338)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 339) Validation is performed for all slaves.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 340)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 341) filter or 4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 342)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 343) Filtering is applied to all slaves. No validation is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 344) performed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 345)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 346) filter_active or 5
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 347)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 348) Filtering is applied to all slaves, validation is performed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 349) only for the active slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 350)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 351) filter_backup or 6
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 352)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 353) Filtering is applied to all slaves, validation is performed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 354) only for backup slaves.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 355)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 356) Validation:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 357)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 358) Enabling validation causes the ARP monitor to examine the incoming
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 359) ARP requests and replies, and only consider a slave to be up if it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 360) is receiving the appropriate ARP traffic.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 361)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 362) For an active slave, the validation checks ARP replies to confirm
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 363) that they were generated by an arp_ip_target. Since backup slaves
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 364) do not typically receive these replies, the validation performed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 365) for backup slaves is on the broadcast ARP request sent out via the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 366) active slave. It is possible that some switch or network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 367) configurations may result in situations wherein the backup slaves
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 368) do not receive the ARP requests; in such a situation, validation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 369) of backup slaves must be disabled.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 370)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 371) The validation of ARP requests on backup slaves is mainly helping
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 372) bonding to decide which slaves are more likely to work in case of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 373) the active slave failure, it doesn't really guarantee that the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 374) backup slave will work if it's selected as the next active slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 375)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 376) Validation is useful in network configurations in which multiple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 377) bonding hosts are concurrently issuing ARPs to one or more targets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 378) beyond a common switch. Should the link between the switch and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 379) target fail (but not the switch itself), the probe traffic
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 380) generated by the multiple bonding instances will fool the standard
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 381) ARP monitor into considering the links as still up. Use of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 382) validation can resolve this, as the ARP monitor will only consider
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 383) ARP requests and replies associated with its own instance of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 384) bonding.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 385)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 386) Filtering:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 387)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 388) Enabling filtering causes the ARP monitor to only use incoming ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 389) packets for link availability purposes. Arriving packets that are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 390) not ARPs are delivered normally, but do not count when determining
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 391) if a slave is available.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 392)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 393) Filtering operates by only considering the reception of ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 394) packets (any ARP packet, regardless of source or destination) when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 395) determining if a slave has received traffic for link availability
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 396) purposes.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 397)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 398) Filtering is useful in network configurations in which significant
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 399) levels of third party broadcast traffic would fool the standard
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 400) ARP monitor into considering the links as still up. Use of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 401) filtering can resolve this, as only ARP traffic is considered for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 402) link availability purposes.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 403)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 404) This option was added in bonding version 3.1.0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 405)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 406) arp_all_targets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 407)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 408) Specifies the quantity of arp_ip_targets that must be reachable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 409) in order for the ARP monitor to consider a slave as being up.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 410) This option affects only active-backup mode for slaves with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 411) arp_validation enabled.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 412)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 413) Possible values are:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 414)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 415) any or 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 416)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 417) consider the slave up only when any of the arp_ip_targets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 418) is reachable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 419)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 420) all or 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 421)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 422) consider the slave up only when all of the arp_ip_targets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 423) are reachable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 424)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 425) downdelay
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 426)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 427) Specifies the time, in milliseconds, to wait before disabling
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 428) a slave after a link failure has been detected. This option
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 429) is only valid for the miimon link monitor. The downdelay
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 430) value should be a multiple of the miimon value; if not, it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 431) will be rounded down to the nearest multiple. The default
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 432) value is 0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 433)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 434) fail_over_mac
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 435)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 436) Specifies whether active-backup mode should set all slaves to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 437) the same MAC address at enslavement (the traditional
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 438) behavior), or, when enabled, perform special handling of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 439) bond's MAC address in accordance with the selected policy.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 440)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 441) Possible values are:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 442)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 443) none or 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 444)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 445) This setting disables fail_over_mac, and causes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 446) bonding to set all slaves of an active-backup bond to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 447) the same MAC address at enslavement time. This is the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 448) default.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 449)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 450) active or 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 451)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 452) The "active" fail_over_mac policy indicates that the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 453) MAC address of the bond should always be the MAC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 454) address of the currently active slave. The MAC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 455) address of the slaves is not changed; instead, the MAC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 456) address of the bond changes during a failover.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 457)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 458) This policy is useful for devices that cannot ever
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 459) alter their MAC address, or for devices that refuse
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 460) incoming broadcasts with their own source MAC (which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 461) interferes with the ARP monitor).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 462)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 463) The down side of this policy is that every device on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 464) the network must be updated via gratuitous ARP,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 465) vs. just updating a switch or set of switches (which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 466) often takes place for any traffic, not just ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 467) traffic, if the switch snoops incoming traffic to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 468) update its tables) for the traditional method. If the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 469) gratuitous ARP is lost, communication may be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 470) disrupted.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 471)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 472) When this policy is used in conjunction with the mii
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 473) monitor, devices which assert link up prior to being
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 474) able to actually transmit and receive are particularly
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 475) susceptible to loss of the gratuitous ARP, and an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 476) appropriate updelay setting may be required.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 477)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 478) follow or 2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 479)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 480) The "follow" fail_over_mac policy causes the MAC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 481) address of the bond to be selected normally (normally
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 482) the MAC address of the first slave added to the bond).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 483) However, the second and subsequent slaves are not set
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 484) to this MAC address while they are in a backup role; a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 485) slave is programmed with the bond's MAC address at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 486) failover time (and the formerly active slave receives
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 487) the newly active slave's MAC address).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 488)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 489) This policy is useful for multiport devices that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 490) either become confused or incur a performance penalty
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 491) when multiple ports are programmed with the same MAC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 492) address.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 493)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 494)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 495) The default policy is none, unless the first slave cannot
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 496) change its MAC address, in which case the active policy is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 497) selected by default.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 498)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 499) This option may be modified via sysfs only when no slaves are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 500) present in the bond.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 501)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 502) This option was added in bonding version 3.2.0. The "follow"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 503) policy was added in bonding version 3.3.0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 504)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 505) lacp_rate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 506)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 507) Option specifying the rate in which we'll ask our link partner
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 508) to transmit LACPDU packets in 802.3ad mode. Possible values
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 509) are:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 510)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 511) slow or 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 512) Request partner to transmit LACPDUs every 30 seconds
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 513)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 514) fast or 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 515) Request partner to transmit LACPDUs every 1 second
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 516)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 517) The default is slow.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 518)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 519) max_bonds
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 520)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 521) Specifies the number of bonding devices to create for this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 522) instance of the bonding driver. E.g., if max_bonds is 3, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 523) the bonding driver is not already loaded, then bond0, bond1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 524) and bond2 will be created. The default value is 1. Specifying
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 525) a value of 0 will load bonding, but will not create any devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 526)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 527) miimon
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 528)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 529) Specifies the MII link monitoring frequency in milliseconds.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 530) This determines how often the link state of each slave is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 531) inspected for link failures. A value of zero disables MII
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 532) link monitoring. A value of 100 is a good starting point.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 533) The use_carrier option, below, affects how the link state is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 534) determined. See the High Availability section for additional
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 535) information. The default value is 0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 536)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 537) min_links
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 538)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 539) Specifies the minimum number of links that must be active before
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 540) asserting carrier. It is similar to the Cisco EtherChannel min-links
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 541) feature. This allows setting the minimum number of member ports that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 542) must be up (link-up state) before marking the bond device as up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 543) (carrier on). This is useful for situations where higher level services
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 544) such as clustering want to ensure a minimum number of low bandwidth
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 545) links are active before switchover. This option only affect 802.3ad
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 546) mode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 547)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 548) The default value is 0. This will cause carrier to be asserted (for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 549) 802.3ad mode) whenever there is an active aggregator, regardless of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 550) number of available links in that aggregator. Note that, because an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 551) aggregator cannot be active without at least one available link,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 552) setting this option to 0 or to 1 has the exact same effect.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 553)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 554) mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 555)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 556) Specifies one of the bonding policies. The default is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 557) balance-rr (round robin). Possible values are:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 558)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 559) balance-rr or 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 560)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 561) Round-robin policy: Transmit packets in sequential
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 562) order from the first available slave through the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 563) last. This mode provides load balancing and fault
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 564) tolerance.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 565)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 566) active-backup or 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 567)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 568) Active-backup policy: Only one slave in the bond is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 569) active. A different slave becomes active if, and only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 570) if, the active slave fails. The bond's MAC address is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 571) externally visible on only one port (network adapter)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 572) to avoid confusing the switch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 573)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 574) In bonding version 2.6.2 or later, when a failover
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 575) occurs in active-backup mode, bonding will issue one
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 576) or more gratuitous ARPs on the newly active slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 577) One gratuitous ARP is issued for the bonding master
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 578) interface and each VLAN interfaces configured above
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 579) it, provided that the interface has at least one IP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 580) address configured. Gratuitous ARPs issued for VLAN
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 581) interfaces are tagged with the appropriate VLAN id.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 582)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 583) This mode provides fault tolerance. The primary
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 584) option, documented below, affects the behavior of this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 585) mode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 586)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 587) balance-xor or 2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 588)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 589) XOR policy: Transmit based on the selected transmit
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 590) hash policy. The default policy is a simple [(source
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 591) MAC address XOR'd with destination MAC address XOR
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 592) packet type ID) modulo slave count]. Alternate transmit
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 593) policies may be selected via the xmit_hash_policy option,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 594) described below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 595)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 596) This mode provides load balancing and fault tolerance.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 597)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 598) broadcast or 3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 599)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 600) Broadcast policy: transmits everything on all slave
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 601) interfaces. This mode provides fault tolerance.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 602)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 603) 802.3ad or 4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 604)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 605) IEEE 802.3ad Dynamic link aggregation. Creates
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 606) aggregation groups that share the same speed and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 607) duplex settings. Utilizes all slaves in the active
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 608) aggregator according to the 802.3ad specification.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 609)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 610) Slave selection for outgoing traffic is done according
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 611) to the transmit hash policy, which may be changed from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 612) the default simple XOR policy via the xmit_hash_policy
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 613) option, documented below. Note that not all transmit
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 614) policies may be 802.3ad compliant, particularly in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 615) regards to the packet mis-ordering requirements of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 616) section 43.2.4 of the 802.3ad standard. Differing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 617) peer implementations will have varying tolerances for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 618) noncompliance.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 619)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 620) Prerequisites:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 621)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 622) 1. Ethtool support in the base drivers for retrieving
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 623) the speed and duplex of each slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 624)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 625) 2. A switch that supports IEEE 802.3ad Dynamic link
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 626) aggregation.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 627)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 628) Most switches will require some type of configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 629) to enable 802.3ad mode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 630)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 631) balance-tlb or 5
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 632)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 633) Adaptive transmit load balancing: channel bonding that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 634) does not require any special switch support.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 635)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 636) In tlb_dynamic_lb=1 mode; the outgoing traffic is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 637) distributed according to the current load (computed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 638) relative to the speed) on each slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 639)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 640) In tlb_dynamic_lb=0 mode; the load balancing based on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 641) current load is disabled and the load is distributed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 642) only using the hash distribution.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 643)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 644) Incoming traffic is received by the current slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 645) If the receiving slave fails, another slave takes over
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 646) the MAC address of the failed receiving slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 647)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 648) Prerequisite:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 649)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 650) Ethtool support in the base drivers for retrieving the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 651) speed of each slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 652)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 653) balance-alb or 6
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 654)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 655) Adaptive load balancing: includes balance-tlb plus
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 656) receive load balancing (rlb) for IPV4 traffic, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 657) does not require any special switch support. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 658) receive load balancing is achieved by ARP negotiation.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 659) The bonding driver intercepts the ARP Replies sent by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 660) the local system on their way out and overwrites the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 661) source hardware address with the unique hardware
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 662) address of one of the slaves in the bond such that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 663) different peers use different hardware addresses for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 664) the server.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 665)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 666) Receive traffic from connections created by the server
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 667) is also balanced. When the local system sends an ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 668) Request the bonding driver copies and saves the peer's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 669) IP information from the ARP packet. When the ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 670) Reply arrives from the peer, its hardware address is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 671) retrieved and the bonding driver initiates an ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 672) reply to this peer assigning it to one of the slaves
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 673) in the bond. A problematic outcome of using ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 674) negotiation for balancing is that each time that an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 675) ARP request is broadcast it uses the hardware address
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 676) of the bond. Hence, peers learn the hardware address
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 677) of the bond and the balancing of receive traffic
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 678) collapses to the current slave. This is handled by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 679) sending updates (ARP Replies) to all the peers with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 680) their individually assigned hardware address such that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 681) the traffic is redistributed. Receive traffic is also
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 682) redistributed when a new slave is added to the bond
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 683) and when an inactive slave is re-activated. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 684) receive load is distributed sequentially (round robin)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 685) among the group of highest speed slaves in the bond.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 686)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 687) When a link is reconnected or a new slave joins the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 688) bond the receive traffic is redistributed among all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 689) active slaves in the bond by initiating ARP Replies
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 690) with the selected MAC address to each of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 691) clients. The updelay parameter (detailed below) must
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 692) be set to a value equal or greater than the switch's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 693) forwarding delay so that the ARP Replies sent to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 694) peers will not be blocked by the switch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 695)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 696) Prerequisites:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 697)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 698) 1. Ethtool support in the base drivers for retrieving
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 699) the speed of each slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 700)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 701) 2. Base driver support for setting the hardware
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 702) address of a device while it is open. This is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 703) required so that there will always be one slave in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 704) team using the bond hardware address (the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 705) curr_active_slave) while having a unique hardware
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 706) address for each slave in the bond. If the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 707) curr_active_slave fails its hardware address is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 708) swapped with the new curr_active_slave that was
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 709) chosen.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 710)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 711) num_grat_arp,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 712) num_unsol_na
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 713)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 714) Specify the number of peer notifications (gratuitous ARPs and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 715) unsolicited IPv6 Neighbor Advertisements) to be issued after a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 716) failover event. As soon as the link is up on the new slave
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 717) (possibly immediately) a peer notification is sent on the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 718) bonding device and each VLAN sub-device. This is repeated at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 719) the rate specified by peer_notif_delay if the number is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 720) greater than 1.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 721)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 722) The valid range is 0 - 255; the default value is 1. These options
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 723) affect only the active-backup mode. These options were added for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 724) bonding versions 3.3.0 and 3.4.0 respectively.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 725)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 726) From Linux 3.0 and bonding version 3.7.1, these notifications
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 727) are generated by the ipv4 and ipv6 code and the numbers of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 728) repetitions cannot be set independently.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 729)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 730) packets_per_slave
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 731)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 732) Specify the number of packets to transmit through a slave before
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 733) moving to the next one. When set to 0 then a slave is chosen at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 734) random.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 735)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 736) The valid range is 0 - 65535; the default value is 1. This option
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 737) has effect only in balance-rr mode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 738)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 739) peer_notif_delay
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 740)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 741) Specify the delay, in milliseconds, between each peer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 742) notification (gratuitous ARP and unsolicited IPv6 Neighbor
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 743) Advertisement) when they are issued after a failover event.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 744) This delay should be a multiple of the link monitor interval
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 745) (arp_interval or miimon, whichever is active). The default
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 746) value is 0 which means to match the value of the link monitor
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 747) interval.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 748)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 749) primary
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 750)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 751) A string (eth0, eth2, etc) specifying which slave is the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 752) primary device. The specified device will always be the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 753) active slave while it is available. Only when the primary is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 754) off-line will alternate devices be used. This is useful when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 755) one slave is preferred over another, e.g., when one slave has
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 756) higher throughput than another.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 757)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 758) The primary option is only valid for active-backup(1),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 759) balance-tlb (5) and balance-alb (6) mode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 760)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 761) primary_reselect
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 762)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 763) Specifies the reselection policy for the primary slave. This
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 764) affects how the primary slave is chosen to become the active slave
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 765) when failure of the active slave or recovery of the primary slave
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 766) occurs. This option is designed to prevent flip-flopping between
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 767) the primary slave and other slaves. Possible values are:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 768)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 769) always or 0 (default)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 770)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 771) The primary slave becomes the active slave whenever it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 772) comes back up.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 773)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 774) better or 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 775)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 776) The primary slave becomes the active slave when it comes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 777) back up, if the speed and duplex of the primary slave is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 778) better than the speed and duplex of the current active
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 779) slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 780)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 781) failure or 2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 782)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 783) The primary slave becomes the active slave only if the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 784) current active slave fails and the primary slave is up.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 785)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 786) The primary_reselect setting is ignored in two cases:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 787)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 788) If no slaves are active, the first slave to recover is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 789) made the active slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 790)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 791) When initially enslaved, the primary slave is always made
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 792) the active slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 793)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 794) Changing the primary_reselect policy via sysfs will cause an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 795) immediate selection of the best active slave according to the new
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 796) policy. This may or may not result in a change of the active
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 797) slave, depending upon the circumstances.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 798)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 799) This option was added for bonding version 3.6.0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 800)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 801) tlb_dynamic_lb
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 802)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 803) Specifies if dynamic shuffling of flows is enabled in tlb
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 804) mode. The value has no effect on any other modes.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 805)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 806) The default behavior of tlb mode is to shuffle active flows across
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 807) slaves based on the load in that interval. This gives nice lb
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 808) characteristics but can cause packet reordering. If re-ordering is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 809) a concern use this variable to disable flow shuffling and rely on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 810) load balancing provided solely by the hash distribution.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 811) xmit-hash-policy can be used to select the appropriate hashing for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 812) the setup.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 813)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 814) The sysfs entry can be used to change the setting per bond device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 815) and the initial value is derived from the module parameter. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 816) sysfs entry is allowed to be changed only if the bond device is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 817) down.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 818)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 819) The default value is "1" that enables flow shuffling while value "0"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 820) disables it. This option was added in bonding driver 3.7.1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 821)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 822)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 823) updelay
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 824)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 825) Specifies the time, in milliseconds, to wait before enabling a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 826) slave after a link recovery has been detected. This option is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 827) only valid for the miimon link monitor. The updelay value
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 828) should be a multiple of the miimon value; if not, it will be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 829) rounded down to the nearest multiple. The default value is 0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 830)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 831) use_carrier
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 832)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 833) Specifies whether or not miimon should use MII or ETHTOOL
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 834) ioctls vs. netif_carrier_ok() to determine the link
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 835) status. The MII or ETHTOOL ioctls are less efficient and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 836) utilize a deprecated calling sequence within the kernel. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 837) netif_carrier_ok() relies on the device driver to maintain its
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 838) state with netif_carrier_on/off; at this writing, most, but
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 839) not all, device drivers support this facility.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 840)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 841) If bonding insists that the link is up when it should not be,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 842) it may be that your network device driver does not support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 843) netif_carrier_on/off. The default state for netif_carrier is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 844) "carrier on," so if a driver does not support netif_carrier,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 845) it will appear as if the link is always up. In this case,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 846) setting use_carrier to 0 will cause bonding to revert to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 847) MII / ETHTOOL ioctl method to determine the link state.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 848)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 849) A value of 1 enables the use of netif_carrier_ok(), a value of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 850) 0 will use the deprecated MII / ETHTOOL ioctls. The default
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 851) value is 1.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 852)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 853) xmit_hash_policy
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 854)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 855) Selects the transmit hash policy to use for slave selection in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 856) balance-xor, 802.3ad, and tlb modes. Possible values are:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 857)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 858) layer2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 859)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 860) Uses XOR of hardware MAC addresses and packet type ID
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 861) field to generate the hash. The formula is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 862)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 863) hash = source MAC XOR destination MAC XOR packet type ID
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 864) slave number = hash modulo slave count
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 865)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 866) This algorithm will place all traffic to a particular
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 867) network peer on the same slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 868)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 869) This algorithm is 802.3ad compliant.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 870)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 871) layer2+3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 872)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 873) This policy uses a combination of layer2 and layer3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 874) protocol information to generate the hash.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 875)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 876) Uses XOR of hardware MAC addresses and IP addresses to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 877) generate the hash. The formula is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 878)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 879) hash = source MAC XOR destination MAC XOR packet type ID
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 880) hash = hash XOR source IP XOR destination IP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 881) hash = hash XOR (hash RSHIFT 16)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 882) hash = hash XOR (hash RSHIFT 8)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 883) And then hash is reduced modulo slave count.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 884)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 885) If the protocol is IPv6 then the source and destination
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 886) addresses are first hashed using ipv6_addr_hash.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 887)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 888) This algorithm will place all traffic to a particular
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 889) network peer on the same slave. For non-IP traffic,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 890) the formula is the same as for the layer2 transmit
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 891) hash policy.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 892)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 893) This policy is intended to provide a more balanced
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 894) distribution of traffic than layer2 alone, especially
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 895) in environments where a layer3 gateway device is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 896) required to reach most destinations.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 897)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 898) This algorithm is 802.3ad compliant.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 899)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 900) layer3+4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 901)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 902) This policy uses upper layer protocol information,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 903) when available, to generate the hash. This allows for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 904) traffic to a particular network peer to span multiple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 905) slaves, although a single connection will not span
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 906) multiple slaves.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 907)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 908) The formula for unfragmented TCP and UDP packets is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 909)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 910) hash = source port, destination port (as in the header)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 911) hash = hash XOR source IP XOR destination IP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 912) hash = hash XOR (hash RSHIFT 16)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 913) hash = hash XOR (hash RSHIFT 8)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 914) And then hash is reduced modulo slave count.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 915)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 916) If the protocol is IPv6 then the source and destination
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 917) addresses are first hashed using ipv6_addr_hash.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 918)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 919) For fragmented TCP or UDP packets and all other IPv4 and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 920) IPv6 protocol traffic, the source and destination port
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 921) information is omitted. For non-IP traffic, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 922) formula is the same as for the layer2 transmit hash
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 923) policy.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 924)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 925) This algorithm is not fully 802.3ad compliant. A
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 926) single TCP or UDP conversation containing both
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 927) fragmented and unfragmented packets will see packets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 928) striped across two interfaces. This may result in out
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 929) of order delivery. Most traffic types will not meet
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 930) this criteria, as TCP rarely fragments traffic, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 931) most UDP traffic is not involved in extended
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 932) conversations. Other implementations of 802.3ad may
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 933) or may not tolerate this noncompliance.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 934)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 935) encap2+3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 936)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 937) This policy uses the same formula as layer2+3 but it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 938) relies on skb_flow_dissect to obtain the header fields
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 939) which might result in the use of inner headers if an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 940) encapsulation protocol is used. For example this will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 941) improve the performance for tunnel users because the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 942) packets will be distributed according to the encapsulated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 943) flows.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 944)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 945) encap3+4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 946)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 947) This policy uses the same formula as layer3+4 but it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 948) relies on skb_flow_dissect to obtain the header fields
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 949) which might result in the use of inner headers if an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 950) encapsulation protocol is used. For example this will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 951) improve the performance for tunnel users because the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 952) packets will be distributed according to the encapsulated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 953) flows.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 954)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 955) The default value is layer2. This option was added in bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 956) version 2.6.3. In earlier versions of bonding, this parameter
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 957) does not exist, and the layer2 policy is the only policy. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 958) layer2+3 value was added for bonding version 3.2.2.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 959)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 960) resend_igmp
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 961)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 962) Specifies the number of IGMP membership reports to be issued after
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 963) a failover event. One membership report is issued immediately after
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 964) the failover, subsequent packets are sent in each 200ms interval.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 965)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 966) The valid range is 0 - 255; the default value is 1. A value of 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 967) prevents the IGMP membership report from being issued in response
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 968) to the failover event.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 969)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 970) This option is useful for bonding modes balance-rr (0), active-backup
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 971) (1), balance-tlb (5) and balance-alb (6), in which a failover can
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 972) switch the IGMP traffic from one slave to another. Therefore a fresh
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 973) IGMP report must be issued to cause the switch to forward the incoming
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 974) IGMP traffic over the newly selected slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 975)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 976) This option was added for bonding version 3.7.0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 977)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 978) lp_interval
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 979)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 980) Specifies the number of seconds between instances where the bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 981) driver sends learning packets to each slaves peer switch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 982)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 983) The valid range is 1 - 0x7fffffff; the default value is 1. This Option
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 984) has effect only in balance-tlb and balance-alb modes.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 985)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 986) 3. Configuring Bonding Devices
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 987) ==============================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 988)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 989) You can configure bonding using either your distro's network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 990) initialization scripts, or manually using either iproute2 or the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 991) sysfs interface. Distros generally use one of three packages for the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 992) network initialization scripts: initscripts, sysconfig or interfaces.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 993) Recent versions of these packages have support for bonding, while older
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 994) versions do not.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 995)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 996) We will first describe the options for configuring bonding for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 997) distros using versions of initscripts, sysconfig and interfaces with full
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 998) or partial support for bonding, then provide information on enabling
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 999) bonding without support from the network initialization scripts (i.e.,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1000) older versions of initscripts or sysconfig).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1001)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1002) If you're unsure whether your distro uses sysconfig,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1003) initscripts or interfaces, or don't know if it's new enough, have no fear.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1004) Determining this is fairly straightforward.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1005)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1006) First, look for a file called interfaces in /etc/network directory.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1007) If this file is present in your system, then your system use interfaces. See
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1008) Configuration with Interfaces Support.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1009)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1010) Else, issue the command::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1011)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1012) $ rpm -qf /sbin/ifup
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1013)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1014) It will respond with a line of text starting with either
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1015) "initscripts" or "sysconfig," followed by some numbers. This is the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1016) package that provides your network initialization scripts.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1017)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1018) Next, to determine if your installation supports bonding,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1019) issue the command::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1020)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1021) $ grep ifenslave /sbin/ifup
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1022)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1023) If this returns any matches, then your initscripts or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1024) sysconfig has support for bonding.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1025)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1026) 3.1 Configuration with Sysconfig Support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1027) ----------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1028)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1029) This section applies to distros using a version of sysconfig
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1030) with bonding support, for example, SuSE Linux Enterprise Server 9.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1031)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1032) SuSE SLES 9's networking configuration system does support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1033) bonding, however, at this writing, the YaST system configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1034) front end does not provide any means to work with bonding devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1035) Bonding devices can be managed by hand, however, as follows.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1036)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1037) First, if they have not already been configured, configure the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1038) slave devices. On SLES 9, this is most easily done by running the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1039) yast2 sysconfig configuration utility. The goal is for to create an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1040) ifcfg-id file for each slave device. The simplest way to accomplish
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1041) this is to configure the devices for DHCP (this is only to get the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1042) file ifcfg-id file created; see below for some issues with DHCP). The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1043) name of the configuration file for each device will be of the form::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1044)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1045) ifcfg-id-xx:xx:xx:xx:xx:xx
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1046)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1047) Where the "xx" portion will be replaced with the digits from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1048) the device's permanent MAC address.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1049)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1050) Once the set of ifcfg-id-xx:xx:xx:xx:xx:xx files has been
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1051) created, it is necessary to edit the configuration files for the slave
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1052) devices (the MAC addresses correspond to those of the slave devices).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1053) Before editing, the file will contain multiple lines, and will look
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1054) something like this::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1055)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1056) BOOTPROTO='dhcp'
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1057) STARTMODE='on'
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1058) USERCTL='no'
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1059) UNIQUE='XNzu.WeZGOGF+4wE'
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1060) _nm_name='bus-pci-0001:61:01.0'
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1061)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1062) Change the BOOTPROTO and STARTMODE lines to the following::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1063)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1064) BOOTPROTO='none'
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1065) STARTMODE='off'
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1066)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1067) Do not alter the UNIQUE or _nm_name lines. Remove any other
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1068) lines (USERCTL, etc).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1069)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1070) Once the ifcfg-id-xx:xx:xx:xx:xx:xx files have been modified,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1071) it's time to create the configuration file for the bonding device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1072) itself. This file is named ifcfg-bondX, where X is the number of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1073) bonding device to create, starting at 0. The first such file is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1074) ifcfg-bond0, the second is ifcfg-bond1, and so on. The sysconfig
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1075) network configuration system will correctly start multiple instances
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1076) of bonding.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1077)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1078) The contents of the ifcfg-bondX file is as follows::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1079)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1080) BOOTPROTO="static"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1081) BROADCAST="10.0.2.255"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1082) IPADDR="10.0.2.10"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1083) NETMASK="255.255.0.0"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1084) NETWORK="10.0.2.0"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1085) REMOTE_IPADDR=""
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1086) STARTMODE="onboot"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1087) BONDING_MASTER="yes"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1088) BONDING_MODULE_OPTS="mode=active-backup miimon=100"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1089) BONDING_SLAVE0="eth0"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1090) BONDING_SLAVE1="bus-pci-0000:06:08.1"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1091)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1092) Replace the sample BROADCAST, IPADDR, NETMASK and NETWORK
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1093) values with the appropriate values for your network.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1094)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1095) The STARTMODE specifies when the device is brought online.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1096) The possible values are:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1097)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1098) ======== ======================================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1099) onboot The device is started at boot time. If you're not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1100) sure, this is probably what you want.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1101)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1102) manual The device is started only when ifup is called
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1103) manually. Bonding devices may be configured this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1104) way if you do not wish them to start automatically
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1105) at boot for some reason.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1106)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1107) hotplug The device is started by a hotplug event. This is not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1108) a valid choice for a bonding device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1109)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1110) off or The device configuration is ignored.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1111) ignore
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1112) ======== ======================================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1113)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1114) The line BONDING_MASTER='yes' indicates that the device is a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1115) bonding master device. The only useful value is "yes."
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1116)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1117) The contents of BONDING_MODULE_OPTS are supplied to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1118) instance of the bonding module for this device. Specify the options
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1119) for the bonding mode, link monitoring, and so on here. Do not include
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1120) the max_bonds bonding parameter; this will confuse the configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1121) system if you have multiple bonding devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1122)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1123) Finally, supply one BONDING_SLAVEn="slave device" for each
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1124) slave. where "n" is an increasing value, one for each slave. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1125) "slave device" is either an interface name, e.g., "eth0", or a device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1126) specifier for the network device. The interface name is easier to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1127) find, but the ethN names are subject to change at boot time if, e.g.,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1128) a device early in the sequence has failed. The device specifiers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1129) (bus-pci-0000:06:08.1 in the example above) specify the physical
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1130) network device, and will not change unless the device's bus location
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1131) changes (for example, it is moved from one PCI slot to another). The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1132) example above uses one of each type for demonstration purposes; most
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1133) configurations will choose one or the other for all slave devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1134)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1135) When all configuration files have been modified or created,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1136) networking must be restarted for the configuration changes to take
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1137) effect. This can be accomplished via the following::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1138)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1139) # /etc/init.d/network restart
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1140)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1141) Note that the network control script (/sbin/ifdown) will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1142) remove the bonding module as part of the network shutdown processing,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1143) so it is not necessary to remove the module by hand if, e.g., the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1144) module parameters have changed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1145)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1146) Also, at this writing, YaST/YaST2 will not manage bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1147) devices (they do not show bonding interfaces on its list of network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1148) devices). It is necessary to edit the configuration file by hand to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1149) change the bonding configuration.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1150)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1151) Additional general options and details of the ifcfg file
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1152) format can be found in an example ifcfg template file::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1153)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1154) /etc/sysconfig/network/ifcfg.template
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1155)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1156) Note that the template does not document the various ``BONDING_*``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1157) settings described above, but does describe many of the other options.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1158)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1159) 3.1.1 Using DHCP with Sysconfig
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1160) -------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1161)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1162) Under sysconfig, configuring a device with BOOTPROTO='dhcp'
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1163) will cause it to query DHCP for its IP address information. At this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1164) writing, this does not function for bonding devices; the scripts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1165) attempt to obtain the device address from DHCP prior to adding any of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1166) the slave devices. Without active slaves, the DHCP requests are not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1167) sent to the network.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1168)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1169) 3.1.2 Configuring Multiple Bonds with Sysconfig
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1170) -----------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1171)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1172) The sysconfig network initialization system is capable of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1173) handling multiple bonding devices. All that is necessary is for each
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1174) bonding instance to have an appropriately configured ifcfg-bondX file
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1175) (as described above). Do not specify the "max_bonds" parameter to any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1176) instance of bonding, as this will confuse sysconfig. If you require
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1177) multiple bonding devices with identical parameters, create multiple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1178) ifcfg-bondX files.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1179)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1180) Because the sysconfig scripts supply the bonding module
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1181) options in the ifcfg-bondX file, it is not necessary to add them to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1182) the system ``/etc/modules.d/*.conf`` configuration files.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1183)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1184) 3.2 Configuration with Initscripts Support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1185) ------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1186)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1187) This section applies to distros using a recent version of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1188) initscripts with bonding support, for example, Red Hat Enterprise Linux
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1189) version 3 or later, Fedora, etc. On these systems, the network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1190) initialization scripts have knowledge of bonding, and can be configured to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1191) control bonding devices. Note that older versions of the initscripts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1192) package have lower levels of support for bonding; this will be noted where
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1193) applicable.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1194)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1195) These distros will not automatically load the network adapter
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1196) driver unless the ethX device is configured with an IP address.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1197) Because of this constraint, users must manually configure a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1198) network-script file for all physical adapters that will be members of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1199) a bondX link. Network script files are located in the directory:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1200)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1201) /etc/sysconfig/network-scripts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1202)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1203) The file name must be prefixed with "ifcfg-eth" and suffixed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1204) with the adapter's physical adapter number. For example, the script
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1205) for eth0 would be named /etc/sysconfig/network-scripts/ifcfg-eth0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1206) Place the following text in the file::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1207)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1208) DEVICE=eth0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1209) USERCTL=no
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1210) ONBOOT=yes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1211) MASTER=bond0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1212) SLAVE=yes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1213) BOOTPROTO=none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1214)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1215) The DEVICE= line will be different for every ethX device and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1216) must correspond with the name of the file, i.e., ifcfg-eth1 must have
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1217) a device line of DEVICE=eth1. The setting of the MASTER= line will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1218) also depend on the final bonding interface name chosen for your bond.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1219) As with other network devices, these typically start at 0, and go up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1220) one for each device, i.e., the first bonding instance is bond0, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1221) second is bond1, and so on.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1222)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1223) Next, create a bond network script. The file name for this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1224) script will be /etc/sysconfig/network-scripts/ifcfg-bondX where X is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1225) the number of the bond. For bond0 the file is named "ifcfg-bond0",
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1226) for bond1 it is named "ifcfg-bond1", and so on. Within that file,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1227) place the following text::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1228)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1229) DEVICE=bond0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1230) IPADDR=192.168.1.1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1231) NETMASK=255.255.255.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1232) NETWORK=192.168.1.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1233) BROADCAST=192.168.1.255
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1234) ONBOOT=yes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1235) BOOTPROTO=none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1236) USERCTL=no
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1237)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1238) Be sure to change the networking specific lines (IPADDR,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1239) NETMASK, NETWORK and BROADCAST) to match your network configuration.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1240)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1241) For later versions of initscripts, such as that found with Fedora
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1242) 7 (or later) and Red Hat Enterprise Linux version 5 (or later), it is possible,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1243) and, indeed, preferable, to specify the bonding options in the ifcfg-bond0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1244) file, e.g. a line of the format::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1245)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1246) BONDING_OPTS="mode=active-backup arp_interval=60 arp_ip_target=192.168.1.254"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1247)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1248) will configure the bond with the specified options. The options
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1249) specified in BONDING_OPTS are identical to the bonding module parameters
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1250) except for the arp_ip_target field when using versions of initscripts older
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1251) than and 8.57 (Fedora 8) and 8.45.19 (Red Hat Enterprise Linux 5.2). When
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1252) using older versions each target should be included as a separate option and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1253) should be preceded by a '+' to indicate it should be added to the list of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1254) queried targets, e.g.,::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1255)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1256) arp_ip_target=+192.168.1.1 arp_ip_target=+192.168.1.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1257)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1258) is the proper syntax to specify multiple targets. When specifying
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1259) options via BONDING_OPTS, it is not necessary to edit
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1260) ``/etc/modprobe.d/*.conf``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1261)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1262) For even older versions of initscripts that do not support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1263) BONDING_OPTS, it is necessary to edit /etc/modprobe.d/*.conf, depending upon
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1264) your distro) to load the bonding module with your desired options when the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1265) bond0 interface is brought up. The following lines in /etc/modprobe.d/*.conf
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1266) will load the bonding module, and select its options:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1267)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1268) alias bond0 bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1269) options bond0 mode=balance-alb miimon=100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1270)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1271) Replace the sample parameters with the appropriate set of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1272) options for your configuration.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1273)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1274) Finally run "/etc/rc.d/init.d/network restart" as root. This
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1275) will restart the networking subsystem and your bond link should be now
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1276) up and running.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1277)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1278) 3.2.1 Using DHCP with Initscripts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1279) ---------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1280)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1281) Recent versions of initscripts (the versions supplied with Fedora
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1282) Core 3 and Red Hat Enterprise Linux 4, or later versions, are reported to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1283) work) have support for assigning IP information to bonding devices via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1284) DHCP.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1285)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1286) To configure bonding for DHCP, configure it as described
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1287) above, except replace the line "BOOTPROTO=none" with "BOOTPROTO=dhcp"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1288) and add a line consisting of "TYPE=Bonding". Note that the TYPE value
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1289) is case sensitive.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1290)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1291) 3.2.2 Configuring Multiple Bonds with Initscripts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1292) -------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1293)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1294) Initscripts packages that are included with Fedora 7 and Red Hat
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1295) Enterprise Linux 5 support multiple bonding interfaces by simply
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1296) specifying the appropriate BONDING_OPTS= in ifcfg-bondX where X is the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1297) number of the bond. This support requires sysfs support in the kernel,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1298) and a bonding driver of version 3.0.0 or later. Other configurations may
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1299) not support this method for specifying multiple bonding interfaces; for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1300) those instances, see the "Configuring Multiple Bonds Manually" section,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1301) below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1302)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1303) 3.3 Configuring Bonding Manually with iproute2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1304) -----------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1305)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1306) This section applies to distros whose network initialization
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1307) scripts (the sysconfig or initscripts package) do not have specific
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1308) knowledge of bonding. One such distro is SuSE Linux Enterprise Server
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1309) version 8.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1310)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1311) The general method for these systems is to place the bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1312) module parameters into a config file in /etc/modprobe.d/ (as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1313) appropriate for the installed distro), then add modprobe and/or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1314) `ip link` commands to the system's global init script. The name of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1315) the global init script differs; for sysconfig, it is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1316) /etc/init.d/boot.local and for initscripts it is /etc/rc.d/rc.local.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1317)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1318) For example, if you wanted to make a simple bond of two e100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1319) devices (presumed to be eth0 and eth1), and have it persist across
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1320) reboots, edit the appropriate file (/etc/init.d/boot.local or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1321) /etc/rc.d/rc.local), and add the following::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1322)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1323) modprobe bonding mode=balance-alb miimon=100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1324) modprobe e100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1325) ifconfig bond0 192.168.1.1 netmask 255.255.255.0 up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1326) ip link set eth0 master bond0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1327) ip link set eth1 master bond0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1328)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1329) Replace the example bonding module parameters and bond0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1330) network configuration (IP address, netmask, etc) with the appropriate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1331) values for your configuration.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1332)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1333) Unfortunately, this method will not provide support for the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1334) ifup and ifdown scripts on the bond devices. To reload the bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1335) configuration, it is necessary to run the initialization script, e.g.,::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1336)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1337) # /etc/init.d/boot.local
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1338)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1339) or::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1340)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1341) # /etc/rc.d/rc.local
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1342)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1343) It may be desirable in such a case to create a separate script
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1344) which only initializes the bonding configuration, then call that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1345) separate script from within boot.local. This allows for bonding to be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1346) enabled without re-running the entire global init script.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1347)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1348) To shut down the bonding devices, it is necessary to first
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1349) mark the bonding device itself as being down, then remove the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1350) appropriate device driver modules. For our example above, you can do
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1351) the following::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1352)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1353) # ifconfig bond0 down
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1354) # rmmod bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1355) # rmmod e100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1356)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1357) Again, for convenience, it may be desirable to create a script
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1358) with these commands.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1359)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1360)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1361) 3.3.1 Configuring Multiple Bonds Manually
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1362) -----------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1363)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1364) This section contains information on configuring multiple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1365) bonding devices with differing options for those systems whose network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1366) initialization scripts lack support for configuring multiple bonds.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1367)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1368) If you require multiple bonding devices, but all with the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1369) options, you may wish to use the "max_bonds" module parameter,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1370) documented above.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1371)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1372) To create multiple bonding devices with differing options, it is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1373) preferable to use bonding parameters exported by sysfs, documented in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1374) section below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1375)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1376) For versions of bonding without sysfs support, the only means to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1377) provide multiple instances of bonding with differing options is to load
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1378) the bonding driver multiple times. Note that current versions of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1379) sysconfig network initialization scripts handle this automatically; if
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1380) your distro uses these scripts, no special action is needed. See the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1381) section Configuring Bonding Devices, above, if you're not sure about your
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1382) network initialization scripts.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1383)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1384) To load multiple instances of the module, it is necessary to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1385) specify a different name for each instance (the module loading system
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1386) requires that every loaded module, even multiple instances of the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1387) module, have a unique name). This is accomplished by supplying multiple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1388) sets of bonding options in ``/etc/modprobe.d/*.conf``, for example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1389)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1390) alias bond0 bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1391) options bond0 -o bond0 mode=balance-rr miimon=100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1392)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1393) alias bond1 bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1394) options bond1 -o bond1 mode=balance-alb miimon=50
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1395)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1396) will load the bonding module two times. The first instance is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1397) named "bond0" and creates the bond0 device in balance-rr mode with an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1398) miimon of 100. The second instance is named "bond1" and creates the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1399) bond1 device in balance-alb mode with an miimon of 50.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1400)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1401) In some circumstances (typically with older distributions),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1402) the above does not work, and the second bonding instance never sees
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1403) its options. In that case, the second options line can be substituted
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1404) as follows::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1405)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1406) install bond1 /sbin/modprobe --ignore-install bonding -o bond1 \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1407) mode=balance-alb miimon=50
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1408)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1409) This may be repeated any number of times, specifying a new and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1410) unique name in place of bond1 for each subsequent instance.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1411)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1412) It has been observed that some Red Hat supplied kernels are unable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1413) to rename modules at load time (the "-o bond1" part). Attempts to pass
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1414) that option to modprobe will produce an "Operation not permitted" error.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1415) This has been reported on some Fedora Core kernels, and has been seen on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1416) RHEL 4 as well. On kernels exhibiting this problem, it will be impossible
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1417) to configure multiple bonds with differing parameters (as they are older
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1418) kernels, and also lack sysfs support).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1419)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1420) 3.4 Configuring Bonding Manually via Sysfs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1421) ------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1422)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1423) Starting with version 3.0.0, Channel Bonding may be configured
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1424) via the sysfs interface. This interface allows dynamic configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1425) of all bonds in the system without unloading the module. It also
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1426) allows for adding and removing bonds at runtime. Ifenslave is no
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1427) longer required, though it is still supported.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1428)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1429) Use of the sysfs interface allows you to use multiple bonds
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1430) with different configurations without having to reload the module.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1431) It also allows you to use multiple, differently configured bonds when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1432) bonding is compiled into the kernel.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1433)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1434) You must have the sysfs filesystem mounted to configure
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1435) bonding this way. The examples in this document assume that you
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1436) are using the standard mount point for sysfs, e.g. /sys. If your
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1437) sysfs filesystem is mounted elsewhere, you will need to adjust the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1438) example paths accordingly.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1439)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1440) Creating and Destroying Bonds
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1441) -----------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1442) To add a new bond foo::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1443)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1444) # echo +foo > /sys/class/net/bonding_masters
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1445)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1446) To remove an existing bond bar::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1447)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1448) # echo -bar > /sys/class/net/bonding_masters
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1449)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1450) To show all existing bonds::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1451)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1452) # cat /sys/class/net/bonding_masters
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1453)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1454) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1455)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1456) due to 4K size limitation of sysfs files, this list may be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1457) truncated if you have more than a few hundred bonds. This is unlikely
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1458) to occur under normal operating conditions.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1459)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1460) Adding and Removing Slaves
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1461) --------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1462) Interfaces may be enslaved to a bond using the file
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1463) /sys/class/net/<bond>/bonding/slaves. The semantics for this file
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1464) are the same as for the bonding_masters file.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1465)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1466) To enslave interface eth0 to bond bond0::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1467)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1468) # ifconfig bond0 up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1469) # echo +eth0 > /sys/class/net/bond0/bonding/slaves
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1470)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1471) To free slave eth0 from bond bond0::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1472)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1473) # echo -eth0 > /sys/class/net/bond0/bonding/slaves
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1474)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1475) When an interface is enslaved to a bond, symlinks between the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1476) two are created in the sysfs filesystem. In this case, you would get
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1477) /sys/class/net/bond0/slave_eth0 pointing to /sys/class/net/eth0, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1478) /sys/class/net/eth0/master pointing to /sys/class/net/bond0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1479)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1480) This means that you can tell quickly whether or not an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1481) interface is enslaved by looking for the master symlink. Thus:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1482) # echo -eth0 > /sys/class/net/eth0/master/bonding/slaves
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1483) will free eth0 from whatever bond it is enslaved to, regardless of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1484) the name of the bond interface.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1485)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1486) Changing a Bond's Configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1487) -------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1488) Each bond may be configured individually by manipulating the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1489) files located in /sys/class/net/<bond name>/bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1490)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1491) The names of these files correspond directly with the command-
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1492) line parameters described elsewhere in this file, and, with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1493) exception of arp_ip_target, they accept the same values. To see the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1494) current setting, simply cat the appropriate file.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1495)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1496) A few examples will be given here; for specific usage
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1497) guidelines for each parameter, see the appropriate section in this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1498) document.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1499)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1500) To configure bond0 for balance-alb mode::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1501)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1502) # ifconfig bond0 down
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1503) # echo 6 > /sys/class/net/bond0/bonding/mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1504) - or -
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1505) # echo balance-alb > /sys/class/net/bond0/bonding/mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1506)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1507) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1508)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1509) The bond interface must be down before the mode can be changed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1510)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1511) To enable MII monitoring on bond0 with a 1 second interval::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1512)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1513) # echo 1000 > /sys/class/net/bond0/bonding/miimon
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1514)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1515) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1516)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1517) If ARP monitoring is enabled, it will disabled when MII
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1518) monitoring is enabled, and vice-versa.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1519)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1520) To add ARP targets::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1521)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1522) # echo +192.168.0.100 > /sys/class/net/bond0/bonding/arp_ip_target
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1523) # echo +192.168.0.101 > /sys/class/net/bond0/bonding/arp_ip_target
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1524)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1525) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1526)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1527) up to 16 target addresses may be specified.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1528)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1529) To remove an ARP target::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1530)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1531) # echo -192.168.0.100 > /sys/class/net/bond0/bonding/arp_ip_target
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1532)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1533) To configure the interval between learning packet transmits::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1534)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1535) # echo 12 > /sys/class/net/bond0/bonding/lp_interval
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1536)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1537) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1538)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1539) the lp_interval is the number of seconds between instances where
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1540) the bonding driver sends learning packets to each slaves peer switch. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1541) default interval is 1 second.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1542)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1543) Example Configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1544) ---------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1545) We begin with the same example that is shown in section 3.3,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1546) executed with sysfs, and without using ifenslave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1547)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1548) To make a simple bond of two e100 devices (presumed to be eth0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1549) and eth1), and have it persist across reboots, edit the appropriate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1550) file (/etc/init.d/boot.local or /etc/rc.d/rc.local), and add the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1551) following::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1552)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1553) modprobe bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1554) modprobe e100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1555) echo balance-alb > /sys/class/net/bond0/bonding/mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1556) ifconfig bond0 192.168.1.1 netmask 255.255.255.0 up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1557) echo 100 > /sys/class/net/bond0/bonding/miimon
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1558) echo +eth0 > /sys/class/net/bond0/bonding/slaves
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1559) echo +eth1 > /sys/class/net/bond0/bonding/slaves
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1560)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1561) To add a second bond, with two e1000 interfaces in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1562) active-backup mode, using ARP monitoring, add the following lines to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1563) your init script::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1564)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1565) modprobe e1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1566) echo +bond1 > /sys/class/net/bonding_masters
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1567) echo active-backup > /sys/class/net/bond1/bonding/mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1568) ifconfig bond1 192.168.2.1 netmask 255.255.255.0 up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1569) echo +192.168.2.100 /sys/class/net/bond1/bonding/arp_ip_target
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1570) echo 2000 > /sys/class/net/bond1/bonding/arp_interval
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1571) echo +eth2 > /sys/class/net/bond1/bonding/slaves
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1572) echo +eth3 > /sys/class/net/bond1/bonding/slaves
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1573)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1574) 3.5 Configuration with Interfaces Support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1575) -----------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1576)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1577) This section applies to distros which use /etc/network/interfaces file
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1578) to describe network interface configuration, most notably Debian and it's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1579) derivatives.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1580)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1581) The ifup and ifdown commands on Debian don't support bonding out of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1582) the box. The ifenslave-2.6 package should be installed to provide bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1583) support. Once installed, this package will provide ``bond-*`` options
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1584) to be used into /etc/network/interfaces.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1585)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1586) Note that ifenslave-2.6 package will load the bonding module and use
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1587) the ifenslave command when appropriate.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1588)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1589) Example Configurations
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1590) ----------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1591)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1592) In /etc/network/interfaces, the following stanza will configure bond0, in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1593) active-backup mode, with eth0 and eth1 as slaves::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1594)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1595) auto bond0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1596) iface bond0 inet dhcp
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1597) bond-slaves eth0 eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1598) bond-mode active-backup
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1599) bond-miimon 100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1600) bond-primary eth0 eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1601)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1602) If the above configuration doesn't work, you might have a system using
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1603) upstart for system startup. This is most notably true for recent
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1604) Ubuntu versions. The following stanza in /etc/network/interfaces will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1605) produce the same result on those systems::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1606)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1607) auto bond0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1608) iface bond0 inet dhcp
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1609) bond-slaves none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1610) bond-mode active-backup
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1611) bond-miimon 100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1612)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1613) auto eth0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1614) iface eth0 inet manual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1615) bond-master bond0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1616) bond-primary eth0 eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1617)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1618) auto eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1619) iface eth1 inet manual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1620) bond-master bond0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1621) bond-primary eth0 eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1622)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1623) For a full list of ``bond-*`` supported options in /etc/network/interfaces and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1624) some more advanced examples tailored to you particular distros, see the files in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1625) /usr/share/doc/ifenslave-2.6.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1626)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1627) 3.6 Overriding Configuration for Special Cases
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1628) ----------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1629)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1630) When using the bonding driver, the physical port which transmits a frame is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1631) typically selected by the bonding driver, and is not relevant to the user or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1632) system administrator. The output port is simply selected using the policies of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1633) the selected bonding mode. On occasion however, it is helpful to direct certain
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1634) classes of traffic to certain physical interfaces on output to implement
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1635) slightly more complex policies. For example, to reach a web server over a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1636) bonded interface in which eth0 connects to a private network, while eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1637) connects via a public network, it may be desirous to bias the bond to send said
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1638) traffic over eth0 first, using eth1 only as a fall back, while all other traffic
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1639) can safely be sent over either interface. Such configurations may be achieved
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1640) using the traffic control utilities inherent in linux.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1641)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1642) By default the bonding driver is multiqueue aware and 16 queues are created
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1643) when the driver initializes (see Documentation/networking/multiqueue.rst
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1644) for details). If more or less queues are desired the module parameter
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1645) tx_queues can be used to change this value. There is no sysfs parameter
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1646) available as the allocation is done at module init time.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1647)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1648) The output of the file /proc/net/bonding/bondX has changed so the output Queue
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1649) ID is now printed for each slave::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1650)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1651) Bonding Mode: fault-tolerance (active-backup)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1652) Primary Slave: None
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1653) Currently Active Slave: eth0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1654) MII Status: up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1655) MII Polling Interval (ms): 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1656) Up Delay (ms): 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1657) Down Delay (ms): 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1658)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1659) Slave Interface: eth0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1660) MII Status: up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1661) Link Failure Count: 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1662) Permanent HW addr: 00:1a:a0:12:8f:cb
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1663) Slave queue ID: 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1664)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1665) Slave Interface: eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1666) MII Status: up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1667) Link Failure Count: 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1668) Permanent HW addr: 00:1a:a0:12:8f:cc
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1669) Slave queue ID: 2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1670)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1671) The queue_id for a slave can be set using the command::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1672)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1673) # echo "eth1:2" > /sys/class/net/bond0/bonding/queue_id
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1674)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1675) Any interface that needs a queue_id set should set it with multiple calls
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1676) like the one above until proper priorities are set for all interfaces. On
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1677) distributions that allow configuration via initscripts, multiple 'queue_id'
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1678) arguments can be added to BONDING_OPTS to set all needed slave queues.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1679)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1680) These queue id's can be used in conjunction with the tc utility to configure
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1681) a multiqueue qdisc and filters to bias certain traffic to transmit on certain
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1682) slave devices. For instance, say we wanted, in the above configuration to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1683) force all traffic bound to 192.168.1.100 to use eth1 in the bond as its output
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1684) device. The following commands would accomplish this::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1685)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1686) # tc qdisc add dev bond0 handle 1 root multiq
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1687)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1688) # tc filter add dev bond0 protocol ip parent 1: prio 1 u32 match ip \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1689) dst 192.168.1.100 action skbedit queue_mapping 2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1690)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1691) These commands tell the kernel to attach a multiqueue queue discipline to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1692) bond0 interface and filter traffic enqueued to it, such that packets with a dst
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1693) ip of 192.168.1.100 have their output queue mapping value overwritten to 2.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1694) This value is then passed into the driver, causing the normal output path
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1695) selection policy to be overridden, selecting instead qid 2, which maps to eth1.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1696)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1697) Note that qid values begin at 1. Qid 0 is reserved to initiate to the driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1698) that normal output policy selection should take place. One benefit to simply
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1699) leaving the qid for a slave to 0 is the multiqueue awareness in the bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1700) driver that is now present. This awareness allows tc filters to be placed on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1701) slave devices as well as bond devices and the bonding driver will simply act as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1702) a pass-through for selecting output queues on the slave device rather than
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1703) output port selection.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1704)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1705) This feature first appeared in bonding driver version 3.7.0 and support for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1706) output slave selection was limited to round-robin and active-backup modes.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1707)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1708) 3.7 Configuring LACP for 802.3ad mode in a more secure way
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1709) ----------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1710)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1711) When using 802.3ad bonding mode, the Actor (host) and Partner (switch)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1712) exchange LACPDUs. These LACPDUs cannot be sniffed, because they are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1713) destined to link local mac addresses (which switches/bridges are not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1714) supposed to forward). However, most of the values are easily predictable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1715) or are simply the machine's MAC address (which is trivially known to all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1716) other hosts in the same L2). This implies that other machines in the L2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1717) domain can spoof LACPDU packets from other hosts to the switch and potentially
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1718) cause mayhem by joining (from the point of view of the switch) another
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1719) machine's aggregate, thus receiving a portion of that hosts incoming
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1720) traffic and / or spoofing traffic from that machine themselves (potentially
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1721) even successfully terminating some portion of flows). Though this is not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1722) a likely scenario, one could avoid this possibility by simply configuring
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1723) few bonding parameters:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1724)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1725) (a) ad_actor_system : You can set a random mac-address that can be used for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1726) these LACPDU exchanges. The value can not be either NULL or Multicast.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1727) Also it's preferable to set the local-admin bit. Following shell code
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1728) generates a random mac-address as described above::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1729)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1730) # sys_mac_addr=$(printf '%02x:%02x:%02x:%02x:%02x:%02x' \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1731) $(( (RANDOM & 0xFE) | 0x02 )) \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1732) $(( RANDOM & 0xFF )) \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1733) $(( RANDOM & 0xFF )) \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1734) $(( RANDOM & 0xFF )) \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1735) $(( RANDOM & 0xFF )) \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1736) $(( RANDOM & 0xFF )))
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1737) # echo $sys_mac_addr > /sys/class/net/bond0/bonding/ad_actor_system
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1738)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1739) (b) ad_actor_sys_prio : Randomize the system priority. The default value
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1740) is 65535, but system can take the value from 1 - 65535. Following shell
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1741) code generates random priority and sets it::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1742)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1743) # sys_prio=$(( 1 + RANDOM + RANDOM ))
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1744) # echo $sys_prio > /sys/class/net/bond0/bonding/ad_actor_sys_prio
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1745)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1746) (c) ad_user_port_key : Use the user portion of the port-key. The default
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1747) keeps this empty. These are the upper 10 bits of the port-key and value
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1748) ranges from 0 - 1023. Following shell code generates these 10 bits and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1749) sets it::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1750)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1751) # usr_port_key=$(( RANDOM & 0x3FF ))
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1752) # echo $usr_port_key > /sys/class/net/bond0/bonding/ad_user_port_key
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1753)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1754)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1755) 4 Querying Bonding Configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1756) =================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1757)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1758) 4.1 Bonding Configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1759) -------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1760)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1761) Each bonding device has a read-only file residing in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1762) /proc/net/bonding directory. The file contents include information
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1763) about the bonding configuration, options and state of each slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1764)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1765) For example, the contents of /proc/net/bonding/bond0 after the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1766) driver is loaded with parameters of mode=0 and miimon=1000 is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1767) generally as follows::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1768)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1769) Ethernet Channel Bonding Driver: 2.6.1 (October 29, 2004)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1770) Bonding Mode: load balancing (round-robin)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1771) Currently Active Slave: eth0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1772) MII Status: up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1773) MII Polling Interval (ms): 1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1774) Up Delay (ms): 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1775) Down Delay (ms): 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1776)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1777) Slave Interface: eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1778) MII Status: up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1779) Link Failure Count: 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1780)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1781) Slave Interface: eth0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1782) MII Status: up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1783) Link Failure Count: 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1784)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1785) The precise format and contents will change depending upon the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1786) bonding configuration, state, and version of the bonding driver.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1787)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1788) 4.2 Network configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1789) -------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1790)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1791) The network configuration can be inspected using the ifconfig
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1792) command. Bonding devices will have the MASTER flag set; Bonding slave
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1793) devices will have the SLAVE flag set. The ifconfig output does not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1794) contain information on which slaves are associated with which masters.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1795)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1796) In the example below, the bond0 interface is the master
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1797) (MASTER) while eth0 and eth1 are slaves (SLAVE). Notice all slaves of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1798) bond0 have the same MAC address (HWaddr) as bond0 for all modes except
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1799) TLB and ALB that require a unique MAC address for each slave::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1800)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1801) # /sbin/ifconfig
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1802) bond0 Link encap:Ethernet HWaddr 00:C0:F0:1F:37:B4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1803) inet addr:XXX.XXX.XXX.YYY Bcast:XXX.XXX.XXX.255 Mask:255.255.252.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1804) UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1805) RX packets:7224794 errors:0 dropped:0 overruns:0 frame:0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1806) TX packets:3286647 errors:1 dropped:0 overruns:1 carrier:0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1807) collisions:0 txqueuelen:0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1808)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1809) eth0 Link encap:Ethernet HWaddr 00:C0:F0:1F:37:B4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1810) UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1811) RX packets:3573025 errors:0 dropped:0 overruns:0 frame:0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1812) TX packets:1643167 errors:1 dropped:0 overruns:1 carrier:0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1813) collisions:0 txqueuelen:100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1814) Interrupt:10 Base address:0x1080
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1815)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1816) eth1 Link encap:Ethernet HWaddr 00:C0:F0:1F:37:B4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1817) UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1818) RX packets:3651769 errors:0 dropped:0 overruns:0 frame:0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1819) TX packets:1643480 errors:0 dropped:0 overruns:0 carrier:0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1820) collisions:0 txqueuelen:100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1821) Interrupt:9 Base address:0x1400
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1822)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1823) 5. Switch Configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1824) =======================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1825)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1826) For this section, "switch" refers to whatever system the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1827) bonded devices are directly connected to (i.e., where the other end of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1828) the cable plugs into). This may be an actual dedicated switch device,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1829) or it may be another regular system (e.g., another computer running
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1830) Linux),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1831)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1832) The active-backup, balance-tlb and balance-alb modes do not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1833) require any specific configuration of the switch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1834)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1835) The 802.3ad mode requires that the switch have the appropriate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1836) ports configured as an 802.3ad aggregation. The precise method used
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1837) to configure this varies from switch to switch, but, for example, a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1838) Cisco 3550 series switch requires that the appropriate ports first be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1839) grouped together in a single etherchannel instance, then that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1840) etherchannel is set to mode "lacp" to enable 802.3ad (instead of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1841) standard EtherChannel).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1842)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1843) The balance-rr, balance-xor and broadcast modes generally
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1844) require that the switch have the appropriate ports grouped together.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1845) The nomenclature for such a group differs between switches, it may be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1846) called an "etherchannel" (as in the Cisco example, above), a "trunk
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1847) group" or some other similar variation. For these modes, each switch
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1848) will also have its own configuration options for the switch's transmit
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1849) policy to the bond. Typical choices include XOR of either the MAC or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1850) IP addresses. The transmit policy of the two peers does not need to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1851) match. For these three modes, the bonding mode really selects a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1852) transmit policy for an EtherChannel group; all three will interoperate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1853) with another EtherChannel group.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1854)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1855)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1856) 6. 802.1q VLAN Support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1857) ======================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1858)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1859) It is possible to configure VLAN devices over a bond interface
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1860) using the 8021q driver. However, only packets coming from the 8021q
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1861) driver and passing through bonding will be tagged by default. Self
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1862) generated packets, for example, bonding's learning packets or ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1863) packets generated by either ALB mode or the ARP monitor mechanism, are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1864) tagged internally by bonding itself. As a result, bonding must
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1865) "learn" the VLAN IDs configured above it, and use those IDs to tag
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1866) self generated packets.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1867)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1868) For reasons of simplicity, and to support the use of adapters
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1869) that can do VLAN hardware acceleration offloading, the bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1870) interface declares itself as fully hardware offloading capable, it gets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1871) the add_vid/kill_vid notifications to gather the necessary
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1872) information, and it propagates those actions to the slaves. In case
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1873) of mixed adapter types, hardware accelerated tagged packets that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1874) should go through an adapter that is not offloading capable are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1875) "un-accelerated" by the bonding driver so the VLAN tag sits in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1876) regular location.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1877)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1878) VLAN interfaces *must* be added on top of a bonding interface
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1879) only after enslaving at least one slave. The bonding interface has a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1880) hardware address of 00:00:00:00:00:00 until the first slave is added.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1881) If the VLAN interface is created prior to the first enslavement, it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1882) would pick up the all-zeroes hardware address. Once the first slave
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1883) is attached to the bond, the bond device itself will pick up the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1884) slave's hardware address, which is then available for the VLAN device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1885)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1886) Also, be aware that a similar problem can occur if all slaves
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1887) are released from a bond that still has one or more VLAN interfaces on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1888) top of it. When a new slave is added, the bonding interface will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1889) obtain its hardware address from the first slave, which might not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1890) match the hardware address of the VLAN interfaces (which was
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1891) ultimately copied from an earlier slave).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1892)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1893) There are two methods to insure that the VLAN device operates
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1894) with the correct hardware address if all slaves are removed from a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1895) bond interface:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1896)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1897) 1. Remove all VLAN interfaces then recreate them
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1898)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1899) 2. Set the bonding interface's hardware address so that it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1900) matches the hardware address of the VLAN interfaces.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1901)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1902) Note that changing a VLAN interface's HW address would set the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1903) underlying device -- i.e. the bonding interface -- to promiscuous
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1904) mode, which might not be what you want.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1905)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1906)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1907) 7. Link Monitoring
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1908) ==================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1909)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1910) The bonding driver at present supports two schemes for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1911) monitoring a slave device's link state: the ARP monitor and the MII
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1912) monitor.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1913)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1914) At the present time, due to implementation restrictions in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1915) bonding driver itself, it is not possible to enable both ARP and MII
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1916) monitoring simultaneously.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1917)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1918) 7.1 ARP Monitor Operation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1919) -------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1920)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1921) The ARP monitor operates as its name suggests: it sends ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1922) queries to one or more designated peer systems on the network, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1923) uses the response as an indication that the link is operating. This
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1924) gives some assurance that traffic is actually flowing to and from one
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1925) or more peers on the local network.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1926)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1927) The ARP monitor relies on the device driver itself to verify
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1928) that traffic is flowing. In particular, the driver must keep up to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1929) date the last receive time, dev->last_rx. Drivers that use NETIF_F_LLTX
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1930) flag must also update netdev_queue->trans_start. If they do not, then the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1931) ARP monitor will immediately fail any slaves using that driver, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1932) those slaves will stay down. If networking monitoring (tcpdump, etc)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1933) shows the ARP requests and replies on the network, then it may be that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1934) your device driver is not updating last_rx and trans_start.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1935)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1936) 7.2 Configuring Multiple ARP Targets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1937) ------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1938)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1939) While ARP monitoring can be done with just one target, it can
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1940) be useful in a High Availability setup to have several targets to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1941) monitor. In the case of just one target, the target itself may go
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1942) down or have a problem making it unresponsive to ARP requests. Having
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1943) an additional target (or several) increases the reliability of the ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1944) monitoring.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1945)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1946) Multiple ARP targets must be separated by commas as follows::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1947)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1948) # example options for ARP monitoring with three targets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1949) alias bond0 bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1950) options bond0 arp_interval=60 arp_ip_target=192.168.0.1,192.168.0.3,192.168.0.9
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1951)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1952) For just a single target the options would resemble::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1953)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1954) # example options for ARP monitoring with one target
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1955) alias bond0 bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1956) options bond0 arp_interval=60 arp_ip_target=192.168.0.100
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1957)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1958)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1959) 7.3 MII Monitor Operation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1960) -------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1961)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1962) The MII monitor monitors only the carrier state of the local
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1963) network interface. It accomplishes this in one of three ways: by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1964) depending upon the device driver to maintain its carrier state, by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1965) querying the device's MII registers, or by making an ethtool query to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1966) the device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1967)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1968) If the use_carrier module parameter is 1 (the default value),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1969) then the MII monitor will rely on the driver for carrier state
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1970) information (via the netif_carrier subsystem). As explained in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1971) use_carrier parameter information, above, if the MII monitor fails to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1972) detect carrier loss on the device (e.g., when the cable is physically
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1973) disconnected), it may be that the driver does not support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1974) netif_carrier.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1975)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1976) If use_carrier is 0, then the MII monitor will first query the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1977) device's (via ioctl) MII registers and check the link state. If that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1978) request fails (not just that it returns carrier down), then the MII
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1979) monitor will make an ethtool ETHOOL_GLINK request to attempt to obtain
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1980) the same information. If both methods fail (i.e., the driver either
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1981) does not support or had some error in processing both the MII register
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1982) and ethtool requests), then the MII monitor will assume the link is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1983) up.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1984)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1985) 8. Potential Sources of Trouble
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1986) ===============================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1987)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1988) 8.1 Adventures in Routing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1989) -------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1990)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1991) When bonding is configured, it is important that the slave
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1992) devices not have routes that supersede routes of the master (or,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1993) generally, not have routes at all). For example, suppose the bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1994) device bond0 has two slaves, eth0 and eth1, and the routing table is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1995) as follows::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1996)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1997) Kernel IP routing table
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1998) Destination Gateway Genmask Flags MSS Window irtt Iface
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1999) 10.0.0.0 0.0.0.0 255.255.0.0 U 40 0 0 eth0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2000) 10.0.0.0 0.0.0.0 255.255.0.0 U 40 0 0 eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2001) 10.0.0.0 0.0.0.0 255.255.0.0 U 40 0 0 bond0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2002) 127.0.0.0 0.0.0.0 255.0.0.0 U 40 0 0 lo
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2003)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2004) This routing configuration will likely still update the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2005) receive/transmit times in the driver (needed by the ARP monitor), but
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2006) may bypass the bonding driver (because outgoing traffic to, in this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2007) case, another host on network 10 would use eth0 or eth1 before bond0).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2008)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2009) The ARP monitor (and ARP itself) may become confused by this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2010) configuration, because ARP requests (generated by the ARP monitor)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2011) will be sent on one interface (bond0), but the corresponding reply
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2012) will arrive on a different interface (eth0). This reply looks to ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2013) as an unsolicited ARP reply (because ARP matches replies on an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2014) interface basis), and is discarded. The MII monitor is not affected
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2015) by the state of the routing table.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2016)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2017) The solution here is simply to insure that slaves do not have
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2018) routes of their own, and if for some reason they must, those routes do
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2019) not supersede routes of their master. This should generally be the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2020) case, but unusual configurations or errant manual or automatic static
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2021) route additions may cause trouble.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2022)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2023) 8.2 Ethernet Device Renaming
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2024) ----------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2025)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2026) On systems with network configuration scripts that do not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2027) associate physical devices directly with network interface names (so
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2028) that the same physical device always has the same "ethX" name), it may
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2029) be necessary to add some special logic to config files in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2030) /etc/modprobe.d/.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2031)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2032) For example, given a modules.conf containing the following::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2033)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2034) alias bond0 bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2035) options bond0 mode=some-mode miimon=50
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2036) alias eth0 tg3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2037) alias eth1 tg3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2038) alias eth2 e1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2039) alias eth3 e1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2040)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2041) If neither eth0 and eth1 are slaves to bond0, then when the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2042) bond0 interface comes up, the devices may end up reordered. This
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2043) happens because bonding is loaded first, then its slave device's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2044) drivers are loaded next. Since no other drivers have been loaded,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2045) when the e1000 driver loads, it will receive eth0 and eth1 for its
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2046) devices, but the bonding configuration tries to enslave eth2 and eth3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2047) (which may later be assigned to the tg3 devices).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2048)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2049) Adding the following::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2050)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2051) add above bonding e1000 tg3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2052)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2053) causes modprobe to load e1000 then tg3, in that order, when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2054) bonding is loaded. This command is fully documented in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2055) modules.conf manual page.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2056)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2057) On systems utilizing modprobe an equivalent problem can occur.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2058) In this case, the following can be added to config files in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2059) /etc/modprobe.d/ as::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2060)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2061) softdep bonding pre: tg3 e1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2062)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2063) This will load tg3 and e1000 modules before loading the bonding one.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2064) Full documentation on this can be found in the modprobe.d and modprobe
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2065) manual pages.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2066)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2067) 8.3. Painfully Slow Or No Failed Link Detection By Miimon
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2068) ---------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2069)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2070) By default, bonding enables the use_carrier option, which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2071) instructs bonding to trust the driver to maintain carrier state.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2072)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2073) As discussed in the options section, above, some drivers do
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2074) not support the netif_carrier_on/_off link state tracking system.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2075) With use_carrier enabled, bonding will always see these links as up,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2076) regardless of their actual state.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2077)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2078) Additionally, other drivers do support netif_carrier, but do
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2079) not maintain it in real time, e.g., only polling the link state at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2080) some fixed interval. In this case, miimon will detect failures, but
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2081) only after some long period of time has expired. If it appears that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2082) miimon is very slow in detecting link failures, try specifying
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2083) use_carrier=0 to see if that improves the failure detection time. If
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2084) it does, then it may be that the driver checks the carrier state at a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2085) fixed interval, but does not cache the MII register values (so the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2086) use_carrier=0 method of querying the registers directly works). If
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2087) use_carrier=0 does not improve the failover, then the driver may cache
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2088) the registers, or the problem may be elsewhere.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2089)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2090) Also, remember that miimon only checks for the device's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2091) carrier state. It has no way to determine the state of devices on or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2092) beyond other ports of a switch, or if a switch is refusing to pass
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2093) traffic while still maintaining carrier on.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2094)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2095) 9. SNMP agents
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2096) ===============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2097)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2098) If running SNMP agents, the bonding driver should be loaded
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2099) before any network drivers participating in a bond. This requirement
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2100) is due to the interface index (ipAdEntIfIndex) being associated to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2101) the first interface found with a given IP address. That is, there is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2102) only one ipAdEntIfIndex for each IP address. For example, if eth0 and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2103) eth1 are slaves of bond0 and the driver for eth0 is loaded before the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2104) bonding driver, the interface for the IP address will be associated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2105) with the eth0 interface. This configuration is shown below, the IP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2106) address 192.168.1.1 has an interface index of 2 which indexes to eth0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2107) in the ifDescr table (ifDescr.2).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2108)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2109) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2110)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2111) interfaces.ifTable.ifEntry.ifDescr.1 = lo
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2112) interfaces.ifTable.ifEntry.ifDescr.2 = eth0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2113) interfaces.ifTable.ifEntry.ifDescr.3 = eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2114) interfaces.ifTable.ifEntry.ifDescr.4 = eth2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2115) interfaces.ifTable.ifEntry.ifDescr.5 = eth3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2116) interfaces.ifTable.ifEntry.ifDescr.6 = bond0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2117) ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.10.10.10.10 = 5
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2118) ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.192.168.1.1 = 2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2119) ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.10.74.20.94 = 4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2120) ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.127.0.0.1 = 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2121)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2122) This problem is avoided by loading the bonding driver before
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2123) any network drivers participating in a bond. Below is an example of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2124) loading the bonding driver first, the IP address 192.168.1.1 is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2125) correctly associated with ifDescr.2.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2126)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2127) interfaces.ifTable.ifEntry.ifDescr.1 = lo
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2128) interfaces.ifTable.ifEntry.ifDescr.2 = bond0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2129) interfaces.ifTable.ifEntry.ifDescr.3 = eth0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2130) interfaces.ifTable.ifEntry.ifDescr.4 = eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2131) interfaces.ifTable.ifEntry.ifDescr.5 = eth2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2132) interfaces.ifTable.ifEntry.ifDescr.6 = eth3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2133) ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.10.10.10.10 = 6
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2134) ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.192.168.1.1 = 2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2135) ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.10.74.20.94 = 5
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2136) ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.127.0.0.1 = 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2137)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2138) While some distributions may not report the interface name in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2139) ifDescr, the association between the IP address and IfIndex remains
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2140) and SNMP functions such as Interface_Scan_Next will report that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2141) association.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2142)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2143) 10. Promiscuous mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2144) ====================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2145)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2146) When running network monitoring tools, e.g., tcpdump, it is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2147) common to enable promiscuous mode on the device, so that all traffic
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2148) is seen (instead of seeing only traffic destined for the local host).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2149) The bonding driver handles promiscuous mode changes to the bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2150) master device (e.g., bond0), and propagates the setting to the slave
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2151) devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2152)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2153) For the balance-rr, balance-xor, broadcast, and 802.3ad modes,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2154) the promiscuous mode setting is propagated to all slaves.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2155)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2156) For the active-backup, balance-tlb and balance-alb modes, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2157) promiscuous mode setting is propagated only to the active slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2158)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2159) For balance-tlb mode, the active slave is the slave currently
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2160) receiving inbound traffic.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2161)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2162) For balance-alb mode, the active slave is the slave used as a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2163) "primary." This slave is used for mode-specific control traffic, for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2164) sending to peers that are unassigned or if the load is unbalanced.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2165)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2166) For the active-backup, balance-tlb and balance-alb modes, when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2167) the active slave changes (e.g., due to a link failure), the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2168) promiscuous setting will be propagated to the new active slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2169)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2170) 11. Configuring Bonding for High Availability
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2171) =============================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2172)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2173) High Availability refers to configurations that provide
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2174) maximum network availability by having redundant or backup devices,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2175) links or switches between the host and the rest of the world. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2176) goal is to provide the maximum availability of network connectivity
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2177) (i.e., the network always works), even though other configurations
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2178) could provide higher throughput.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2179)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2180) 11.1 High Availability in a Single Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2181) --------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2182)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2183) If two hosts (or a host and a single switch) are directly
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2184) connected via multiple physical links, then there is no availability
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2185) penalty to optimizing for maximum bandwidth. In this case, there is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2186) only one switch (or peer), so if it fails, there is no alternative
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2187) access to fail over to. Additionally, the bonding load balance modes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2188) support link monitoring of their members, so if individual links fail,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2189) the load will be rebalanced across the remaining devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2190)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2191) See Section 12, "Configuring Bonding for Maximum Throughput"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2192) for information on configuring bonding with one peer device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2193)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2194) 11.2 High Availability in a Multiple Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2195) ----------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2196)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2197) With multiple switches, the configuration of bonding and the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2198) network changes dramatically. In multiple switch topologies, there is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2199) a trade off between network availability and usable bandwidth.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2200)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2201) Below is a sample network, configured to maximize the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2202) availability of the network::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2203)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2204) | |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2205) |port3 port3|
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2206) +-----+----+ +-----+----+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2207) | |port2 ISL port2| |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2208) | switch A +--------------------------+ switch B |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2209) | | | |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2210) +-----+----+ +-----++---+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2211) |port1 port1|
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2212) | +-------+ |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2213) +-------------+ host1 +---------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2214) eth0 +-------+ eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2215)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2216) In this configuration, there is a link between the two
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2217) switches (ISL, or inter switch link), and multiple ports connecting to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2218) the outside world ("port3" on each switch). There is no technical
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2219) reason that this could not be extended to a third switch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2220)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2221) 11.2.1 HA Bonding Mode Selection for Multiple Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2222) -------------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2223)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2224) In a topology such as the example above, the active-backup and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2225) broadcast modes are the only useful bonding modes when optimizing for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2226) availability; the other modes require all links to terminate on the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2227) same peer for them to behave rationally.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2228)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2229) active-backup:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2230) This is generally the preferred mode, particularly if
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2231) the switches have an ISL and play together well. If the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2232) network configuration is such that one switch is specifically
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2233) a backup switch (e.g., has lower capacity, higher cost, etc),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2234) then the primary option can be used to insure that the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2235) preferred link is always used when it is available.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2236)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2237) broadcast:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2238) This mode is really a special purpose mode, and is suitable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2239) only for very specific needs. For example, if the two
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2240) switches are not connected (no ISL), and the networks beyond
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2241) them are totally independent. In this case, if it is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2242) necessary for some specific one-way traffic to reach both
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2243) independent networks, then the broadcast mode may be suitable.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2244)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2245) 11.2.2 HA Link Monitoring Selection for Multiple Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2246) ----------------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2247)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2248) The choice of link monitoring ultimately depends upon your
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2249) switch. If the switch can reliably fail ports in response to other
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2250) failures, then either the MII or ARP monitors should work. For
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2251) example, in the above example, if the "port3" link fails at the remote
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2252) end, the MII monitor has no direct means to detect this. The ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2253) monitor could be configured with a target at the remote end of port3,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2254) thus detecting that failure without switch support.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2255)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2256) In general, however, in a multiple switch topology, the ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2257) monitor can provide a higher level of reliability in detecting end to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2258) end connectivity failures (which may be caused by the failure of any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2259) individual component to pass traffic for any reason). Additionally,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2260) the ARP monitor should be configured with multiple targets (at least
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2261) one for each switch in the network). This will insure that,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2262) regardless of which switch is active, the ARP monitor has a suitable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2263) target to query.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2264)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2265) Note, also, that of late many switches now support a functionality
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2266) generally referred to as "trunk failover." This is a feature of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2267) switch that causes the link state of a particular switch port to be set
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2268) down (or up) when the state of another switch port goes down (or up).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2269) Its purpose is to propagate link failures from logically "exterior" ports
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2270) to the logically "interior" ports that bonding is able to monitor via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2271) miimon. Availability and configuration for trunk failover varies by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2272) switch, but this can be a viable alternative to the ARP monitor when using
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2273) suitable switches.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2274)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2275) 12. Configuring Bonding for Maximum Throughput
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2276) ==============================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2277)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2278) 12.1 Maximizing Throughput in a Single Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2279) ------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2280)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2281) In a single switch configuration, the best method to maximize
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2282) throughput depends upon the application and network environment. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2283) various load balancing modes each have strengths and weaknesses in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2284) different environments, as detailed below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2285)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2286) For this discussion, we will break down the topologies into
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2287) two categories. Depending upon the destination of most traffic, we
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2288) categorize them into either "gatewayed" or "local" configurations.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2289)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2290) In a gatewayed configuration, the "switch" is acting primarily
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2291) as a router, and the majority of traffic passes through this router to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2292) other networks. An example would be the following::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2293)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2294)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2295) +----------+ +----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2296) | |eth0 port1| | to other networks
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2297) | Host A +---------------------+ router +------------------->
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2298) | +---------------------+ | Hosts B and C are out
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2299) | |eth1 port2| | here somewhere
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2300) +----------+ +----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2301)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2302) The router may be a dedicated router device, or another host
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2303) acting as a gateway. For our discussion, the important point is that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2304) the majority of traffic from Host A will pass through the router to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2305) some other network before reaching its final destination.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2306)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2307) In a gatewayed network configuration, although Host A may
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2308) communicate with many other systems, all of its traffic will be sent
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2309) and received via one other peer on the local network, the router.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2310)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2311) Note that the case of two systems connected directly via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2312) multiple physical links is, for purposes of configuring bonding, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2313) same as a gatewayed configuration. In that case, it happens that all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2314) traffic is destined for the "gateway" itself, not some other network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2315) beyond the gateway.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2316)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2317) In a local configuration, the "switch" is acting primarily as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2318) a switch, and the majority of traffic passes through this switch to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2319) reach other stations on the same network. An example would be the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2320) following::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2321)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2322) +----------+ +----------+ +--------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2323) | |eth0 port1| +-------+ Host B |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2324) | Host A +------------+ switch |port3 +--------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2325) | +------------+ | +--------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2326) | |eth1 port2| +------------------+ Host C |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2327) +----------+ +----------+port4 +--------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2328)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2329)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2330) Again, the switch may be a dedicated switch device, or another
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2331) host acting as a gateway. For our discussion, the important point is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2332) that the majority of traffic from Host A is destined for other hosts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2333) on the same local network (Hosts B and C in the above example).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2334)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2335) In summary, in a gatewayed configuration, traffic to and from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2336) the bonded device will be to the same MAC level peer on the network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2337) (the gateway itself, i.e., the router), regardless of its final
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2338) destination. In a local configuration, traffic flows directly to and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2339) from the final destinations, thus, each destination (Host B, Host C)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2340) will be addressed directly by their individual MAC addresses.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2341)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2342) This distinction between a gatewayed and a local network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2343) configuration is important because many of the load balancing modes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2344) available use the MAC addresses of the local network source and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2345) destination to make load balancing decisions. The behavior of each
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2346) mode is described below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2347)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2348)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2349) 12.1.1 MT Bonding Mode Selection for Single Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2350) -----------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2351)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2352) This configuration is the easiest to set up and to understand,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2353) although you will have to decide which bonding mode best suits your
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2354) needs. The trade offs for each mode are detailed below:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2355)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2356) balance-rr:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2357) This mode is the only mode that will permit a single
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2358) TCP/IP connection to stripe traffic across multiple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2359) interfaces. It is therefore the only mode that will allow a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2360) single TCP/IP stream to utilize more than one interface's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2361) worth of throughput. This comes at a cost, however: the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2362) striping generally results in peer systems receiving packets out
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2363) of order, causing TCP/IP's congestion control system to kick
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2364) in, often by retransmitting segments.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2365)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2366) It is possible to adjust TCP/IP's congestion limits by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2367) altering the net.ipv4.tcp_reordering sysctl parameter. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2368) usual default value is 3. But keep in mind TCP stack is able
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2369) to automatically increase this when it detects reorders.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2370)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2371) Note that the fraction of packets that will be delivered out of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2372) order is highly variable, and is unlikely to be zero. The level
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2373) of reordering depends upon a variety of factors, including the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2374) networking interfaces, the switch, and the topology of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2375) configuration. Speaking in general terms, higher speed network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2376) cards produce more reordering (due to factors such as packet
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2377) coalescing), and a "many to many" topology will reorder at a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2378) higher rate than a "many slow to one fast" configuration.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2379)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2380) Many switches do not support any modes that stripe traffic
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2381) (instead choosing a port based upon IP or MAC level addresses);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2382) for those devices, traffic for a particular connection flowing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2383) through the switch to a balance-rr bond will not utilize greater
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2384) than one interface's worth of bandwidth.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2385)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2386) If you are utilizing protocols other than TCP/IP, UDP for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2387) example, and your application can tolerate out of order
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2388) delivery, then this mode can allow for single stream datagram
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2389) performance that scales near linearly as interfaces are added
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2390) to the bond.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2391)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2392) This mode requires the switch to have the appropriate ports
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2393) configured for "etherchannel" or "trunking."
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2394)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2395) active-backup:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2396) There is not much advantage in this network topology to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2397) the active-backup mode, as the inactive backup devices are all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2398) connected to the same peer as the primary. In this case, a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2399) load balancing mode (with link monitoring) will provide the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2400) same level of network availability, but with increased
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2401) available bandwidth. On the plus side, active-backup mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2402) does not require any configuration of the switch, so it may
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2403) have value if the hardware available does not support any of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2404) the load balance modes.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2405)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2406) balance-xor:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2407) This mode will limit traffic such that packets destined
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2408) for specific peers will always be sent over the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2409) interface. Since the destination is determined by the MAC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2410) addresses involved, this mode works best in a "local" network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2411) configuration (as described above), with destinations all on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2412) the same local network. This mode is likely to be suboptimal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2413) if all your traffic is passed through a single router (i.e., a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2414) "gatewayed" network configuration, as described above).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2415)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2416) As with balance-rr, the switch ports need to be configured for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2417) "etherchannel" or "trunking."
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2418)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2419) broadcast:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2420) Like active-backup, there is not much advantage to this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2421) mode in this type of network topology.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2422)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2423) 802.3ad:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2424) This mode can be a good choice for this type of network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2425) topology. The 802.3ad mode is an IEEE standard, so all peers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2426) that implement 802.3ad should interoperate well. The 802.3ad
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2427) protocol includes automatic configuration of the aggregates,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2428) so minimal manual configuration of the switch is needed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2429) (typically only to designate that some set of devices is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2430) available for 802.3ad). The 802.3ad standard also mandates
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2431) that frames be delivered in order (within certain limits), so
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2432) in general single connections will not see misordering of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2433) packets. The 802.3ad mode does have some drawbacks: the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2434) standard mandates that all devices in the aggregate operate at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2435) the same speed and duplex. Also, as with all bonding load
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2436) balance modes other than balance-rr, no single connection will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2437) be able to utilize more than a single interface's worth of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2438) bandwidth.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2439)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2440) Additionally, the linux bonding 802.3ad implementation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2441) distributes traffic by peer (using an XOR of MAC addresses
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2442) and packet type ID), so in a "gatewayed" configuration, all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2443) outgoing traffic will generally use the same device. Incoming
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2444) traffic may also end up on a single device, but that is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2445) dependent upon the balancing policy of the peer's 802.3ad
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2446) implementation. In a "local" configuration, traffic will be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2447) distributed across the devices in the bond.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2448)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2449) Finally, the 802.3ad mode mandates the use of the MII monitor,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2450) therefore, the ARP monitor is not available in this mode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2451)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2452) balance-tlb:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2453) The balance-tlb mode balances outgoing traffic by peer.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2454) Since the balancing is done according to MAC address, in a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2455) "gatewayed" configuration (as described above), this mode will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2456) send all traffic across a single device. However, in a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2457) "local" network configuration, this mode balances multiple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2458) local network peers across devices in a vaguely intelligent
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2459) manner (not a simple XOR as in balance-xor or 802.3ad mode),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2460) so that mathematically unlucky MAC addresses (i.e., ones that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2461) XOR to the same value) will not all "bunch up" on a single
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2462) interface.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2463)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2464) Unlike 802.3ad, interfaces may be of differing speeds, and no
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2465) special switch configuration is required. On the down side,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2466) in this mode all incoming traffic arrives over a single
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2467) interface, this mode requires certain ethtool support in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2468) network device driver of the slave interfaces, and the ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2469) monitor is not available.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2470)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2471) balance-alb:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2472) This mode is everything that balance-tlb is, and more.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2473) It has all of the features (and restrictions) of balance-tlb,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2474) and will also balance incoming traffic from local network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2475) peers (as described in the Bonding Module Options section,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2476) above).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2477)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2478) The only additional down side to this mode is that the network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2479) device driver must support changing the hardware address while
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2480) the device is open.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2481)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2482) 12.1.2 MT Link Monitoring for Single Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2483) ----------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2484)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2485) The choice of link monitoring may largely depend upon which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2486) mode you choose to use. The more advanced load balancing modes do not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2487) support the use of the ARP monitor, and are thus restricted to using
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2488) the MII monitor (which does not provide as high a level of end to end
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2489) assurance as the ARP monitor).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2490)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2491) 12.2 Maximum Throughput in a Multiple Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2492) -----------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2493)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2494) Multiple switches may be utilized to optimize for throughput
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2495) when they are configured in parallel as part of an isolated network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2496) between two or more systems, for example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2497)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2498) +-----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2499) | Host A |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2500) +-+---+---+-+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2501) | | |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2502) +--------+ | +---------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2503) | | |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2504) +------+---+ +-----+----+ +-----+----+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2505) | Switch A | | Switch B | | Switch C |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2506) +------+---+ +-----+----+ +-----+----+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2507) | | |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2508) +--------+ | +---------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2509) | | |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2510) +-+---+---+-+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2511) | Host B |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2512) +-----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2513)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2514) In this configuration, the switches are isolated from one
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2515) another. One reason to employ a topology such as this is for an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2516) isolated network with many hosts (a cluster configured for high
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2517) performance, for example), using multiple smaller switches can be more
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2518) cost effective than a single larger switch, e.g., on a network with 24
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2519) hosts, three 24 port switches can be significantly less expensive than
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2520) a single 72 port switch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2521)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2522) If access beyond the network is required, an individual host
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2523) can be equipped with an additional network device connected to an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2524) external network; this host then additionally acts as a gateway.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2525)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2526) 12.2.1 MT Bonding Mode Selection for Multiple Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2527) -------------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2528)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2529) In actual practice, the bonding mode typically employed in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2530) configurations of this type is balance-rr. Historically, in this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2531) network configuration, the usual caveats about out of order packet
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2532) delivery are mitigated by the use of network adapters that do not do
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2533) any kind of packet coalescing (via the use of NAPI, or because the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2534) device itself does not generate interrupts until some number of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2535) packets has arrived). When employed in this fashion, the balance-rr
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2536) mode allows individual connections between two hosts to effectively
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2537) utilize greater than one interface's bandwidth.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2538)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2539) 12.2.2 MT Link Monitoring for Multiple Switch Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2540) ------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2541)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2542) Again, in actual practice, the MII monitor is most often used
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2543) in this configuration, as performance is given preference over
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2544) availability. The ARP monitor will function in this topology, but its
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2545) advantages over the MII monitor are mitigated by the volume of probes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2546) needed as the number of systems involved grows (remember that each
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2547) host in the network is configured with bonding).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2548)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2549) 13. Switch Behavior Issues
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2550) ==========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2551)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2552) 13.1 Link Establishment and Failover Delays
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2553) -------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2554)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2555) Some switches exhibit undesirable behavior with regard to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2556) timing of link up and down reporting by the switch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2557)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2558) First, when a link comes up, some switches may indicate that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2559) the link is up (carrier available), but not pass traffic over the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2560) interface for some period of time. This delay is typically due to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2561) some type of autonegotiation or routing protocol, but may also occur
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2562) during switch initialization (e.g., during recovery after a switch
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2563) failure). If you find this to be a problem, specify an appropriate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2564) value to the updelay bonding module option to delay the use of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2565) relevant interface(s).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2566)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2567) Second, some switches may "bounce" the link state one or more
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2568) times while a link is changing state. This occurs most commonly while
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2569) the switch is initializing. Again, an appropriate updelay value may
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2570) help.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2571)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2572) Note that when a bonding interface has no active links, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2573) driver will immediately reuse the first link that goes up, even if the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2574) updelay parameter has been specified (the updelay is ignored in this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2575) case). If there are slave interfaces waiting for the updelay timeout
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2576) to expire, the interface that first went into that state will be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2577) immediately reused. This reduces down time of the network if the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2578) value of updelay has been overestimated, and since this occurs only in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2579) cases with no connectivity, there is no additional penalty for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2580) ignoring the updelay.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2581)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2582) In addition to the concerns about switch timings, if your
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2583) switches take a long time to go into backup mode, it may be desirable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2584) to not activate a backup interface immediately after a link goes down.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2585) Failover may be delayed via the downdelay bonding module option.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2586)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2587) 13.2 Duplicated Incoming Packets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2588) --------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2589)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2590) NOTE: Starting with version 3.0.2, the bonding driver has logic to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2591) suppress duplicate packets, which should largely eliminate this problem.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2592) The following description is kept for reference.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2593)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2594) It is not uncommon to observe a short burst of duplicated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2595) traffic when the bonding device is first used, or after it has been
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2596) idle for some period of time. This is most easily observed by issuing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2597) a "ping" to some other host on the network, and noticing that the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2598) output from ping flags duplicates (typically one per slave).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2599)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2600) For example, on a bond in active-backup mode with five slaves
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2601) all connected to one switch, the output may appear as follows::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2602)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2603) # ping -n 10.0.4.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2604) PING 10.0.4.2 (10.0.4.2) from 10.0.3.10 : 56(84) bytes of data.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2605) 64 bytes from 10.0.4.2: icmp_seq=1 ttl=64 time=13.7 ms
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2606) 64 bytes from 10.0.4.2: icmp_seq=1 ttl=64 time=13.8 ms (DUP!)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2607) 64 bytes from 10.0.4.2: icmp_seq=1 ttl=64 time=13.8 ms (DUP!)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2608) 64 bytes from 10.0.4.2: icmp_seq=1 ttl=64 time=13.8 ms (DUP!)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2609) 64 bytes from 10.0.4.2: icmp_seq=1 ttl=64 time=13.8 ms (DUP!)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2610) 64 bytes from 10.0.4.2: icmp_seq=2 ttl=64 time=0.216 ms
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2611) 64 bytes from 10.0.4.2: icmp_seq=3 ttl=64 time=0.267 ms
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2612) 64 bytes from 10.0.4.2: icmp_seq=4 ttl=64 time=0.222 ms
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2613)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2614) This is not due to an error in the bonding driver, rather, it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2615) is a side effect of how many switches update their MAC forwarding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2616) tables. Initially, the switch does not associate the MAC address in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2617) the packet with a particular switch port, and so it may send the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2618) traffic to all ports until its MAC forwarding table is updated. Since
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2619) the interfaces attached to the bond may occupy multiple ports on a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2620) single switch, when the switch (temporarily) floods the traffic to all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2621) ports, the bond device receives multiple copies of the same packet
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2622) (one per slave device).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2623)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2624) The duplicated packet behavior is switch dependent, some
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2625) switches exhibit this, and some do not. On switches that display this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2626) behavior, it can be induced by clearing the MAC forwarding table (on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2627) most Cisco switches, the privileged command "clear mac address-table
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2628) dynamic" will accomplish this).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2629)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2630) 14. Hardware Specific Considerations
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2631) ====================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2632)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2633) This section contains additional information for configuring
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2634) bonding on specific hardware platforms, or for interfacing bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2635) with particular switches or other devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2636)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2637) 14.1 IBM BladeCenter
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2638) --------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2639)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2640) This applies to the JS20 and similar systems.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2641)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2642) On the JS20 blades, the bonding driver supports only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2643) balance-rr, active-backup, balance-tlb and balance-alb modes. This is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2644) largely due to the network topology inside the BladeCenter, detailed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2645) below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2646)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2647) JS20 network adapter information
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2648) --------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2649)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2650) All JS20s come with two Broadcom Gigabit Ethernet ports
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2651) integrated on the planar (that's "motherboard" in IBM-speak). In the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2652) BladeCenter chassis, the eth0 port of all JS20 blades is hard wired to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2653) I/O Module #1; similarly, all eth1 ports are wired to I/O Module #2.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2654) An add-on Broadcom daughter card can be installed on a JS20 to provide
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2655) two more Gigabit Ethernet ports. These ports, eth2 and eth3, are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2656) wired to I/O Modules 3 and 4, respectively.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2657)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2658) Each I/O Module may contain either a switch or a passthrough
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2659) module (which allows ports to be directly connected to an external
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2660) switch). Some bonding modes require a specific BladeCenter internal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2661) network topology in order to function; these are detailed below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2662)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2663) Additional BladeCenter-specific networking information can be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2664) found in two IBM Redbooks (www.ibm.com/redbooks):
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2665)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2666) - "IBM eServer BladeCenter Networking Options"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2667) - "IBM eServer BladeCenter Layer 2-7 Network Switching"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2668)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2669) BladeCenter networking configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2670) ------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2671)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2672) Because a BladeCenter can be configured in a very large number
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2673) of ways, this discussion will be confined to describing basic
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2674) configurations.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2675)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2676) Normally, Ethernet Switch Modules (ESMs) are used in I/O
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2677) modules 1 and 2. In this configuration, the eth0 and eth1 ports of a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2678) JS20 will be connected to different internal switches (in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2679) respective I/O modules).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2680)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2681) A passthrough module (OPM or CPM, optical or copper,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2682) passthrough module) connects the I/O module directly to an external
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2683) switch. By using PMs in I/O module #1 and #2, the eth0 and eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2684) interfaces of a JS20 can be redirected to the outside world and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2685) connected to a common external switch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2686)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2687) Depending upon the mix of ESMs and PMs, the network will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2688) appear to bonding as either a single switch topology (all PMs) or as a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2689) multiple switch topology (one or more ESMs, zero or more PMs). It is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2690) also possible to connect ESMs together, resulting in a configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2691) much like the example in "High Availability in a Multiple Switch
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2692) Topology," above.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2693)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2694) Requirements for specific modes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2695) -------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2696)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2697) The balance-rr mode requires the use of passthrough modules
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2698) for devices in the bond, all connected to an common external switch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2699) That switch must be configured for "etherchannel" or "trunking" on the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2700) appropriate ports, as is usual for balance-rr.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2701)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2702) The balance-alb and balance-tlb modes will function with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2703) either switch modules or passthrough modules (or a mix). The only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2704) specific requirement for these modes is that all network interfaces
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2705) must be able to reach all destinations for traffic sent over the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2706) bonding device (i.e., the network must converge at some point outside
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2707) the BladeCenter).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2708)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2709) The active-backup mode has no additional requirements.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2710)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2711) Link monitoring issues
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2712) ----------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2713)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2714) When an Ethernet Switch Module is in place, only the ARP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2715) monitor will reliably detect link loss to an external switch. This is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2716) nothing unusual, but examination of the BladeCenter cabinet would
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2717) suggest that the "external" network ports are the ethernet ports for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2718) the system, when it fact there is a switch between these "external"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2719) ports and the devices on the JS20 system itself. The MII monitor is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2720) only able to detect link failures between the ESM and the JS20 system.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2721)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2722) When a passthrough module is in place, the MII monitor does
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2723) detect failures to the "external" port, which is then directly
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2724) connected to the JS20 system.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2725)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2726) Other concerns
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2727) --------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2728)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2729) The Serial Over LAN (SoL) link is established over the primary
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2730) ethernet (eth0) only, therefore, any loss of link to eth0 will result
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2731) in losing your SoL connection. It will not fail over with other
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2732) network traffic, as the SoL system is beyond the control of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2733) bonding driver.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2734)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2735) It may be desirable to disable spanning tree on the switch
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2736) (either the internal Ethernet Switch Module, or an external switch) to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2737) avoid fail-over delay issues when using bonding.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2738)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2739)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2740) 15. Frequently Asked Questions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2741) ==============================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2742)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2743) 1. Is it SMP safe?
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2744) -------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2745)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2746) Yes. The old 2.0.xx channel bonding patch was not SMP safe.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2747) The new driver was designed to be SMP safe from the start.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2748)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2749) 2. What type of cards will work with it?
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2750) -----------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2751)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2752) Any Ethernet type cards (you can even mix cards - a Intel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2753) EtherExpress PRO/100 and a 3com 3c905b, for example). For most modes,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2754) devices need not be of the same speed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2755)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2756) Starting with version 3.2.1, bonding also supports Infiniband
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2757) slaves in active-backup mode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2758)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2759) 3. How many bonding devices can I have?
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2760) ----------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2761)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2762) There is no limit.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2763)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2764) 4. How many slaves can a bonding device have?
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2765) ----------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2766)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2767) This is limited only by the number of network interfaces Linux
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2768) supports and/or the number of network cards you can place in your
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2769) system.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2770)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2771) 5. What happens when a slave link dies?
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2772) ----------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2773)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2774) If link monitoring is enabled, then the failing device will be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2775) disabled. The active-backup mode will fail over to a backup link, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2776) other modes will ignore the failed link. The link will continue to be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2777) monitored, and should it recover, it will rejoin the bond (in whatever
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2778) manner is appropriate for the mode). See the sections on High
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2779) Availability and the documentation for each mode for additional
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2780) information.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2781)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2782) Link monitoring can be enabled via either the miimon or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2783) arp_interval parameters (described in the module parameters section,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2784) above). In general, miimon monitors the carrier state as sensed by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2785) the underlying network device, and the arp monitor (arp_interval)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2786) monitors connectivity to another host on the local network.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2787)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2788) If no link monitoring is configured, the bonding driver will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2789) be unable to detect link failures, and will assume that all links are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2790) always available. This will likely result in lost packets, and a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2791) resulting degradation of performance. The precise performance loss
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2792) depends upon the bonding mode and network configuration.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2793)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2794) 6. Can bonding be used for High Availability?
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2795) ----------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2796)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2797) Yes. See the section on High Availability for details.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2798)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2799) 7. Which switches/systems does it work with?
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2800) ---------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2801)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2802) The full answer to this depends upon the desired mode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2803)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2804) In the basic balance modes (balance-rr and balance-xor), it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2805) works with any system that supports etherchannel (also called
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2806) trunking). Most managed switches currently available have such
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2807) support, and many unmanaged switches as well.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2808)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2809) The advanced balance modes (balance-tlb and balance-alb) do
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2810) not have special switch requirements, but do need device drivers that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2811) support specific features (described in the appropriate section under
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2812) module parameters, above).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2813)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2814) In 802.3ad mode, it works with systems that support IEEE
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2815) 802.3ad Dynamic Link Aggregation. Most managed and many unmanaged
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2816) switches currently available support 802.3ad.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2817)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2818) The active-backup mode should work with any Layer-II switch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2819)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2820) 8. Where does a bonding device get its MAC address from?
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2821) ---------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2822)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2823) When using slave devices that have fixed MAC addresses, or when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2824) the fail_over_mac option is enabled, the bonding device's MAC address is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2825) the MAC address of the active slave.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2826)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2827) For other configurations, if not explicitly configured (with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2828) ifconfig or ip link), the MAC address of the bonding device is taken from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2829) its first slave device. This MAC address is then passed to all following
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2830) slaves and remains persistent (even if the first slave is removed) until
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2831) the bonding device is brought down or reconfigured.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2832)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2833) If you wish to change the MAC address, you can set it with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2834) ifconfig or ip link::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2835)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2836) # ifconfig bond0 hw ether 00:11:22:33:44:55
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2837)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2838) # ip link set bond0 address 66:77:88:99:aa:bb
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2839)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2840) The MAC address can be also changed by bringing down/up the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2841) device and then changing its slaves (or their order)::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2842)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2843) # ifconfig bond0 down ; modprobe -r bonding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2844) # ifconfig bond0 .... up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2845) # ifenslave bond0 eth...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2846)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2847) This method will automatically take the address from the next
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2848) slave that is added.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2849)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2850) To restore your slaves' MAC addresses, you need to detach them
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2851) from the bond (``ifenslave -d bond0 eth0``). The bonding driver will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2852) then restore the MAC addresses that the slaves had before they were
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2853) enslaved.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2854)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2855) 16. Resources and Links
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2856) =======================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2857)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2858) The latest version of the bonding driver can be found in the latest
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2859) version of the linux kernel, found on http://kernel.org
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2860)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2861) The latest version of this document can be found in the latest kernel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2862) source (named Documentation/networking/bonding.rst).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2863)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2864) Discussions regarding the development of the bonding driver take place
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2865) on the main Linux network mailing list, hosted at vger.kernel.org. The list
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2866) address is:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2867)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2868) netdev@vger.kernel.org
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2869)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2870) The administrative interface (to subscribe or unsubscribe) can
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2871) be found at:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2872)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2873) http://vger.kernel.org/vger-lists.html#netdev
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags