Orange Pi5 kernel

Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards

3 Commits   0 Branches   0 Tags
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   1) .. SPDX-License-Identifier: GPL-2.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   2) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   3) ====================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   4) Virtual Routing and Forwarding (VRF)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   5) ====================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   6) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   7) The VRF Device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   8) ==============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   9) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  10) The VRF device combined with ip rules provides the ability to create virtual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  11) routing and forwarding domains (aka VRFs, VRF-lite to be specific) in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  12) Linux network stack. One use case is the multi-tenancy problem where each
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  13) tenant has their own unique routing tables and in the very least need
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  14) different default gateways.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  15) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  16) Processes can be "VRF aware" by binding a socket to the VRF device. Packets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  17) through the socket then use the routing table associated with the VRF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  18) device. An important feature of the VRF device implementation is that it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  19) impacts only Layer 3 and above so L2 tools (e.g., LLDP) are not affected
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  20) (ie., they do not need to be run in each VRF). The design also allows
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  21) the use of higher priority ip rules (Policy Based Routing, PBR) to take
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  22) precedence over the VRF device rules directing specific traffic as desired.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  23) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  24) In addition, VRF devices allow VRFs to be nested within namespaces. For
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  25) example network namespaces provide separation of network interfaces at the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  26) device layer, VLANs on the interfaces within a namespace provide L2 separation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  27) and then VRF devices provide L3 separation.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  28) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  29) Design
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  30) ------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  31) A VRF device is created with an associated route table. Network interfaces
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  32) are then enslaved to a VRF device::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  33) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  34) 	 +-----------------------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  35) 	 |           vrf-blue          |  ===> route table 10
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  36) 	 +-----------------------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  37) 	    |        |            |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  38) 	 +------+ +------+     +-------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  39) 	 | eth1 | | eth2 | ... |    bond1    |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  40) 	 +------+ +------+     +-------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  41) 				  |       |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  42) 			      +------+ +------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  43) 			      | eth8 | | eth9 |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  44) 			      +------+ +------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  45) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  46) Packets received on an enslaved device and are switched to the VRF device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  47) in the IPv4 and IPv6 processing stacks giving the impression that packets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  48) flow through the VRF device. Similarly on egress routing rules are used to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  49) send packets to the VRF device driver before getting sent out the actual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  50) interface. This allows tcpdump on a VRF device to capture all packets into
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  51) and out of the VRF as a whole\ [1]_. Similarly, netfilter\ [2]_ and tc rules
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  52) can be applied using the VRF device to specify rules that apply to the VRF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  53) domain as a whole.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  54) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  55) .. [1] Packets in the forwarded state do not flow through the device, so those
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  56)        packets are not seen by tcpdump. Will revisit this limitation in a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  57)        future release.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  58) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  59) .. [2] Iptables on ingress supports PREROUTING with skb->dev set to the real
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  60)        ingress device and both INPUT and PREROUTING rules with skb->dev set to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  61)        the VRF device. For egress POSTROUTING and OUTPUT rules can be written
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  62)        using either the VRF device or real egress device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  63) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  64) Setup
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  65) -----
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  66) 1. VRF device is created with an association to a FIB table.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  67)    e.g,::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  68) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  69) 	ip link add vrf-blue type vrf table 10
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  70) 	ip link set dev vrf-blue up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  71) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  72) 2. An l3mdev FIB rule directs lookups to the table associated with the device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  73)    A single l3mdev rule is sufficient for all VRFs. The VRF device adds the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  74)    l3mdev rule for IPv4 and IPv6 when the first device is created with a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  75)    default preference of 1000. Users may delete the rule if desired and add
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  76)    with a different priority or install per-VRF rules.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  77) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  78)    Prior to the v4.8 kernel iif and oif rules are needed for each VRF device::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  79) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  80)        ip ru add oif vrf-blue table 10
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  81)        ip ru add iif vrf-blue table 10
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  82) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  83) 3. Set the default route for the table (and hence default route for the VRF)::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  84) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  85)        ip route add table 10 unreachable default metric 4278198272
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  86) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  87)    This high metric value ensures that the default unreachable route can
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  88)    be overridden by a routing protocol suite.  FRRouting interprets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  89)    kernel metrics as a combined admin distance (upper byte) and priority
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  90)    (lower 3 bytes).  Thus the above metric translates to [255/8192].
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  91) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  92) 4. Enslave L3 interfaces to a VRF device::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  93) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  94)        ip link set dev eth1 master vrf-blue
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  95) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  96)    Local and connected routes for enslaved devices are automatically moved to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  97)    the table associated with VRF device. Any additional routes depending on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  98)    the enslaved device are dropped and will need to be reinserted to the VRF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  99)    FIB table following the enslavement.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101)    The IPv6 sysctl option keep_addr_on_down can be enabled to keep IPv6 global
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)    addresses as VRF enslavement changes::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104)        sysctl -w net.ipv6.conf.all.keep_addr_on_down=1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) 5. Additional VRF routes are added to associated table::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108)        ip route add table 10 ...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) Applications
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) ------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) Applications that are to work within a VRF need to bind their socket to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) VRF device::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116)     setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) or to specify the output device using cmsg and IP_PKTINFO.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) By default the scope of the port bindings for unbound sockets is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) limited to the default VRF. That is, it will not be matched by packets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) arriving on interfaces enslaved to an l3mdev and processes may bind to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) the same port if they bind to an l3mdev.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) TCP & UDP services running in the default VRF context (ie., not bound
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) to any VRF device) can work across all VRF domains by enabling the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) tcp_l3mdev_accept and udp_l3mdev_accept sysctl options::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129)     sysctl -w net.ipv4.tcp_l3mdev_accept=1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130)     sysctl -w net.ipv4.udp_l3mdev_accept=1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) These options are disabled by default so that a socket in a VRF is only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) selected for packets in that VRF. There is a similar option for RAW
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) sockets, which is enabled by default for reasons of backwards compatibility.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) This is so as to specify the output device with cmsg and IP_PKTINFO, but
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) using a socket not bound to the corresponding VRF. This allows e.g. older ping
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) implementations to be run with specifying the device but without executing it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) in the VRF. This option can be disabled so that packets received in a VRF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) context are only handled by a raw socket bound to the VRF, and packets in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) default VRF are only handled by a socket not bound to any VRF::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142)     sysctl -w net.ipv4.raw_l3mdev_accept=0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) netfilter rules on the VRF device can be used to limit access to services
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) running in the default VRF context as well.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) --------------------------------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) Using iproute2 for VRFs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) =======================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) iproute2 supports the vrf keyword as of v4.7. For backwards compatibility this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) section lists both commands where appropriate -- with the vrf keyword and the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) older form without it.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) 1. Create a VRF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157)    To instantiate a VRF device and associate it with a table::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159)        $ ip link add dev NAME type vrf table ID
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161)    As of v4.8 the kernel supports the l3mdev FIB rule where a single rule
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)    covers all VRFs. The l3mdev rule is created for IPv4 and IPv6 on first
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163)    device create.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165) 2. List VRFs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167)    To list VRFs that have been created::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)        $ ip [-d] link show type vrf
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) 	 NOTE: The -d option is needed to show the table id
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172)    For example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174)        $ ip -d link show type vrf
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175)        11: mgmt: <NOARP,MASTER,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) 	   link/ether 72:b3:ba:91:e2:24 brd ff:ff:ff:ff:ff:ff promiscuity 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) 	   vrf table 1 addrgenmode eui64
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178)        12: red: <NOARP,MASTER,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) 	   link/ether b6:6f:6e:f6:da:73 brd ff:ff:ff:ff:ff:ff promiscuity 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) 	   vrf table 10 addrgenmode eui64
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181)        13: blue: <NOARP,MASTER,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) 	   link/ether 36:62:e8:7d:bb:8c brd ff:ff:ff:ff:ff:ff promiscuity 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) 	   vrf table 66 addrgenmode eui64
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184)        14: green: <NOARP,MASTER,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) 	   link/ether e6:28:b8:63:70:bb brd ff:ff:ff:ff:ff:ff promiscuity 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) 	   vrf table 81 addrgenmode eui64
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189)    Or in brief output::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191)        $ ip -br link show type vrf
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192)        mgmt         UP             72:b3:ba:91:e2:24 <NOARP,MASTER,UP,LOWER_UP>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193)        red          UP             b6:6f:6e:f6:da:73 <NOARP,MASTER,UP,LOWER_UP>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194)        blue         UP             36:62:e8:7d:bb:8c <NOARP,MASTER,UP,LOWER_UP>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195)        green        UP             e6:28:b8:63:70:bb <NOARP,MASTER,UP,LOWER_UP>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) 3. Assign a Network Interface to a VRF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200)    Network interfaces are assigned to a VRF by enslaving the netdevice to a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201)    VRF device::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203)        $ ip link set dev NAME master NAME
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205)    On enslavement connected and local routes are automatically moved to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206)    table associated with the VRF device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208)    For example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210)        $ ip link set dev eth0 master mgmt
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213) 4. Show Devices Assigned to a VRF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215)    To show devices that have been assigned to a specific VRF add the master
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216)    option to the ip command::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218)        $ ip link show vrf NAME
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219)        $ ip link show master NAME
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221)    For example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223)        $ ip link show vrf red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224)        3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP mode DEFAULT group default qlen 1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) 	   link/ether 02:00:00:00:02:02 brd ff:ff:ff:ff:ff:ff
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226)        4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP mode DEFAULT group default qlen 1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227) 	   link/ether 02:00:00:00:02:03 brd ff:ff:ff:ff:ff:ff
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228)        7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop master red state DOWN mode DEFAULT group default qlen 1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229) 	   link/ether 02:00:00:00:02:06 brd ff:ff:ff:ff:ff:ff
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232)    Or using the brief output::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234)        $ ip -br link show vrf red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235)        eth1             UP             02:00:00:00:02:02 <BROADCAST,MULTICAST,UP,LOWER_UP>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236)        eth2             UP             02:00:00:00:02:03 <BROADCAST,MULTICAST,UP,LOWER_UP>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237)        eth5             DOWN           02:00:00:00:02:06 <BROADCAST,MULTICAST>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240) 5. Show Neighbor Entries for a VRF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242)    To list neighbor entries associated with devices enslaved to a VRF device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243)    add the master option to the ip command::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245)        $ ip [-6] neigh show vrf NAME
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246)        $ ip [-6] neigh show master NAME
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248)    For example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250)        $  ip neigh show vrf red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251)        10.2.1.254 dev eth1 lladdr a6:d9:c7:4f:06:23 REACHABLE
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252)        10.2.2.254 dev eth2 lladdr 5e:54:01:6a:ee:80 REACHABLE
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254)        $ ip -6 neigh show vrf red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255)        2002:1::64 dev eth1 lladdr a6:d9:c7:4f:06:23 REACHABLE
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258) 6. Show Addresses for a VRF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260)    To show addresses for interfaces associated with a VRF add the master
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261)    option to the ip command::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263)        $ ip addr show vrf NAME
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 264)        $ ip addr show master NAME
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 265) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 266)    For example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 267) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 268) 	$ ip addr show vrf red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 269) 	3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 270) 	    link/ether 02:00:00:00:02:02 brd ff:ff:ff:ff:ff:ff
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 271) 	    inet 10.2.1.2/24 brd 10.2.1.255 scope global eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 272) 	       valid_lft forever preferred_lft forever
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 273) 	    inet6 2002:1::2/120 scope global
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 274) 	       valid_lft forever preferred_lft forever
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 275) 	    inet6 fe80::ff:fe00:202/64 scope link
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 276) 	       valid_lft forever preferred_lft forever
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 277) 	4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 278) 	    link/ether 02:00:00:00:02:03 brd ff:ff:ff:ff:ff:ff
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 279) 	    inet 10.2.2.2/24 brd 10.2.2.255 scope global eth2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 280) 	       valid_lft forever preferred_lft forever
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 281) 	    inet6 2002:2::2/120 scope global
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 282) 	       valid_lft forever preferred_lft forever
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 283) 	    inet6 fe80::ff:fe00:203/64 scope link
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 284) 	       valid_lft forever preferred_lft forever
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 285) 	7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop master red state DOWN group default qlen 1000
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 286) 	    link/ether 02:00:00:00:02:06 brd ff:ff:ff:ff:ff:ff
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 287) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 288)    Or in brief format::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 289) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 290) 	$ ip -br addr show vrf red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 291) 	eth1             UP             10.2.1.2/24 2002:1::2/120 fe80::ff:fe00:202/64
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 292) 	eth2             UP             10.2.2.2/24 2002:2::2/120 fe80::ff:fe00:203/64
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 293) 	eth5             DOWN
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 294) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 295) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 296) 7. Show Routes for a VRF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 297) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 298)    To show routes for a VRF use the ip command to display the table associated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 299)    with the VRF device::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 300) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 301)        $ ip [-6] route show vrf NAME
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 302)        $ ip [-6] route show table ID
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 303) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 304)    For example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 305) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 306) 	$ ip route show vrf red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 307) 	unreachable default  metric 4278198272
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 308) 	broadcast 10.2.1.0 dev eth1  proto kernel  scope link  src 10.2.1.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 309) 	10.2.1.0/24 dev eth1  proto kernel  scope link  src 10.2.1.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 310) 	local 10.2.1.2 dev eth1  proto kernel  scope host  src 10.2.1.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 311) 	broadcast 10.2.1.255 dev eth1  proto kernel  scope link  src 10.2.1.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 312) 	broadcast 10.2.2.0 dev eth2  proto kernel  scope link  src 10.2.2.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 313) 	10.2.2.0/24 dev eth2  proto kernel  scope link  src 10.2.2.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 314) 	local 10.2.2.2 dev eth2  proto kernel  scope host  src 10.2.2.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 315) 	broadcast 10.2.2.255 dev eth2  proto kernel  scope link  src 10.2.2.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 316) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 317) 	$ ip -6 route show vrf red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 318) 	local 2002:1:: dev lo  proto none  metric 0  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 319) 	local 2002:1::2 dev lo  proto none  metric 0  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 320) 	2002:1::/120 dev eth1  proto kernel  metric 256  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 321) 	local 2002:2:: dev lo  proto none  metric 0  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 322) 	local 2002:2::2 dev lo  proto none  metric 0  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 323) 	2002:2::/120 dev eth2  proto kernel  metric 256  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 324) 	local fe80:: dev lo  proto none  metric 0  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 325) 	local fe80:: dev lo  proto none  metric 0  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 326) 	local fe80::ff:fe00:202 dev lo  proto none  metric 0  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 327) 	local fe80::ff:fe00:203 dev lo  proto none  metric 0  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 328) 	fe80::/64 dev eth1  proto kernel  metric 256  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 329) 	fe80::/64 dev eth2  proto kernel  metric 256  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 330) 	ff00::/8 dev red  metric 256  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 331) 	ff00::/8 dev eth1  metric 256  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 332) 	ff00::/8 dev eth2  metric 256  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 333) 	unreachable default dev lo  metric 4278198272  error -101 pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 334) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 335) 8. Route Lookup for a VRF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 336) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 337)    A test route lookup can be done for a VRF::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 338) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 339)        $ ip [-6] route get vrf NAME ADDRESS
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 340)        $ ip [-6] route get oif NAME ADDRESS
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 341) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 342)    For example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 343) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 344) 	$ ip route get 10.2.1.40 vrf red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 345) 	10.2.1.40 dev eth1  table red  src 10.2.1.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 346) 	    cache
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 347) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 348) 	$ ip -6 route get 2002:1::32 vrf red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 349) 	2002:1::32 from :: dev eth1  table red  proto kernel  src 2002:1::2  metric 256  pref medium
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 350) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 351) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 352) 9. Removing Network Interface from a VRF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 353) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 354)    Network interfaces are removed from a VRF by breaking the enslavement to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 355)    the VRF device::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 356) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 357)        $ ip link set dev NAME nomaster
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 358) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 359)    Connected routes are moved back to the default table and local entries are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 360)    moved to the local table.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 361) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 362)    For example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 363) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 364)     $ ip link set dev eth0 nomaster
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 365) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 366) --------------------------------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 367) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 368) Commands used in this example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 369) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 370)      cat >> /etc/iproute2/rt_tables.d/vrf.conf <<EOF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 371)      1  mgmt
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 372)      10 red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 373)      66 blue
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 374)      81 green
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 375)      EOF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 376) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 377)      function vrf_create
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 378)      {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 379) 	 VRF=$1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 380) 	 TBID=$2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 381) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 382) 	 # create VRF device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 383) 	 ip link add ${VRF} type vrf table ${TBID}
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 384) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 385) 	 if [ "${VRF}" != "mgmt" ]; then
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 386) 	     ip route add table ${TBID} unreachable default metric 4278198272
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 387) 	 fi
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 388) 	 ip link set dev ${VRF} up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 389)      }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 390) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 391)      vrf_create mgmt 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 392)      ip link set dev eth0 master mgmt
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 393) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 394)      vrf_create red 10
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 395)      ip link set dev eth1 master red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 396)      ip link set dev eth2 master red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 397)      ip link set dev eth5 master red
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 398) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 399)      vrf_create blue 66
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 400)      ip link set dev eth3 master blue
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 401) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 402)      vrf_create green 81
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 403)      ip link set dev eth4 master green
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 404) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 405) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 406)      Interface addresses from /etc/network/interfaces:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 407)      auto eth0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 408)      iface eth0 inet static
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 409) 	   address 10.0.0.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 410) 	   netmask 255.255.255.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 411) 	   gateway 10.0.0.254
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 412) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 413)      iface eth0 inet6 static
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 414) 	   address 2000:1::2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 415) 	   netmask 120
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 416) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 417)      auto eth1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 418)      iface eth1 inet static
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 419) 	   address 10.2.1.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 420) 	   netmask 255.255.255.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 421) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 422)      iface eth1 inet6 static
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 423) 	   address 2002:1::2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 424) 	   netmask 120
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 425) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 426)      auto eth2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 427)      iface eth2 inet static
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 428) 	   address 10.2.2.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 429) 	   netmask 255.255.255.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 430) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 431)      iface eth2 inet6 static
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 432) 	   address 2002:2::2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 433) 	   netmask 120
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 434) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 435)      auto eth3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 436)      iface eth3 inet static
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 437) 	   address 10.2.3.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 438) 	   netmask 255.255.255.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 439) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 440)      iface eth3 inet6 static
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 441) 	   address 2002:3::2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 442) 	   netmask 120
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 443) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 444)      auto eth4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 445)      iface eth4 inet static
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 446) 	   address 10.2.4.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 447) 	   netmask 255.255.255.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 448) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 449)      iface eth4 inet6 static
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 450) 	   address 2002:4::2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 451) 	   netmask 120