Orange Pi5 kernel

^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   1) .. SPDX-License-Identifier: GPL-2.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   2) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   3) ====================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   4) HOWTO for the linux packet generator
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   5) ====================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   6) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   7) Enable CONFIG_NET_PKTGEN to compile and build pktgen either in-kernel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   8) or as a module.  A module is preferred; modprobe pktgen if needed.  Once
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   9) running, pktgen creates a thread for each CPU with affinity to that CPU.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  10) Monitoring and controlling is done via /proc.  It is easiest to select a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  11) suitable sample script and configure that.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  12) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  13) On a dual CPU::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  14) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  15)     ps aux | grep pkt
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  16)     root       129  0.3  0.0     0    0 ?        SW    2003 523:20 [kpktgend_0]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  17)     root       130  0.3  0.0     0    0 ?        SW    2003 509:50 [kpktgend_1]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  18) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  19) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  20) For monitoring and control pktgen creates::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  21) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  22) 	/proc/net/pktgen/pgctrl
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  23) 	/proc/net/pktgen/kpktgend_X
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  24) 	/proc/net/pktgen/ethX
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  25) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  26) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  27) Tuning NIC for max performance
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  28) ==============================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  29) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  30) The default NIC settings are (likely) not tuned for pktgen's artificial
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  31) overload type of benchmarking, as this could hurt the normal use-case.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  32) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  33) Specifically increasing the TX ring buffer in the NIC::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  34) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  35)  # ethtool -G ethX tx 1024
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  36) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  37) A larger TX ring can improve pktgen's performance, while it can hurt
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  38) in the general case, 1) because the TX ring buffer might get larger
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  39) than the CPU's L1/L2 cache, 2) because it allows more queueing in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  40) NIC HW layer (which is bad for bufferbloat).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  41) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  42) One should hesitate to conclude that packets/descriptors in the HW
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  43) TX ring cause delay.  Drivers usually delay cleaning up the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  44) ring-buffers for various performance reasons, and packets stalling
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  45) the TX ring might just be waiting for cleanup.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  46) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  47) This cleanup issue is specifically the case for the driver ixgbe
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  48) (Intel 82599 chip).  This driver (ixgbe) combines TX+RX ring cleanups,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  49) and the cleanup interval is affected by the ethtool --coalesce setting
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  50) of parameter "rx-usecs".
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  51) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  52) For ixgbe use e.g. "30" resulting in approx 33K interrupts/sec (1/30*10^6)::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  53) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  54)  # ethtool -C ethX rx-usecs 30
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  55) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  56) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  57) Kernel threads
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  58) ==============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  59) Pktgen creates a thread for each CPU with affinity to that CPU.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  60) Which is controlled through procfile /proc/net/pktgen/kpktgend_X.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  61) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  62) Example: /proc/net/pktgen/kpktgend_0::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  63) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  64)  Running:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  65)  Stopped: eth4@0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  66)  Result: OK: add_device=eth4@0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  67) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  68) Most important are the devices assigned to the thread.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  69) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  70) The two basic thread commands are:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  71) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  72)  * add_device DEVICE@NAME -- adds a single device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  73)  * rem_device_all         -- remove all associated devices
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  74) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  75) When adding a device to a thread, a corresponding procfile is created
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  76) which is used for configuring this device. Thus, device names need to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  77) be unique.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  78) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  79) To support adding the same device to multiple threads, which is useful
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  80) with multi queue NICs, the device naming scheme is extended with "@":
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  81) device@something
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  82) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  83) The part after "@" can be anything, but it is custom to use the thread
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  84) number.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  85) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  86) Viewing devices
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  87) ===============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  88) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  89) The Params section holds configured information.  The Current section
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  90) holds running statistics.  The Result is printed after a run or after
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  91) interruption.  Example::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  92) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  93)     /proc/net/pktgen/eth4@0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  94) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  95)     Params: count 100000  min_pkt_size: 60  max_pkt_size: 60
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  96) 	frags: 0  delay: 0  clone_skb: 64  ifname: eth4@0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  97) 	flows: 0 flowlen: 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  98) 	queue_map_min: 0  queue_map_max: 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  99) 	dst_min: 192.168.81.2  dst_max:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) 	src_min:   src_max:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) 	src_mac: 90:e2:ba:0a:56:b4 dst_mac: 00:1b:21:3c:9d:f8
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) 	udp_src_min: 9  udp_src_max: 109  udp_dst_min: 9  udp_dst_max: 9
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) 	src_mac_count: 0  dst_mac_count: 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) 	Flags: UDPSRC_RND  NO_TIMESTAMP  QUEUE_MAP_CPU
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105)     Current:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) 	pkts-sofar: 100000  errors: 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) 	started: 623913381008us  stopped: 623913396439us idle: 25us
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) 	seq_num: 100001  cur_dst_mac_offset: 0  cur_src_mac_offset: 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) 	cur_saddr: 192.168.8.3  cur_daddr: 192.168.81.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) 	cur_udp_dst: 9  cur_udp_src: 42
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) 	cur_queue_map: 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) 	flows: 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113)     Result: OK: 15430(c15405+d25) usec, 100000 (60byte,0frags)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114)     6480562pps 3110Mb/sec (3110669760bps) errors: 0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) Configuring devices
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) ===================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) This is done via the /proc interface, and most easily done via pgset
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) as defined in the sample scripts.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) You need to specify PGDEV environment variable to use functions from sample
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) scripts, i.e.::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124)     export PGDEV=/proc/net/pktgen/eth4@0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125)     source samples/pktgen/functions.sh
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) Examples::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129)  pg_ctrl start           starts injection.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130)  pg_ctrl stop            aborts injection. Also, ^C aborts generator.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132)  pgset "clone_skb 1"     sets the number of copies of the same packet
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133)  pgset "clone_skb 0"     use single SKB for all transmits
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134)  pgset "burst 8"         uses xmit_more API to queue 8 copies of the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) 			 packet and update HW tx queue tail pointer once.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) 			 "burst 1" is the default
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137)  pgset "pkt_size 9014"   sets packet size to 9014
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138)  pgset "frags 5"         packet will consist of 5 fragments
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139)  pgset "count 200000"    sets number of packets to send, set to zero
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) 			 for continuous sends until explicitly stopped.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142)  pgset "delay 5000"      adds delay to hard_start_xmit(). nanoseconds
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144)  pgset "dst 10.0.0.1"    sets IP destination address
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) 			 (BEWARE! This generator is very aggressive!)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147)  pgset "dst_min 10.0.0.1"            Same as dst
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148)  pgset "dst_max 10.0.0.254"          Set the maximum destination IP.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149)  pgset "src_min 10.0.0.1"            Set the minimum (or only) source IP.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150)  pgset "src_max 10.0.0.254"          Set the maximum source IP.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151)  pgset "dst6 fec0::1"     IPV6 destination address
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152)  pgset "src6 fec0::2"     IPV6 source address
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153)  pgset "dstmac 00:00:00:00:00:00"    sets MAC destination address
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154)  pgset "srcmac 00:00:00:00:00:00"    sets MAC source address
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156)  pgset "queue_map_min 0" Sets the min value of tx queue interval
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157)  pgset "queue_map_max 7" Sets the max value of tx queue interval, for multiqueue devices
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) 			 To select queue 1 of a given device,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) 			 use queue_map_min=1 and queue_map_max=1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161)  pgset "src_mac_count 1" Sets the number of MACs we'll range through.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162) 			 The 'minimum' MAC is what you set with srcmac.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164)  pgset "dst_mac_count 1" Sets the number of MACs we'll range through.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165) 			 The 'minimum' MAC is what you set with dstmac.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167)  pgset "flag [name]"     Set a flag to determine behaviour.  Current flags
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) 			 are: IPSRC_RND # IP source is random (between min/max)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169) 			      IPDST_RND # IP destination is random
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) 			      UDPSRC_RND, UDPDST_RND,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) 			      MACSRC_RND, MACDST_RND
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) 			      TXSIZE_RND, IPV6,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) 			      MPLS_RND, VID_RND, SVID_RND
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) 			      FLOW_SEQ,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) 			      QUEUE_MAP_RND # queue map random
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) 			      QUEUE_MAP_CPU # queue map mirrors smp_processor_id()
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) 			      UDPCSUM,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) 			      IPSEC # IPsec encapsulation (needs CONFIG_XFRM)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) 			      NODE_ALLOC # node specific memory allocation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) 			      NO_TIMESTAMP # disable timestamping
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181)  pgset 'flag ![name]'    Clear a flag to determine behaviour.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) 			 Note that you might need to use single quote in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) 			 interactive mode, so that your shell wouldn't expand
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) 			 the specified flag as a history command.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186)  pgset "spi [SPI_VALUE]" Set specific SA used to transform packet.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188)  pgset "udp_src_min 9"   set UDP source port min, If < udp_src_max, then
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) 			 cycle through the port range.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191)  pgset "udp_src_max 9"   set UDP source port max.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192)  pgset "udp_dst_min 9"   set UDP destination port min, If < udp_dst_max, then
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) 			 cycle through the port range.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194)  pgset "udp_dst_max 9"   set UDP destination port max.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196)  pgset "mpls 0001000a,0002000a,0000000a" set MPLS labels (in this example
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) 					 outer label=16,middle label=32,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) 					 inner label=0 (IPv4 NULL)) Note that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) 					 there must be no spaces between the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) 					 arguments. Leading zeros are required.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201) 					 Do not set the bottom of stack bit,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) 					 that's done automatically. If you do
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) 					 set the bottom of stack bit, that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) 					 indicates that you want to randomly
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205) 					 generate that address and the flag
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206) 					 MPLS_RND will be turned on. You
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207) 					 can have any mix of random and fixed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208) 					 labels in the label stack.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210)  pgset "mpls 0"		  turn off mpls (or any invalid argument works too!)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212)  pgset "vlan_id 77"       set VLAN ID 0-4095
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213)  pgset "vlan_p 3"         set priority bit 0-7 (default 0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214)  pgset "vlan_cfi 0"       set canonical format identifier 0-1 (default 0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216)  pgset "svlan_id 22"      set SVLAN ID 0-4095
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217)  pgset "svlan_p 3"        set priority bit 0-7 (default 0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218)  pgset "svlan_cfi 0"      set canonical format identifier 0-1 (default 0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220)  pgset "vlan_id 9999"     > 4095 remove vlan and svlan tags
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221)  pgset "svlan 9999"       > 4095 remove svlan tag
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224)  pgset "tos XX"           set former IPv4 TOS field (e.g. "tos 28" for AF11 no ECN, default 00)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225)  pgset "traffic_class XX" set former IPv6 TRAFFIC CLASS (e.g. "traffic_class B8" for EF no ECN, default 00)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227)  pgset "rate 300M"        set rate to 300 Mb/s
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228)  pgset "ratep 1000000"    set rate to 1Mpps
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230)  pgset "xmit_mode netif_receive"  RX inject into stack netif_receive_skb()
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231) 				  Works with "burst" but not with "clone_skb".
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232) 				  Default xmit_mode is "start_xmit".
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234) Sample scripts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235) ==============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237) A collection of tutorial scripts and helpers for pktgen is in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238) samples/pktgen directory. The helper parameters.sh file support easy
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) and consistent parameter parsing across the sample scripts.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241) Usage example and help::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243)  ./pktgen_sample01_simple.sh -i eth4 -m 00:1B:21:3C:9D:F8 -d 192.168.8.2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) Usage:::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247)   ./pktgen_sample01_simple.sh [-vx] -i ethX
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249)   -i : ($DEV)       output interface/device (required)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250)   -s : ($PKT_SIZE)  packet size
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251)   -d : ($DEST_IP)   destination IP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252)   -m : ($DST_MAC)   destination MAC-addr
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253)   -t : ($THREADS)   threads to start
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254)   -c : ($SKB_CLONE) SKB clones send before alloc new SKB
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255)   -b : ($BURST)     HW level bursting of SKBs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256)   -v : ($VERBOSE)   verbose
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257)   -x : ($DEBUG)     debug
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259) The global variables being set are also listed.  E.g. the required
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260) interface/device parameter "-i" sets variable $DEV.  Copy the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261) pktgen_sampleXX scripts and modify them to fit your own needs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263) The old scripts::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 264) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 265)     pktgen.conf-1-2                  # 1 CPU 2 dev
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 266)     pktgen.conf-1-1-rdos             # 1 CPU 1 dev w. route DoS
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 267)     pktgen.conf-1-1-ip6              # 1 CPU 1 dev ipv6
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 268)     pktgen.conf-1-1-ip6-rdos         # 1 CPU 1 dev ipv6  w. route DoS
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 269)     pktgen.conf-1-1-flows            # 1 CPU 1 dev multiple flows.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 270) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 271) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 272) Interrupt affinity
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 273) ===================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 274) Note that when adding devices to a specific CPU it is a good idea to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 275) also assign /proc/irq/XX/smp_affinity so that the TX interrupts are bound
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 276) to the same CPU.  This reduces cache bouncing when freeing skbs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 277) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 278) Plus using the device flag QUEUE_MAP_CPU, which maps the SKBs TX queue
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 279) to the running threads CPU (directly from smp_processor_id()).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 280) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 281) Enable IPsec
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 282) ============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 283) Default IPsec transformation with ESP encapsulation plus transport mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 284) can be enabled by simply setting::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 285) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 286)     pgset "flag IPSEC"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 287)     pgset "flows 1"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 288) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 289) To avoid breaking existing testbed scripts for using AH type and tunnel mode,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 290) you can use "pgset spi SPI_VALUE" to specify which transformation mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 291) to employ.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 292) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 293) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 294) Current commands and configuration options
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 295) ==========================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 296) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 297) **Pgcontrol commands**::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 298) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 299)     start
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 300)     stop
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 301)     reset
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 302) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 303) **Thread commands**::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 304) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 305)     add_device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 306)     rem_device_all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 307) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 308) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 309) **Device commands**::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 310) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 311)     count
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 312)     clone_skb
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 313)     burst
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 314)     debug
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 315) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 316)     frags
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 317)     delay
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 318) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 319)     src_mac_count
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 320)     dst_mac_count
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 321) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 322)     pkt_size
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 323)     min_pkt_size
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 324)     max_pkt_size
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 325) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 326)     queue_map_min
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 327)     queue_map_max
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 328)     skb_priority
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 329) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 330)     tos           (ipv4)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 331)     traffic_class (ipv6)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 332) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 333)     mpls
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 334) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 335)     udp_src_min
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 336)     udp_src_max
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 337) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 338)     udp_dst_min
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 339)     udp_dst_max
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 340) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 341)     node
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 342) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 343)     flag
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 344)     IPSRC_RND
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 345)     IPDST_RND
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 346)     UDPSRC_RND
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 347)     UDPDST_RND
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 348)     MACSRC_RND
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 349)     MACDST_RND
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 350)     TXSIZE_RND
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 351)     IPV6
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 352)     MPLS_RND
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 353)     VID_RND
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 354)     SVID_RND
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 355)     FLOW_SEQ
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 356)     QUEUE_MAP_RND
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 357)     QUEUE_MAP_CPU
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 358)     UDPCSUM
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 359)     IPSEC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 360)     NODE_ALLOC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 361)     NO_TIMESTAMP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 362) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 363)     spi (ipsec)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 364) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 365)     dst_min
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 366)     dst_max
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 367) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 368)     src_min
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 369)     src_max
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 370) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 371)     dst_mac
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 372)     src_mac
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 373) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 374)     clear_counters
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 375) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 376)     src6
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 377)     dst6
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 378)     dst6_max
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 379)     dst6_min
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 380) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 381)     flows
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 382)     flowlen
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 383) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 384)     rate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 385)     ratep
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 386) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 387)     xmit_mode <start_xmit|netif_receive>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 388) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 389)     vlan_cfi
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 390)     vlan_id
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 391)     vlan_p
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 392) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 393)     svlan_cfi
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 394)     svlan_id
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 395)     svlan_p
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 396) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 397) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 398) References:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 399) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 400) - ftp://robur.slu.se/pub/Linux/net-development/pktgen-testing/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 401) - tp://robur.slu.se/pub/Linux/net-development/pktgen-testing/examples/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 402) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 403) Paper from Linux-Kongress in Erlangen 2004.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 404) - ftp://robur.slu.se/pub/Linux/net-development/pktgen-testing/pktgen_paper.pdf
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 405) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 406) Thanks to:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 407) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 408) Grant Grundler for testing on IA-64 and parisc, Harald Welte,  Lennert Buytenhek
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 409) Stephen Hemminger, Andi Kleen, Dave Miller and many others.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 410) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 411) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 412) Good luck with the linux net-development.