My outbound to tun0 interface has been limited to 330Mbps by carrier company.
I want to keep OSPF/BGP packet when the interface is under heavy load. So I wrote this policy, and it result a commit error.
vyos@bsp-asbr2-cm# show qos policy
shaper test {
bandwidth 330mbit
class 10 {
bandwidth 320mbit
match ospf {
ip {
protocol ospf
}
}
queue-type fair-queue
}
default {
bandwidth 300mbit
queue-type fair-queue
}
}
[edit]
vyos@bsp-asbr2-cm# set qos int tun0 eg test
[edit]
vyos@bsp-asbr2-cm# commit
[ qos ]
VyOS had an issue completing a command.
Report time: 2023-03-03 10:31:49
Image version: VyOS 1.4-rolling-202302150317
Release train: current
Built by: [email protected]
Built on: Wed 15 Feb 2023 03:17 UTC
Build UUID: e62b2d4d-c09c-4dd6-a722-884b782e4d13
Build commit ID: 5207b6f510d677
Architecture: x86_64
Boot via: installed image
System type: VMware guest
Hardware vendor: VMware, Inc.
Hardware model: VMware20,1
Hardware S/N: VMware-56 4d fb 13 43 c4 e5 63-b0 9e 49 a8 b7 33 09 da
Hardware UUID: 13fb4d56-c443-63e5-b09e-49a8b73309da
Traceback (most recent call last):
File "/usr/libexec/vyos/conf_mode/qos.py", line 255, in <module>
apply(c)
File "/usr/libexec/vyos/conf_mode/qos.py", line 246, in apply
tmp.update(shaper_config, direction)
File "/usr/lib/python3/dist-packages/vyos/qos/trafficshaper.py", line 100, in update
super().update(config, direction)
File "/usr/lib/python3/dist-packages/vyos/qos/base.py", line 246, in update
self._cmd(filter_cmd)
File "/usr/lib/python3/dist-packages/vyos/qos/base.py", line 36, in _cmd
return cmd(command)
File "/usr/lib/python3/dist-packages/vyos/util.py", line 161, in cmd
raise OSError(code, feedback)
PermissionError: [Errno 1] failed to run command: tc filter replace dev tun0 parent 1: protocol all u32 match ip protocol ospf 0xff action police rate 320000000 burst 15k flowid 1:a
returned:
exit code: 1
noteworthy:
cmd 'tc qdisc del dev lo parent ffff:'
returned (out):
returned (err):
Error: Cannot find specified qdisc on specified device.
cmd 'tc filter replace dev tun0 parent 1: protocol all u32 match ip protocol ospf 0xff action police rate 320000000 burst 15k flowid 1:a'
returned (out):
returned (err):
Illegal "match"
[[qos]] failed
Commit failed
[edit]
vyos@bsp-asbr2-cm#Btw, when I enable a 300mbit/s shaper on tunnel, the actual limit is around 140-150mbit/s, iperf3 said lots of retry.
vyos@bsp-asbr2-cm# show qos
interface tun0 {
egress test
}
policy {
shaper test {
bandwidth 330mbit
default {
bandwidth 300mbit
queue-type fair-queue
}
}
}
[edit]
vyos@bsp-asbr2-cm#hikari@mynas:~$ iperf3 -c 198.18.0.10 -t 100 Connecting to host 198.18.0.10, port 5201 [ 5] local 10.96.4.101 port 49502 connected to 198.18.0.10 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 19.5 MBytes 164 Mbits/sec 2459 1.51 MBytes [ 5] 1.00-2.00 sec 16.2 MBytes 136 Mbits/sec 1891 1.16 MBytes [ 5] 2.00-3.00 sec 17.5 MBytes 147 Mbits/sec 660 1.13 MBytes [ 5] 3.00-4.00 sec 17.5 MBytes 147 Mbits/sec 1263 1.22 MBytes [ 5] 4.00-5.00 sec 16.2 MBytes 136 Mbits/sec 1423 1.20 MBytes [ 5] 5.00-6.00 sec 17.5 MBytes 147 Mbits/sec 1561 1.21 MBytes [ 5] 6.00-7.00 sec 17.5 MBytes 147 Mbits/sec 1588 1.25 MBytes [ 5] 7.00-8.00 sec 17.5 MBytes 147 Mbits/sec 1616 1.27 MBytes [ 5] 8.00-9.00 sec 17.5 MBytes 147 Mbits/sec 2112 1.28 MBytes [ 5] 9.00-10.00 sec 17.5 MBytes 147 Mbits/sec 2206 1.31 MBytes [ 5] 10.00-11.00 sec 12.5 MBytes 105 Mbits/sec 1133 1.18 MBytes ^C[ 5] 11.00-11.60 sec 10.0 MBytes 140 Mbits/sec 770 1.28 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-11.60 sec 197 MBytes 142 Mbits/sec 18682 sender [ 5] 0.00-11.60 sec 0.00 Bytes 0.00 bits/sec receiver iperf3: interrupt - the client has terminated hikari@mynas:~$
Iperf3 result when I disabled this qos rule:
hikari@mynas:~$ iperf3 -c 198.18.0.10 -t 100 Connecting to host 198.18.0.10, port 5201 [ 5] local 10.96.4.101 port 49762 connected to 198.18.0.10 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 35.8 MBytes 300 Mbits/sec 0 4.92 MBytes [ 5] 1.00-2.00 sec 42.5 MBytes 357 Mbits/sec 1 4.92 MBytes [ 5] 2.00-3.00 sec 42.5 MBytes 357 Mbits/sec 0 4.92 MBytes [ 5] 3.00-4.00 sec 38.8 MBytes 325 Mbits/sec 0 4.92 MBytes [ 5] 4.00-5.00 sec 45.0 MBytes 378 Mbits/sec 0 4.92 MBytes [ 5] 5.00-6.00 sec 42.5 MBytes 357 Mbits/sec 0 4.92 MBytes [ 5] 6.00-7.00 sec 42.5 MBytes 357 Mbits/sec 0 4.92 MBytes [ 5] 7.00-8.00 sec 42.5 MBytes 357 Mbits/sec 0 4.92 MBytes [ 5] 8.00-9.00 sec 42.5 MBytes 357 Mbits/sec 0 4.92 MBytes [ 5] 9.00-10.00 sec 42.5 MBytes 357 Mbits/sec 0 4.92 MBytes [ 5] 10.00-11.00 sec 32.5 MBytes 273 Mbits/sec 0 5.14 MBytes ^C[ 5] 11.00-11.30 sec 12.5 MBytes 353 Mbits/sec 0 5.14 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-11.30 sec 462 MBytes 343 Mbits/sec 1 sender [ 5] 0.00-11.30 sec 0.00 Bytes 0.00 bits/sec receiver iperf3: interrupt - the client has terminated hikari@mynas:~$
My hardware platform is Supermicro's X10SDV with Intel Xeon D1521 CPU, hypervisor configuration for this instance is 2vCPU+1GB of RAM.
Because the IPsec performance in normal setting is too poor and I changed kernel cmdline to this:
vyos@bsp-asbr2-cm:~$ cat /proc/cmdline BOOT_IMAGE=/boot/1.4-rolling-202302150317/vmlinuz boot=live rootdelay=5 noautologin net.ifnames=0 biosdevname=0 vyos-union=/boot/1.4-rolling-202302150317 console=ttyS0,115200 console=tty0 mitigations=off vyos@bsp-asbr2-cm:~$
Btw, if I manually set shaper rule by hand, it works correctly:
sudo tc qdisc add root dev tun0 handle 1: htb default 2 sudo tc class add dev tun0 parent 1: classid 1:1 htb rate 330mbit sudo tc class add dev tun0 parent 1:1 classid 1:2 htb rate 300mbit sudo tc class add dev tun0 parent 1:1 classid 1:3 htb rate 320mbit