My outbound to tun0 interface has been limited to 330Mbps by carrier company.
I want to keep OSPF/BGP packet when the interface is under heavy load. So I wrote this policy, and it result a commit error.
```
vyos@bsp-asbr2-cm# show qos policy
shaper test {
bandwidth 330mbit
class 10 {
bandwidth 320mbit
match ospf {
ip {
protocol ospf
}
}
queue-type fair-queue
}
default {
bandwidth 300mbit
queue-type fair-queue
}
}
[edit]
vyos@bsp-asbr2-cm# set qos int tun0 eg test
[edit]
vyos@bsp-asbr2-cm# commit
[ qos ]
VyOS had an issue completing a command.
We are sorry that you encountered a problem while using VyOS.
There are a few things you can do to help us (and yourself):
- Contact us using the online help desk if you have a subscription:
https://support.vyos.io/
- Make sure you are running the latest version of VyOS available at:
https://vyos.net/get/
- Consult the community forum to see how to handle this issue:
https://forum.vyos.io
- Join us on Slack where our users exchange help and advice:
https://vyos.slack.com
When reporting problems, please include as much information as possible:
- do not obfuscate any data (feel free to contact us privately if your
business policy requires it)
- and include all the information presented below
Report time: 2023-03-03 10:31:49
Image version: VyOS 1.4-rolling-202302150317
Release train: current
Built by:
[email protected]
Built on: Wed 15 Feb 2023 03:17 UTC
Build UUID: e62b2d4d-c09c-4dd6-a722-884b782e4d13
Build commit ID: 5207b6f510d677
Architecture: x86_64
Boot via: installed image
System type: VMware guest
Hardware vendor: VMware, Inc.
Hardware model: VMware20,1
Hardware S/N: VMware-56 4d fb 13 43 c4 e5 63-b0 9e 49 a8 b7 33 09 da
Hardware UUID: 13fb4d56-c443-63e5-b09e-49a8b73309da
Traceback (most recent call last):
File "/usr/libexec/vyos/conf_mode/qos.py", line 255, in <module>
apply(c)
File "/usr/libexec/vyos/conf_mode/qos.py", line 246, in apply
tmp.update(shaper_config, direction)
File "/usr/lib/python3/dist-packages/vyos/qos/trafficshaper.py", line 100, in update
super().update(config, direction)
File "/usr/lib/python3/dist-packages/vyos/qos/base.py", line 246, in update
self._cmd(filter_cmd)
File "/usr/lib/python3/dist-packages/vyos/qos/base.py", line 36, in _cmd
return cmd(command)
File "/usr/lib/python3/dist-packages/vyos/util.py", line 161, in cmd
raise OSError(code, feedback)
PermissionError: [Errno 1] failed to run command: tc filter replace dev tun0 parent 1: protocol all u32 match ip protocol ospf 0xff action police rate 320000000 burst 15k flowid 1:a
returned:
exit code: 1
noteworthy:
cmd 'tc qdisc del dev lo parent ffff:'
returned (out):
returned (err):
Error: Cannot find specified qdisc on specified device.
cmd 'tc qdisc del dev lo root'
returned (out):
returned (err):
Error: Cannot delete qdisc with handle of zero.
cmd 'tc qdisc del dev eth0 parent ffff:'
returned (out):
returned (err):
Error: Cannot find specified qdisc on specified device.
cmd 'tc qdisc del dev eth0 root'
returned (out):
returned (err):
Error: Cannot delete qdisc with handle of zero.
cmd 'tc qdisc del dev eth1 parent ffff:'
returned (out):
returned (err):
Error: Cannot delete qdisc with handle of zero.
cmd 'tc qdisc del dev eth1 root'
returned (out):
returned (err):
Error: Cannot delete qdisc with handle of zero.
cmd 'tc qdisc del dev eth2 parent ffff:'
returned (out):
returned (err):
Error: Cannot find specified qdisc on specified device.
cmd 'tc qdisc del dev eth2 root'
returned (out):
returned (err):
Error: Cannot delete qdisc with handle of zero.
cmd 'tc qdisc del dev eth3 parent ffff:'
returned (out):
returned (err):
Error: Cannot find specified qdisc on specified device.
cmd 'tc qdisc del dev eth3 root'
returned (out):
returned (err):
Error: Cannot delete qdisc with handle of zero.
cmd 'tc qdisc del dev cm_up parent ffff:'
returned (out):
returned (err):
Error: Cannot find specified qdisc on specified device.
cmd 'tc qdisc del dev cm_up root'
returned (out):
returned (err):
Error: Cannot delete qdisc with handle of zero.
cmd 'tc qdisc del dev dum1 parent ffff:'
returned (out):
returned (err):
Error: Cannot find specified qdisc on specified device.
cmd 'tc qdisc del dev dum1 root'
returned (out):
returned (err):
Error: Cannot delete qdisc with handle of zero.
cmd 'tc qdisc del dev dum0 parent ffff:'
returned (out):
returned (err):
Error: Cannot find specified qdisc on specified device.
cmd 'tc qdisc del dev dum0 root'
returned (out):
returned (err):
Error: Cannot delete qdisc with handle of zero.
cmd 'tc qdisc del dev gre0 parent ffff:'
returned (out):
returned (err):
Error: Cannot find specified qdisc on specified device.
cmd 'tc qdisc del dev gre0 root'
returned (out):
returned (err):
Error: Cannot delete qdisc with handle of zero.
cmd 'tc qdisc del dev gretap0 parent ffff:'
returned (out):
returned (err):
Error: Cannot find specified qdisc on specified device.
cmd 'tc qdisc del dev gretap0 root'
returned (out):
returned (err):
Error: Cannot delete qdisc with handle of zero.
cmd 'tc qdisc del dev erspan0 parent ffff:'
returned (out):
returned (err):
Error: Cannot find specified qdisc on specified device.
cmd 'tc qdisc del dev erspan0 root'
returned (out):
returned (err):
Error: Cannot delete qdisc with handle of zero.
cmd 'tc qdisc del dev tun0 parent ffff:'
returned (out):
returned (err):
Error: Cannot find specified qdisc on specified device.
cmd 'tc filter replace dev tun0 parent 1: protocol all u32 match ip protocol ospf 0xff action police rate 320000000 burst 15k flowid 1:a'
returned (out):
returned (err):
Illegal "match"
[[qos]] failed
Commit failed
[edit]
vyos@bsp-asbr2-cm#
```
Btw, when I enable a 300mbit/s shaper on tunnel, the actual limit is around 140-150mbit/s, iperf3 said lots of retry.
```
vyos@bsp-asbr2-cm# show qos
interface tun0 {
egress test
}
policy {
shaper test {
bandwidth 330mbit
default {
bandwidth 300mbit
queue-type fair-queue
}
}
}
[edit]
vyos@bsp-asbr2-cm#
```
```
hikari@mynas:~$ iperf3 -c 198.18.0.10 -t 100
Connecting to host 198.18.0.10, port 5201
[ 5] local 10.96.4.101 port 49502 connected to 198.18.0.10 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 19.5 MBytes 164 Mbits/sec 2459 1.51 MBytes
[ 5] 1.00-2.00 sec 16.2 MBytes 136 Mbits/sec 1891 1.16 MBytes
[ 5] 2.00-3.00 sec 17.5 MBytes 147 Mbits/sec 660 1.13 MBytes
[ 5] 3.00-4.00 sec 17.5 MBytes 147 Mbits/sec 1263 1.22 MBytes
[ 5] 4.00-5.00 sec 16.2 MBytes 136 Mbits/sec 1423 1.20 MBytes
[ 5] 5.00-6.00 sec 17.5 MBytes 147 Mbits/sec 1561 1.21 MBytes
[ 5] 6.00-7.00 sec 17.5 MBytes 147 Mbits/sec 1588 1.25 MBytes
[ 5] 7.00-8.00 sec 17.5 MBytes 147 Mbits/sec 1616 1.27 MBytes
[ 5] 8.00-9.00 sec 17.5 MBytes 147 Mbits/sec 2112 1.28 MBytes
[ 5] 9.00-10.00 sec 17.5 MBytes 147 Mbits/sec 2206 1.31 MBytes
[ 5] 10.00-11.00 sec 12.5 MBytes 105 Mbits/sec 1133 1.18 MBytes
^C[ 5] 11.00-11.60 sec 10.0 MBytes 140 Mbits/sec 770 1.28 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-11.60 sec 197 MBytes 142 Mbits/sec 18682 sender
[ 5] 0.00-11.60 sec 0.00 Bytes 0.00 bits/sec receiver
iperf3: interrupt - the client has terminated
hikari@mynas:~$
```
Iperf3 result when I disabled this qos rule:
```
hikari@mynas:~$ iperf3 -c 198.18.0.10 -t 100
Connecting to host 198.18.0.10, port 5201
[ 5] local 10.96.4.101 port 49762 connected to 198.18.0.10 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 35.8 MBytes 300 Mbits/sec 0 4.92 MBytes
[ 5] 1.00-2.00 sec 42.5 MBytes 357 Mbits/sec 1 4.92 MBytes
[ 5] 2.00-3.00 sec 42.5 MBytes 357 Mbits/sec 0 4.92 MBytes
[ 5] 3.00-4.00 sec 38.8 MBytes 325 Mbits/sec 0 4.92 MBytes
[ 5] 4.00-5.00 sec 45.0 MBytes 378 Mbits/sec 0 4.92 MBytes
[ 5] 5.00-6.00 sec 42.5 MBytes 357 Mbits/sec 0 4.92 MBytes
[ 5] 6.00-7.00 sec 42.5 MBytes 357 Mbits/sec 0 4.92 MBytes
[ 5] 7.00-8.00 sec 42.5 MBytes 357 Mbits/sec 0 4.92 MBytes
[ 5] 8.00-9.00 sec 42.5 MBytes 357 Mbits/sec 0 4.92 MBytes
[ 5] 9.00-10.00 sec 42.5 MBytes 357 Mbits/sec 0 4.92 MBytes
[ 5] 10.00-11.00 sec 32.5 MBytes 273 Mbits/sec 0 5.14 MBytes
^C[ 5] 11.00-11.30 sec 12.5 MBytes 353 Mbits/sec 0 5.14 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-11.30 sec 462 MBytes 343 Mbits/sec 1 sender
[ 5] 0.00-11.30 sec 0.00 Bytes 0.00 bits/sec receiver
iperf3: interrupt - the client has terminated
hikari@mynas:~$
```
My hardware platform is Supermicro's X10SDV with Intel Xeon D1521 CPU, hypervisor configuration for this instance is 2vCPU+1GB of RAM.
Because the IPsec performance in normal setting is too poor and I changed kernel cmdline to this:
```
vyos@bsp-asbr2-cm:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/1.4-rolling-202302150317/vmlinuz boot=live rootdelay=5 noautologin net.ifnames=0 biosdevname=0 vyos-union=/boot/1.4-rolling-202302150317 console=ttyS0,115200 console=tty0 mitigations=off
vyos@bsp-asbr2-cm:~$
```
Btw, if I manually set shaper rule by hand, it works correctly:
```
sudo tc qdisc add root dev tun0 handle 1: htb default 2
sudo tc class add dev tun0 parent 1: classid 1:1 htb rate 330mbit
sudo tc class add dev tun0 parent 1:1 classid 1:2 htb rate 300mbit
sudo tc class add dev tun0 parent 1:1 classid 1:3 htb rate 320mbit
```