Page MenuHomeVyOS Platform

QoS shaper incorrect rate limit the traffic
Closed, ResolvedPublicBUG

Description

There is a simple configuration:

set qos interface eth0 egress 'test'
set qos policy shaper test bandwidth '330mbit'
set qos policy shaper test default bandwidth '300mbit'
set qos policy shaper test default queue-type 'fair-queue'

The debug of commit:

vyos@r14# commit
[ qos ]
DEBUG/QoS: tc qdisc replace dev eth0 root handle 1: htb r2q 206 default 1
DEBUG/QoS: tc class replace dev eth0 parent 1: classid 1:1 htb rate 330000000
DEBUG/QoS: tc class replace dev eth0 parent 1:1 classid 1:1 htb rate 300000000 burst 15k quantum 1514 prio 20
DEBUG/QoS: tc qdisc replace dev eth0 parent 1:1 sfq
{'bandwidth': '330mbit',
 'default': {'bandwidth': '300mbit',
             'burst': '15k',
             'codel_quantum': '1514',
             'flows': '1024',
             'interval': '100',
             'priority': '20',
             'queue_type': 'fair-queue',
             'target': '5'}}
DEBUG/QoS: tc filter replace dev eth0 parent 1: prio 255 protocol all basic action police rate 300000000 burst 15k

Expected traffic 300mbit
Get

vyos@r14# sudo iperf3 -c 192.168.122.11
Connecting to host 192.168.122.11, port 5201
[  5] local 192.168.122.14 port 5992 connected to 192.168.122.11 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  1.75 MBytes  14.7 Mbits/sec  996   74.9 KBytes       
[  5]   1.00-2.00   sec   573 KBytes  4.69 Mbits/sec  570   31.1 KBytes       
[  5]   2.00-3.00   sec   954 KBytes  7.82 Mbits/sec  565   22.6 KBytes       
[  5]   3.00-4.00   sec   700 KBytes  5.73 Mbits/sec  609    117 KBytes       
[  5]   4.00-5.00   sec   382 KBytes  3.13 Mbits/sec  403   26.9 KBytes       
[  5]   5.00-6.00   sec   382 KBytes  3.13 Mbits/sec  580   45.2 KBytes       
[  5]   6.00-7.00   sec  1.30 MBytes  11.0 Mbits/sec  974   35.4 KBytes       
[  5]   7.00-8.00   sec   954 KBytes  7.82 Mbits/sec  495   22.6 KBytes       
[  5]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec  487   36.8 KBytes       
[  5]   9.00-10.00  sec   954 KBytes  7.82 Mbits/sec  606   28.3 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  7.84 MBytes  6.57 Mbits/sec  6285             sender
[  5]   0.00-10.00  sec  6.60 MBytes  5.54 Mbits/sec                  receiver

iperf Done.

Show filter

vyos@r14# sudo tc -s -d filter show dev eth0
filter parent 1: protocol all pref 255 basic chain 0 
filter parent 1: protocol all pref 255 basic chain 0 handle 0x1 
	action order 1:  police 0x1 rate 300Mbit burst 15300b mtu 2Kb action reclassify overhead 0b linklayer ethernet 
	ref 1 bind 1  installed 138 sec used 0 sec firstused 138 sec
	Action statistics:
	Sent 171713394 bytes 113616 pkt (dropped 0, overlimits 8468 requeues 0) 
	backlog 0b 0p requeues 0

The main issue with this command tc filter

tc filter replace dev eth0 parent 1: prio 255 protocol all basic action police rate 300000000 burst 15k

If we exclude it:

tc qdisc del dev eth0 root
tc qdisc replace dev eth0 root handle 1: htb r2q 206 default 1
tc class replace dev eth0 parent 1: classid 1:1 htb rate 330000000
tc class replace dev eth0 parent 1:1 classid 1:1 htb rate 300000000 burst 15k quantum 1514 prio 20
tc qdisc replace dev eth0 parent 1:1 sfq

It shapes correct:

vyos@r14# sudo iperf3 -c 192.168.122.11
Connecting to host 192.168.122.11, port 5201
[  5] local 192.168.122.14 port 15616 connected to 192.168.122.11 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  35.8 MBytes   300 Mbits/sec    0    184 KBytes       
[  5]   1.00-2.00   sec  35.0 MBytes   293 Mbits/sec    0    215 KBytes       
[  5]   2.00-3.00   sec  33.9 MBytes   285 Mbits/sec    0    181 KBytes       
[  5]   3.00-4.00   sec  33.9 MBytes   285 Mbits/sec    0    181 KBytes       
[  5]   4.00-5.00   sec  34.8 MBytes   292 Mbits/sec    0    221 KBytes       
[  5]   5.00-6.00   sec  34.1 MBytes   286 Mbits/sec    0    212 KBytes       
[  5]   6.00-7.00   sec  33.9 MBytes   285 Mbits/sec    0    221 KBytes       
[  5]   7.00-8.00   sec  33.9 MBytes   285 Mbits/sec    0    229 KBytes       
[  5]   8.00-9.00   sec  34.7 MBytes   291 Mbits/sec    0    215 KBytes       
[  5]   9.00-10.00  sec  33.9 MBytes   285 Mbits/sec    0   5.66 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   344 MBytes   289 Mbits/sec    0             sender
[  5]   0.00-10.00  sec   342 MBytes   287 Mbits/sec                  receiver

iperf Done.

Details

Difficulty level
Unknown (require assessment)
Version
VyOS 1.4-rolling-202306130317
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Unspecified (please specify)

Event Timeline

The same config in 1.3 doesn't have tc filter in 1.3

set interfaces ethernet eth0 traffic-policy out 'test'
set traffic-policy shaper test bandwidth '330mbit'
set traffic-policy shaper test default bandwidth '300mbit'
set traffic-policy shaper test default queue-type 'fair-queue'

vyos@r11# commit
[ traffic-policy shaper test ]
qdisc add dev eth0 root handle 1: htb r2q 207 default 2
class add dev eth0 parent 1: classid 1:1 htb rate 330000000
class add dev eth0 parent 1:1 classid 1:2 htb rate 300000000 burst 15k
qdisc add dev eth0 parent 1:2 sfq
Viacheslav changed the task status from Open to In progress.Jun 16 2023, 11:16 AM
Viacheslav claimed this task.

PR https://github.com/vyos/vyos-1x/pull/2044
Config

set qos interface eth0 egress 'test'
set qos policy shaper test bandwidth '330mbit'
set qos policy shaper test class 23 bandwidth '250mbit'
set qos policy shaper test class 23 match 10 ip destination address '192.168.122.11'
set qos policy shaper test default bandwidth '300mbit'
set qos policy shaper test default queue-type 'fair-queue'

after fixing the expected speed 300 Mbit (default speed)

vyos@r14# sudo iperf3 -c 192.168.122.1 -t 5
Connecting to host 192.168.122.1, port 5201
[  5] local 192.168.122.14 port 60154 connected to 192.168.122.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  36.6 MBytes   307 Mbits/sec    0    238 KBytes       
[  5]   1.00-2.00   sec  33.9 MBytes   285 Mbits/sec    0    218 KBytes       
[  5]   2.00-3.00   sec  33.9 MBytes   285 Mbits/sec    0    223 KBytes       
[  5]   3.00-4.00   sec  34.8 MBytes   292 Mbits/sec    0    215 KBytes       
[  5]   4.00-5.00   sec  33.9 MBytes   285 Mbits/sec    0    221 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec   173 MBytes   291 Mbits/sec    0             sender
[  5]   0.00-5.04   sec   171 MBytes   285 Mbits/sec                  receiver

iperf Done.

after fixing the expected speed 250 Mbit (class 23)

vyos@r14# sudo iperf3 -c 192.168.122.11 -t 5
Connecting to host 192.168.122.11, port 5201
[  5] local 192.168.122.14 port 29104 connected to 192.168.122.11 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  30.2 MBytes   254 Mbits/sec  3229    156 KBytes       
[  5]   1.00-2.00   sec  28.7 MBytes   241 Mbits/sec  3254    288 KBytes       
[  5]   2.00-3.00   sec  27.7 MBytes   232 Mbits/sec  3614    127 KBytes       
[  5]   3.00-4.00   sec  28.6 MBytes   240 Mbits/sec  3597    158 KBytes       
[  5]   4.00-5.00   sec  27.8 MBytes   233 Mbits/sec  3753    158 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec   143 MBytes   240 Mbits/sec  17447             sender
[  5]   0.00-5.04   sec   140 MBytes   234 Mbits/sec                  receiver

iperf Done.
[edit]
vyos@r14#
Viacheslav changed the task status from In progress to Needs testing.Jun 17 2023, 1:04 AM
Viacheslav moved this task from Need Triage to Finished on the VyOS 1.4 Sagitta board.

It fixes for shaper but breaks the policer
Policer uses limits in tc filter

VyOS config for policer

set qos interface eth0 ingress '1G-in'
set qos policy limiter 1G-in default bandwidth '1gbit'
set qos policy limiter 1G-in default burst '125000000b'

commit:

vyos@r14# commit
[ qos ]

DEBUG/QoS: tc qdisc add dev eth0 handle ffff: ingress

{'default': {'bandwidth': '1gbit',
             'burst': '125000000b',
             'exceed': 'drop',
             'not_exceed': 'ok'}}

Expected:

tc qdisc add dev eth0 handle ffff: ingress
tc filter add dev eth0 parent ffff: prio 255 protocol all basic  action police rate 1000000000 conform-exceed drop burst 125000000b flowid ffff:1

PR https://github.com/vyos/vyos-1x/pull/2065

set qos interface eth0 ingress '300m-in'
set qos policy limiter 300m-in default bandwidth '300mbit'
set qos policy limiter 300m-in default burst '125000000b'
commit

commit

vyos@r14# commit
[ qos ]
DEBUG/QoS: tc qdisc add dev eth0 handle ffff: ingress
{'default': {'bandwidth': '300mbit',
             'burst': '125000000b',
             'exceed': 'drop',
             'not_exceed': 'ok'}}
DEBUG/QoS: tc filter replace dev eth0 parent ffff: prio 255 protocol all basic action police conform-exceed drop/ok rate 300000000 burst 125000000b

[edit]
vyos@r14#