Page MenuHomeVyOS Platform

Upgrade from 1.3.4 to 1.4 Rolling fails QoS
Open, HighPublicBUG

Description

Today I tried to upgrade my 1.3.4 Vyos home router to 1.4 rolling (vyos-1.4-rolling-202311160128-amd64.iso built by myself)

It mostly worked and the router came online, but missing all the traffic-policy configuration.

Attached are two files, one the "show log" from the 1.4 boot up and another the sanitised configuration from 1.3.4

Details

Difficulty level
Unknown (require assessment)
Version
vyos-1.4-rolling-202311160128
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

tjh changed the visibility from "Public (No Login Required)" to "All Users".

Things to note that I'm not sure if they play a part:

  1. Traffic policy is applied to a PPPoE interface
  2. I have
set system sysctl custom net.core.default_qdisc value 'fq'

in my config

Simple reproducer - doesn't need to be an upgrade, just apply this to 1.4

set qos policy shaper pppoe-in bandwidth '710mbit'
set qos policy shaper pppoe-in class 5 bandwidth '4%'
set qos policy shaper pppoe-in class 5 burst '2mb'
set qos policy shaper pppoe-in class 5 description 'TCP SYN/ACK'
set qos policy shaper pppoe-in class 5 match tiny4 ip tcp ack
set qos policy shaper pppoe-in class 5 match tiny4 ip tcp syn
set qos policy shaper pppoe-in class 5 priority '0'
set qos policy shaper pppoe-in class 5 queue-limit '100'
set qos policy shaper pppoe-in class 5 queue-type 'fq-codel'
set qos policy shaper pppoe-in class 10 bandwidth '1%'
set qos policy shaper pppoe-in class 10 burst '1mb'
set qos policy shaper pppoe-in class 10 description 'DNS Traffic'
set qos policy shaper pppoe-in class 10 match dns ip protocol 'udp'
set qos policy shaper pppoe-in class 10 match dns ip source port '53'
set qos policy shaper pppoe-in class 10 priority '1'
set qos policy shaper pppoe-in class 10 queue-limit '100'
set qos policy shaper pppoe-in class 10 queue-type 'fq-codel'
set qos policy shaper pppoe-in default bandwidth '95%'
set qos policy shaper pppoe-in default burst '15k'
set qos policy shaper pppoe-in default ceiling '100%'
set qos policy shaper pppoe-in default codel-quantum '8000'
set qos policy shaper pppoe-in default priority '7'
set qos policy shaper pppoe-in default queue-type 'fq-codel'
set qos interface eth0 egress pppoe-in


commit

vyos@vyos# commit





2:52
yos@vyos# commit

WARNING: Interface speed cannot be determined (assuming 10 Mbit/s)


WARNING: Interface speed cannot be determined (assuming 10 Mbit/s)


WARNING: Interface speed cannot be determined (assuming 10 Mbit/s)


WARNING: Interface speed cannot be determined (assuming 10 Mbit/s)
                                                                                                                                     [ qos ]                                                                                                                              Error: Cannot find specified qdisc on specified device.
Error: Cannot delete qdisc with handle of zero.
Error: Cannot find specified qdisc on specified device.

WARNING: Interface speed cannot be determined (assuming 10 Mbit/s)


WARNING: Interface speed cannot be determined (assuming 10 Mbit/s)


WARNING: Interface speed cannot be determined (assuming 10 Mbit/s)

WARNING: Interface speed cannot be determined (assuming 10 Mbit/s)
VyOS had an issue completing a command.

We are sorry that you encountered a problem while using VyOS.
There are a few things you can do to help us (and yourself):
- Contact us using the online help desk if you have a subscription:
  https://support.vyos.io/
- Make sure you are running the latest version of VyOS available at:
  https://vyos.net/get/
- Consult the community forum to see how to handle this issue:
  https://forum.vyos.io
- Join us on Slack where our users exchange help and advice:
  https://vyos.slack.com

When reporting problems, please include as much information as possible:
- do not obfuscate any data (feel free to contact us privately if your
  business policy requires it)
- and include all the information presented below

Report time:      2023-11-17 01:42:54
Image version:    VyOS 1.4-rolling-202311160128
Release train:    sagitta

Built by:         [email protected]
Built on:         Thu 16 Nov 2023 01:28 UTC
Build UUID:       8aebc454-28c2-40c8-99d7-6c4058d6bf64
Build commit ID:  9f960353196858

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (i440FX + PIIX, 1996)
Hardware S/N:
Hardware UUID:    e5ceb56e-2461-42cf-b9d8-f94c718b735d

Traceback (most recent call last):
  File "/usr/libexec/vyos/conf_mode/qos.py", line 240, in <module>
    apply(c)                                                                                                                           File "/usr/libexec/vyos/conf_mode/qos.py", line 231, in apply                                                                          tmp.update(shaper_config, direction)
  File "/usr/lib/python3/dist-packages/vyos/qos/trafficshaper.py", line 118, in update
    super().update(config, direction)
  File "/usr/lib/python3/dist-packages/vyos/qos/base.py", line 361, in update
    self._build_base_qdisc(config['default'], default_cls_id)
  File "/usr/lib/python3/dist-packages/vyos/qos/base.py", line 144, in _build_base_qdisc
    self._cmd(default_tc)
  File "/usr/lib/python3/dist-packages/vyos/qos/base.py", line 74, in _cmd
    return cmd(command)
           ^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/vyos/utils/process.py", line 155, in cmd
    raise OSError(code, feedback)
FileNotFoundError: [Errno 2] failed to run command: tc qdisc replace dev eth0 parent 1:b fq_codel quantum 8000 flows 1024 interval 100 interval 100 target 5 noecn
FileNotFoundError: [Errno 2] failed to run command: tc qdisc replace dev eth0 parent 1:b fq_codel quantum 8000 flows 1024 interval 100 interval 100 target 5 noecn
returned:
exit code: 2

noteworthy:
cmd 'tc qdisc replace dev eth0 parent 1:b fq_codel quantum 8000 flows 1024 interval 100 interval 100 target 5 noecn'
returned (out):

returned (err):
Error: Specified class not found.

[[qos]] failed
Commit failed
[edit]
vyos@vyos#

This is on a virtio interface:

tim@ferrari:~$ show interfaces ethernet eth1 physical
Settings for eth1:
        Supported ports: [ ]
        Supported link modes:   Not reported
        Supported pause frame use: No
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  Not reported
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Port: Other
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        Link detected: yes
Ring parameters for eth1:
Pre-set maximums:
RX:             1024
RX Mini:        0
RX Jumbo:       0
TX:             256
Current hardware settings:
RX:             1024
RX Mini:        0
RX Jumbo:       0
TX:             256

driver: virtio_net
version: 1.0.0
firmware-version:
expansion-rom-version:
bus-info: 0000:00:13.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
Viacheslav renamed this task from Upgrade from 1.3.4 to 1.4 Rolling fails. to Upgrade from 1.3.4 to 1.4 Rolling fails QoS.Nov 17 2023, 8:49 AM
Viacheslav added a project: VyOS 1.5 Circinus.

The cause could be due to wrong calculations for bandwidth/ceil class default T5974

Update:
It is completely different issue

set qos policy shaper SHAPE bandwidth '710mbit'

set qos policy shaper SHAPE class 5 bandwidth '2%'
set qos policy shaper SHAPE class 5 match tiny4 ip tcp ack
set qos policy shaper SHAPE class 5 match tiny4 ip tcp syn

set qos policy shaper SHAPE class 10 bandwidth '1%'
set qos policy shaper SHAPE class 10 match dns ip protocol 'udp'
set qos policy shaper SHAPE class 10 match dns ip source port '53'

set qos policy shaper SHAPE default bandwidth '95%'
set qos policy shaper SHAPE default ceiling '100%'

set qos interface eth1 egress SHAPE


commit

Expected TC (as in 1.3):

tc qdisc add dev eth1 root handle 1: htb r2q 444 default b

tc class add dev eth1 parent 1: classid 1:1 htb rate 710000000
tc class add dev eth1 parent 1:1 classid 1:b htb rate 674500000 ceil 710000000 burst 15k
tc qdisc add dev eth1 parent 1:b fq_codel noecn
tc class add dev eth1 parent 1:1 classid 1:5 htb rate 14200000 burst 15k
tc qdisc add dev eth1 parent 1:5 fq_codel noecn

tc filter add dev eth1 parent 1: prio 1 protocol all u32 match ip protocol 6 0xff match u8 0x12 0x12 at 33 flowid 1:5
tc class add dev eth1 parent 1:1 classid 1:a htb rate 7100000 burst 15k
tc qdisc add dev eth1 parent 1:a fq_codel noecn
tc filter add dev eth1 parent 1: prio 2 protocol all u32 match ip protocol 17 0xff match ip sport 53 0xffff flowid 1:a

Get for 1.5

tc qdisc replace dev eth1 root handle 1: htb r2q 443 default 6   <========= ### We get "6" but in 1.3 we get "b"

tc class replace dev eth1 parent 1: classid 1:1 htb rate 710000000
tc class replace dev eth1 parent 1:1 classid 1:6 htb rate 674500000 burst 15k quantum 1514 prio 20 ceil 710000000
tc qdisc replace dev eth1 parent 1:b fq_codel quantum 1514 flows 1024 interval 100 interval 100 target 5 noecn
tc class replace dev eth1 parent 1:1 classid 1:5 htb rate 14200000 burst 15k quantum 1514
tc qdisc replace dev eth1 parent 1:5 sfq

tc filter add dev eth1 parent 1: protocol all prio 1 u32 match u8 0x12 0x12 at 33 flowid 1:5
tc class replace dev eth1 parent 1:1 classid 1:a htb rate 7100000 burst 15k quantum 1514
tc qdisc replace dev eth1 parent 1:a sfq

tc qdisc replace dev eth1 parent 1:6 sfq
tc qdisc replace dev eth1 parent 1:5 fq_codel quantum 1514 flows 1024 interval 100 interval 100 target 5 noecn

tc filter add dev eth1 parent 1: protocol all prio 1 u32 match u8 0x12 0x12 at 33 flowid 1:5 action police rate 20000000 burst 15k flowid 1:5    <=========  ### We do not have this in 1.3
tc qdisc replace dev eth1 parent 1:a fq_codel quantum 1514 flows 1024 interval 100 interval 100 target 5 noecn
tc filter add dev eth1 parent 1: protocol all prio 1 u32 match ip sport 53 0xffff match ip protocol 17 0xff flowid 1:a
tc filter add dev eth1 parent 1: protocol all prio 1 u32 match ip sport 53 0xffff match ip protocol 17 0xff flowid 1:a action police rate 10000000 burst 15k flowid 1:a   <=========  ### We do not have this in 1.3

In 1.3 we do not have rate for the filter and some other commands are different

dmbaturin changed the visibility from "All Users" to "Public (No Login Required)".Thu, Feb 8, 3:15 PM

The latest subtask is T6030
At least for now, I'm not expecting commit fails