Configuration fails to load on boot if offloading has VLAN interfaces defined
Closed, ResolvedPublicBUG
Actions

Assigned To

Authored By

	GurliGebis
	Jan 12 2024, 2:00 PM

Description

It seems like when offloading is used, and VLAN interfaces is added to the flowtable, the system fails to load the entire firewall section during boot.

By changing this in my config:

set firewall flowtable OFFLOAD interface 'eth0'
set firewall flowtable OFFLOAD interface 'eth1'
set firewall flowtable OFFLOAD interface 'eth1.10'
set firewall flowtable OFFLOAD interface 'eth1.20'

To:

set firewall flowtable OFFLOAD interface 'eth0'
set firewall flowtable OFFLOAD interface 'eth1'

Got it working, so either it should be fixed, so VLAN interfaces work, of the CLI should prevent adding VLAN interfaces to flowtables in the first place.
With the VLAN interfaces defined, the following error is thrown at startup: Starting VyOS router: migrate configure failed!

Details

Difficulty level: Unknown (require assessment)
Version: VyOS 1.4-rolling-202401121029
Why the issue appeared?: Will be filled on close
Is it a breaking change?: Unspecified (possibly destroys the router)
Issue type: Bug (incorrect behavior)

Event Timeline

GurliGebis created this task.Jan 12 2024, 2:00 PM

Viacheslav triaged this task as High priority.Jan 12 2024, 2:03 PM

pasik added a subscriber: pasik.Jan 12 2024, 2:05 PM

Just thinking aloud - could it be the period that is causing issues with the loading?

During a boot that raises the migration error, can you alter the grub boot command to add vyos-config-debug? See https://docs.vyos.io/en/latest/contributing/debugging.html#kernel

You should get some output that will help with troubleshooting.

Booting now gives this:

Here is the commands used on a clean install to get into this state:

set firewall flowtable OFFLOAD interface 'eth0'
set firewall flowtable OFFLOAD interface 'eth1'
set firewall flowtable OFFLOAD interface 'eth1.10'
set firewall flowtable OFFLOAD interface 'eth1.20'
set interfaces ethernet eth0 address '172.16.15.151/24'
set interfaces ethernet eth1 address '10.16.15.1/24'
set interfaces ethernet eth1 vif 10 address '10.16.14.1/24'
set interfaces ethernet eth1 vif 20 address '10.16.20.1/24'

Here is the content of the /tmp/boot-config-trace file:

Traceback (most recent call last):
  File "/usr/libexec/vyos/vyos-boot-config-loader.py", line 144, in <module>
    commit_out = session.commit()
                 ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/vyos/configsession.py", line 187, in commit
    out = self.__run_command([COMMIT])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/vyos/configsession.py", line 143, in __run_command
    raise ConfigSessionError(output)
vyos.configsession.ConfigSessionError:  Processing the Priority Queue
  Entering the _commit_check_cfg_node
   Executing the "system host-name vyos" ...
   Elapsed 0.001 sec:
  Elapsed 0.001 sec: _commit_check_cfg_node
  Entering the _commit_exec_cfg_node
   Executing the "system host-name vyos" ...
   Elapsed 0.163 sec:
  Elapsed 0.163 sec: _commit_exec_cfg_node
  Entering the _commit_check_cfg_node
   Executing the "system console device ttyS0" ...
   Elapsed 0.001 sec:
   Executing the "system console device ttyS0 speed 115200" ...
   Elapsed 0.001 sec:
  Elapsed 0.003 sec: _commit_check_cfg_node
  Entering the _commit_exec_cfg_node
   Executing the "system console" ...
   Elapsed 0.233 sec:
  Elapsed 0.233 sec: _commit_exec_cfg_node
  Entering the _commit_check_cfg_node
   Executing the "firewall flowtable OFFLOAD" ...
   Elapsed 0.001 sec:
  Elapsed 0.001 sec: _commit_check_cfg_node
  Entering the _commit_exec_cfg_node
   Executing the "firewall" ...
   Elapsed 0.009 sec:
  Elapsed 0.009 sec: _commit_exec_cfg_node
[[firewall]] failed
  Entering the _commit_check_cfg_node
  Elapsed 0.000 sec: _commit_check_cfg_node
  Entering the _commit_exec_cfg_node
   Executing the "system conntrack" ...
   Elapsed 0.141 sec:
  Elapsed 0.141 sec: _commit_exec_cfg_node
  Entering the _commit_check_cfg_node
   Executing the "interfaces loopback lo" ...
   Elapsed 0.001 sec:
  Elapsed 0.001 sec: _commit_check_cfg_node
  Entering the _commit_exec_cfg_node
   Executing the "interfaces loopback lo" ...
   Elapsed 0.090 sec:
  Elapsed 0.090 sec: _commit_exec_cfg_node
  Entering the _commit_check_cfg_node
   Executing the "interfaces ethernet eth0" ...
   Elapsed 0.001 sec:
   Executing the "interfaces ethernet eth0 address 172.16.15.151/24" ...
   Elapsed 0.007 sec:
   Executing the "interfaces ethernet eth0 hw-id 00:15:5d:0f:29:32" ...
   Elapsed 0.004 sec:
  Elapsed 0.014 sec: _commit_check_cfg_node
  Entering the _commit_exec_cfg_node
   Executing the "interfaces ethernet eth0" ...
   Elapsed 0.138 sec:
  Elapsed 0.138 sec: _commit_exec_cfg_node
  Entering the _commit_check_cfg_node
   Executing the "interfaces ethernet eth1" ...
   Elapsed 0.001 sec:
   Executing the "interfaces ethernet eth1 address 10.16.15.1/24" ...
   Elapsed 0.004 sec:
   Executing the "interfaces ethernet eth1 hw-id 00:15:5d:0f:29:33" ...
   Elapsed 0.004 sec:
   Executing the "interfaces ethernet eth1 vif 10" ...
   Elapsed 0.032 sec:
   Executing the "interfaces ethernet eth1 vif 10 address 10.16.14.1/24" ...
   Elapsed 0.004 sec:
   Executing the "interfaces ethernet eth1 vif 20" ...
   Elapsed 0.003 sec:
   Executing the "interfaces ethernet eth1 vif 20 address 10.16.20.1/24" ...
   Elapsed 0.004 sec:
  Elapsed 0.055 sec: _commit_check_cfg_node
  Entering the _commit_exec_cfg_node
   Executing the "interfaces ethernet eth1" ...
   Elapsed 0.206 sec:
  Elapsed 0.206 sec: _commit_exec_cfg_node
  Entering the _commit_check_cfg_node
   Executing the "system syslog global facility all" ...
   Elapsed 0.002 sec:
   Executing the "system syslog global facility all level info" ...
   Elapsed 0.001 sec:
   Executing the "system syslog global facility local7" ...
   Elapsed 0.001 sec:
   Executing the "system syslog global facility local7 level debug" ...
   Elapsed 0.001 sec:
  Elapsed 0.007 sec: _commit_check_cfg_node
  Entering the _commit_exec_cfg_node
   Executing the "system syslog" ...
   Elapsed 0.501 sec:
  Elapsed 0.501 sec: _commit_exec_cfg_node
  Entering the _commit_check_cfg_node
   Executing the "system login user vyos" ...
   Elapsed 0.002 sec:
   Executing the "system login user vyos authentication encrypted-password $6$rounds=656000$dL9LyenIWfGWLWOh$8t6wAQ5LV7u/lMCdwD4XyB7HZr4FCe7Xb3TodhHb4wqrjAytIWz10mSt6nJCahZLKRjNCOjkiBy5Eu9WNG2b8/" ...
   Elapsed 0.001 sec:
  Elapsed 0.003 sec: _commit_check_cfg_node
  Entering the _commit_exec_cfg_node
   Executing the "system login" ...
   Elapsed 1.011 sec:
  Elapsed 1.011 sec: _commit_exec_cfg_node
  Entering the _commit_check_cfg_node
   Executing the "system config-management commit-revisions 100" ...
   Elapsed 0.004 sec:
  Elapsed 0.004 sec: _commit_check_cfg_node
  Entering the _commit_exec_cfg_node
   Executing the "system config-management" ...
   Elapsed 0.010 sec:
  Elapsed 0.010 sec: _commit_exec_cfg_node
  Entering the _commit_check_cfg_node
   Executing the "service ntp allow-client address 0.0.0.0/0" ...
   Elapsed 0.004 sec:
   Executing the "service ntp allow-client address ::/0" ...
   Elapsed 0.003 sec:
   Executing the "service ntp server time1.vyos.net" ...
   Elapsed 0.007 sec:
   Executing the "service ntp server time2.vyos.net" ...
   Elapsed 0.004 sec:
   Executing the "service ntp server time3.vyos.net" ...
   Elapsed 0.004 sec:
  Elapsed 0.025 sec: _commit_check_cfg_node
  Entering the _commit_exec_cfg_node
   Executing the "service ntp" ...
   Elapsed 0.417 sec:
  Elapsed 0.417 sec: _commit_exec_cfg_node
  Entering the _commit_check_cfg_node
   Executing the "service ssh port 22" ...
   Elapsed 0.003 sec:
  Elapsed 0.003 sec: _commit_check_cfg_node
  Entering the _commit_exec_cfg_node
   Executing the "service ssh" ...
   Elapsed 0.344 sec:
  Elapsed 0.344 sec: _commit_exec_cfg_node
 Elapsed 3.392 sec: Commit execute priority tree
Commit failed

The issue is only on boot, if after booting you run the load command, it loads fine and commit works without any issues.

I just did a test - without the VLAN interfaces added, the VLAN traffic is still offloaded.
So the CLI should be updated to prevent VLAN's from being added (since it doesn't make any sense to add them, since they work when the parent interface is added)

My only comment here would be to hesitate when putting in restrictions; as Doug Gwyn once said:

Unix was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.

I believe there could be a use case here, for example, based on the Linux kernel's nf_flowtable documentation, it looks like only a single layer of VLAN tags would be included in the hashtable:

The flowtable uses a resizable hashtable. Lookups are based on the following n-tuple selectors: layer 2 protocol encapsulation (VLAN and PPPoE), layer 3 source and destination, layer 4 source and destination ports and the input interface (useful in case there are several conntrack zones in place).

https://docs.kernel.org/networking/nf_flowtable.html

I assume this means that for something like 802.1ad (Q-in-Q) it may be necessary to put the VLAN interface in a flowtable, but this is speculation so I could be wrong.

syncer assigned this task to dmbaturin.Feb 2 2024, 1:30 PM

PR https://github.com/vyos/vyos-1x/pull/2999

Viacheslav closed this task as Resolved.Feb 13 2024, 4:39 PM

Viacheslav moved this task from Need Triage to Finished on the VyOS 1.4 Sagitta board.

Viacheslav added a project: VyOS 1.5 Circinus.

Viacheslav moved this task from Need Triage to Finished on the VyOS 1.5 Circinus board.

	F4155114: billede.png
	Jan 12 2024, 3:08 PM

Configuration fails to load on boot if offloading has VLAN interfaces definedClosed, ResolvedPublicBUGActions

Description

Details

Event Timeline

Configuration fails to load on boot if offloading has VLAN interfaces defined
Closed, ResolvedPublicBUG
Actions