Exception when trying to set up VXLAN over Wireguard
Closed, ResolvedPublicBUG
Actions

Assigned To

Authored By

	aderouineau
	Jul 26 2022, 2:01 AM

Description

[email protected]# compare
[edit interfaces vxlan vxlan0]
+address 2407:c280:ee:15::100/64
+mtu 1370
+remote 2407:c280:ee:ffff::1
+source-interface wg2
+vni 100

[email protected]# commit

WARNING: RFC7348 recommends VXLAN tunnels preserve a 1500 byte MTU
[ interfaces vxlan vxlan0 ]

WARNING: RFC7348 recommends VXLAN tunnels preserve a 1500 byte MTU
VyOS had an issue completing a command.

We are sorry that you encountered a problem while using VyOS.
There are a few things you can do to help us (and yourself):
- Contact us using the online help desk if you have a subscription:
  https://support.vyos.io/
- Make sure you are running the latest version of VyOS available at:
  https://vyos.net/get/
- Consult the community forum to see how to handle this issue:
  https://forum.vyos.io
- Join us on Slack where our users exchange help and advice:
  https://vyos.slack.com

When reporting problems, please include as much information as possible:
- do not obfuscate any data (feel free to contact us privately if your
  business policy requires it)
- and include all the information presented below

Report time:      2022-07-26 01:51:08
Image version:    VyOS 1.4-rolling-202207250217
Release train:    sagitta

Built by:         [email protected]
Built on:         Mon 25 Jul 2022 02:17 UTC
Build UUID:       2ac7cf68-8f44-4a3f-862d-ecc6a336110e
Build commit ID:  30e90e1f9f8edd

Architecture:     x86_64
Boot via:         installed image
System type:      VMware guest

Hardware vendor:  VMware, Inc.
Hardware model:   VMware Virtual Platform
Hardware S/N:     VMware-56 4d 1c 86 ab 92 b7 fd-81 53 b7 ce 69 1c 64 6f
Hardware UUID:    861c4d56-92ab-fdb7-8153-b7ce691c646f

Traceback (most recent call last):
  File "/usr/libexec/vyos/conf_mode/interfaces-vxlan.py", line 176, in <module>
    apply(c)
  File "/usr/libexec/vyos/conf_mode/interfaces-vxlan.py", line 167, in apply
    v.update(vxlan)
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/interface.py", line 1539, in update
    self.set_mtu(config.get('mtu'))
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/interface.py", line 443, in set_mtu
    return self.set_interface('mtu', mtu)
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/control.py", line 183, in set_interface
    return self._set_command(self.config, name, value)
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/control.py", line 110, in _set_command
    return self._command_set[name].get('format', lambda _: _)(self._cmd(cmd))
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/control.py", line 52, in _cmd
    return cmd(command, self.debug)
  File "/usr/lib/python3/dist-packages/vyos/util.py", line 161, in cmd
    raise OSError(code, feedback)
FileNotFoundError: [Errno 2] failed to run command: ip link set dev vxlan0 mtu 1370
returned:
exit code: 2

noteworthy:
cmd 'nft -c delete element inet vrf_zones ct_iface_map { "vxlan0" }'
returned (out):

returned (err):
Error: Could not process rule: No such file or directory
delete element inet vrf_zones ct_iface_map { vxlan0 }
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cmd 'ip link set dev vxlan0 mtu 1370'
returned (out):

returned (err):
RTNETLINK answers: Invalid argument

[[interfaces vxlan vxlan0]] failed
Commit failed

Details

Difficulty level: Unknown (require assessment)
Version: 1.4-rolling-202207250217
Why the issue appeared?: Will be filled on close
Is it a breaking change?: Unspecified (possibly destroys the router)
Issue type: Bug (incorrect behavior)

Event Timeline

aderouineau created this task.Jul 26 2022, 2:01 AM

aderouineau created this object in space S1 VyOS Public.

Viacheslav changed the subtype of this task from "Task" to "Bug".Jul 26 2022, 9:48 AM

@aderouineau Describe please all steps of how to reproduce it (with commands set xxx)
I don't have any issues with it

set interfaces vxlan vxlan0 group '239.0.0.241'
set interfaces vxlan vxlan0 mtu '1370'
set interfaces vxlan vxlan0 port '4789'
set interfaces vxlan vxlan0 source-interface 'wg0'
set interfaces vxlan vxlan0 vni '123'
set interfaces wireguard wg0 address '100.64.0.1/24'
set interfaces wireguard wg0 peer PEER01 allowed-ips '0.0.0.0/0'
set interfaces wireguard wg0 peer PEER01 public-key 'VVfR5S0yi+QPEJRLr25ZAfzFnwZM40G5WCZ/7ou7h3k='
set interfaces wireguard wg0 private-key 'yGOy08Kv8KUe8rsO6WHeo5jC7YdOAzQK0SJkDFQWlmA='

Commit

[email protected]# commit
[ interfaces vxlan vxlan0 ]

WARNING: RFC7348 recommends VXLAN tunnels preserve a 1500 byte MTU

[edit]
[email protected]# run show int
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
dum0             -                                 u/u  
eth0             192.168.122.14/24                 u/u  WAN
eth1             192.0.2.1/24                      u/u  
                 2001:db8::1/64                         

lo               127.0.0.1/8                       u/u  
                 ::1/128                                
vxlan0           -                                 u/u  
wg0              100.64.0.1/24                     u/u  
[edit]
[email protected]#

Here is my WG config:

set interfaces wireguard wg2 address 'REDACTED_IPV6/64'
set interfaces wireguard wg2 peer mypeer address 'REDACTED_IPV4'
set interfaces wireguard wg2 peer mypeer allowed-ips '::/0'
set interfaces wireguard wg2 peer mypeer persistent-keepalive '60'
set interfaces wireguard wg2 peer mypeer port '51820'
set interfaces wireguard wg2 peer mypeer public-key 'REDACTED'
set interfaces wireguard wg2 private-key 'REDACTED'
set interfaces wireguard wg2 vrf 'test'

pasik added a subscriber: pasik.Jul 27 2022, 9:24 AM

I can reproduce it:

set vrf name test table '1010'
set interfaces vxlan vxlan0 address 2001:db8:2020::1/64
set interfaces vxlan vxlan0 remote '2001:db8:2222::1'
set interfaces vxlan vxlan0 mtu '1370'
set interfaces vxlan vxlan0 port '4789'
set interfaces vxlan vxlan0 source-interface 'wg0'
set interfaces vxlan vxlan0 vni '123'
set interfaces wireguard wg0 address '2001:db8:4411::1/64'
set interfaces wireguard wg0 peer PEER01 allowed-ips '::/0'
set interfaces wireguard wg0 peer PEER01 public-key 'VVfR5S0yi+QPEJRLr25ZAfzFnwZM40G5WCZ/7ou7h3k='
set interfaces wireguard wg0 private-key 'yGOy08Kv8KUe8rsO6WHeo5jC7YdOAzQK0SJkDFQWlmA='
set interfaces wireguard wg0 vrf 'test'

Commit:

Traceback (most recent call last):
  File "/usr/libexec/vyos/conf_mode/interfaces-vxlan.py", line 176, in <module>
    apply(c)
  File "/usr/libexec/vyos/conf_mode/interfaces-vxlan.py", line 167, in apply
    v.update(vxlan)
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/interface.py", line 1539, in update
    self.set_mtu(config.get('mtu'))
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/interface.py", line 443, in set_mtu
    return self.set_interface('mtu', mtu)
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/control.py", line 183, in set_interface
    return self._set_command(self.config, name, value)
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/control.py", line 110, in _set_command
    return self._command_set[name].get('format', lambda _: _)(self._cmd(cmd))
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/control.py", line 52, in _cmd
    return cmd(command, self.debug)
  File "/usr/lib/python3/dist-packages/vyos/util.py", line 161, in cmd
    raise OSError(code, feedback)
FileNotFoundError: [Errno 2] failed to run command: ip link set dev vxlan0 mtu 1370
returned: 
exit code: 2

noteworthy:
cmd 'nft -c delete element inet vrf_zones ct_iface_map { "vxlan0" }'
returned (out):

returned (err):
Error: Could not process rule: No such file or directory
delete element inet vrf_zones ct_iface_map { vxlan0 }
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cmd 'ip link set dev vxlan0 mtu 1370'
returned (out):

It doesn't like MTU 1370 for vxlanX + IPv6 peer

[email protected]# sudo ip link set dev vxlan0 mtu 1370
RTNETLINK answers: Invalid argument
[edit]
[email protected]#

Try to change MTU to 1350 or not set this parameter, is should be auto-calculated

set interfaces vxlan vxlan0 mtu '1350'

Require more checks https://github.com/vyos/vyos-1x/blob/8bbaafd554d003078607121e2de662c9c8edee40/src/conf_mode/interfaces-vxlan.py#L111-L124

As it checks if source_address in the config, but it seems not used in this case for vxlanX
As result it minus MTU - 50 bytes, expected MTU - 70 bytes

As an option, we can check the list of remote addresses, and if one of them is IPv6 add +20 to overhead

Viacheslav changed the task status from Open to In progress.Jul 27 2022, 11:37 AM

Viacheslav claimed this task.

Viacheslav added a project: VyOS 1.4 Sagitta.

PR https://github.com/vyos/vyos-1x/pull/1440

set vrf name test table '1010'
set interfaces vxlan vxlan0 address 2001:db8:2020::1/64
set interfaces vxlan vxlan0 remote '2001:db8:2222::1'
set interfaces vxlan vxlan0 mtu '1370'
set interfaces vxlan vxlan0 port '4789'
set interfaces vxlan vxlan0 source-interface 'wg0'
set interfaces vxlan vxlan0 vni '123'
set interfaces wireguard wg0 address '2001:db8:4411::1/64'
set interfaces wireguard wg0 peer PEER01 allowed-ips '::/0'
set interfaces wireguard wg0 peer PEER01 public-key 'VVfR5S0yi+QPEJRLr25ZAfzFnwZM40G5WCZ/7ou7h3k='
set interfaces wireguard wg0 private-key 'yGOy08Kv8KUe8rsO6WHeo5jC7YdOAzQK0SJkDFQWlmA='
set interfaces wireguard wg0 vrf 'test'

Commit:

[email protected]# commit
[ interfaces vxlan vxlan0 ]

WARNING: RFC7348 recommends VXLAN tunnels preserve a 1500 byte MTU
Underlaying device MTU is to small (1420 bytes) for VXLAN overhead (70
bytes!)

[[interfaces vxlan vxlan0]] failed
Commit failed
[edit]
[email protected]#

The reason I set an MTU is because I get the following error when unset:

WARNING: RFC7348 recommends VXLAN tunnels preserve a 1500 byte MTU

Underlaying device MTU is to small (1420 bytes) for VXLAN overhead (50
bytes!)

Setting MTU to 1350 does apply without issues. I guess there are issues with overhead calculation when checking the MTU?

As a side note, "underlying" and "too small" are spelled incorrectly in the error message.

Viacheslav changed the task status from In progress to Needs testing.Jul 30 2022, 9:37 AM

Viacheslav closed this task as Resolved.Aug 18 2022, 7:38 PM

Viacheslav moved this task from Need Triage to Finished on the VyOS 1.4 Sagitta board.

Exception when trying to set up VXLAN over WireguardClosed, ResolvedPublicBUGActions

Description

Details

Event Timeline

Exception when trying to set up VXLAN over Wireguard
Closed, ResolvedPublicBUG
Actions