Page MenuHomeVyOS Platform

Better VRF support
Open, NormalPublicFEATURE REQUEST

Description

In some core network, public Internet will be treated as a VRF in the router. But VyOS doesn't seem to have really good support on VRFs.

For example, you can't use IPsec/OpenVPN to punch a tunnel through a VRF (T5049, T4031). And NTP/DNS client can't correctly handle VRF.

e.g. If I set a public accessible DNS server, the dns service works only in ip vrf exec internet ping <some domain>, but not in ping <some domain>.

And source NAT is not working when I want to NAT between two VRFs, too (T3655). VyOS can correct translate the outbound packet in the source nat rule, but doesn't handle the reply (inbound) packet correctly.

Details

Version
1.4-rolling-202303280317
Is it a breaking change?
Perfectly compatible

Event Timeline

diodep triaged this task as Wishlist priority.
diodep created this object in space S1 VyOS Public.
Viacheslav changed the subtype of this task from "Task" to "Feature Request".Mar 28 2023, 12:35 PM

Do you know how to tell Linux to use nameservers within a VRF?
What you mean IPSec/OpenVPN "punch a tunnel through a VRF" ? So the underlay should run via a VRF? source-interface binding does not work?

In T5116#147640, @c-po wrote:

Do you know how to tell Linux to use nameservers within a VRF?
What you mean IPSec/OpenVPN "punch a tunnel through a VRF" ? So the underlay should run via a VRF? source-interface binding does not work?

First, I don't know how to tell applications use a nameserver within a VRF, but maybe you can use dnsmasq to relay DNS server from a VRF to global, it solves problem but not pretty.

Secondly, the underlay network should run via a VRF, source-interface binding doesn't work, I have to change systemd charon service to run in a VRF (ip vrf exec ...), but it's too ugly too, and can't run multiple IPsec tunnel over different VRFs.

I think the only solution is to use network namespaces
https://docs.strongswan.org/docs/5.9/howtos/nameSpaces.html

Well, use VRF can implement this, but we have to figure out a proper way to start bunch of strongswan in different VRFs.

Because I tried ip vrf exec internet_vrf charon and it worked out, maybe I should say acceptable. But if I doing this, IPsec VPN's source interface have to in one of VRF, not all VRFs.

Out of the blue it seems like "network namespaces" would solve alot of current VRF compatability issues within VyOS:

https://linuxhint.com/use-linux-network-namespace/

dmbaturin edited projects, added VyOS Rolling; removed Restricted Project.Oct 14 2024, 9:20 AM
dmbaturin changed Is it a breaking change? from Unspecified (possibly destroys the router) to Perfectly compatible.
dmbaturin changed Issue type from Unspecified (please specify) to improvement.

A workaround for NTP and DNS-server is to use containers and allow-host-networks in case the container itself have support for VRF (aka SO_BINDTODEVICE seen through "ss -atulpn" that the process listens to for example 192.0.2.1%PROD:123 instead of just 192.0.2.1:123).

Example using VyOS Stream 2026.03 (having two VRF's named MGMT and PROD):

set container name ntp allow-host-networks
set container name ntp environment ENABLE_NTS value 'false'
set container name ntp environment ENABLE_SYSCLK value 'false'
set container name ntp environment LOG_LEVEL value '0'
set container name ntp environment NOCLIENTLOG value 'false'
set container name ntp environment NTP_DIRECTIVES value 'ratelimit\nrtcsync\nbindacqdevice MGMT\nbinddevice PROD'
set container name ntp environment NTP_SERVERS value '194.58.200.20'
set container name ntp environment TZ value 'Europe/Stockholm'
set container name ntp image 'docker.io/dockurr/chrony:latest'
set container name ntp memory '32'
set container name ntp restart 'on-failure'
set service ntp allow-client address '10.99.0.0/24'
set service ntp server 194.58.200.20 prefer
set service ntp vrf 'MGMT'

The above will use internal NTP service of VyOS to sync to upstream (at VRF MGMT) and set the local system clock.

Also let clients at VRF MGMT (10.99.0.0/24) to query VyOS.

While the container will not affect the local system clock and just like the internal NTP service it will also sync to upstream (at VRF MGMT).

But at the same time let clients at VRF PROD to use it as a NTP-server.

The variable NTP_DIRECTIVES can be used to also set an allow-client list.

For example something like this would limit incoming NTP-queries to only allow NTP-clients from srcip 192.0.2.0/24:

set container name ntp environment NTP_DIRECTIVES value 'ratelimit\nrtcsync\nbindacqdevice MGMT\nbinddevice PROD\nallow all 192.0.2.0/24'

For offline environments a container image can be loaded on a box with Internet access like so:

docker pull docker.io/dockurr/chrony:latest

Then saved:

docker save -o ~/dockurr_chrony_latest_`date +%y%m%d`.tar dockurr/chrony:latest

Compressed:

gzip -9 ~/dockurr_chrony_latest_260722.tar

Uploaded to VyOS (or first copied to USB or such in case the internet box wont be able to SSH to the VyOS box):

scp ~/dockurr_chrony_latest_260722.tar.gz vyos@192.0.2.1:/config

And finally on the VyOS box loaded into podman (note you must use "sudo podman"):

sudo podman load -i /config/dockurr_chrony_latest_260722.tar.gz

Now the image is available through set container name test image <imagename>.

Optionally on the box connected to Internet after the image have been saved the local docker cache can be cleaned:

docker system prune -a --volumes

Or just leave out the -a --volumes part if the box is handling other images aswell.

Ref:

https://chrony-project.org/doc/latest/chrony.conf.html#allow

https://github.com/dockur/chrony/