Static routes not installed into kernel nor frr
Closed, ResolvedPublicBUG
Actions

Assigned To

Authored By

	ernstjo
	Jul 22 2021, 10:11 AM

Description

Hi,

i have an issue with static routes, sometimes static routes are not installed into the kernel nor an static route config item in frr:

set protocols static route6 xxx:xxx:b046:1::/64 next-hop xxx:xxx:b004:3::2 
set protocols static route6 xxx:xxx:b046:c000::/56 next-hop xxx:xxx:b004:3::2

run show inter
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
tun0       xxx:xxx:b004:3::1/64            u/u  Cus: xxx [1G]

Routes are missing because they were not configured from vyos into frr:

vtysh -c 'show running-config' | grep 'ipv6 route'
ipv6 route ::/0 xxx:xxx:4b:62::2
ipv6 route ::/0 xxx:xxx:121::254 254

run show ipv6 route static 
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

S>  ::/0 [1/0] via xxx:xxx:4b:62::2 (recursive), weight 1, 1d12h14m
  *              via fe80::da84:66ff:feea:7015, eth1, weight 1, 1d12h14m
S   ::/0 [254/0] via xxx:xxxx:121::254, dum1, weight 1, 02w4d09h

If i add that routes manually via vtysh it just works fine. So, vyos does not configure frr probably in case of static routes.

Details

Difficulty level: Unknown (require assessment)
Version: 1.4-rolling-202106190417
Why the issue appeared?: Will be filled on close
Is it a breaking change?: Unspecified (possibly destroys the router)

Event Timeline

ernstjo created this task.Jul 22 2021, 10:11 AM

pasik added a subscriber: pasik.Jul 22 2021, 10:59 AM

@ernstjo Can you share an example of your tunnel interface?
I don't understand yet how to reproduce it.
If you delete routes and add again, do you get the same result?

Try to touch frr debug to collect more information
https://docs.vyos.io/en/latest/debugging.html#frr

It's a basic tunnel setup and also reproducible with routes which do not point to a tunnel interface and happens also with physical interfaces.

Tunnel config

set interfaces tunnel tun0 address 'xxx:xxx:b004:3::1/64'
set interfaces tunnel tun0 description 'Cus: xxx [1G]'
set interfaces tunnel tun0 encapsulation 'sit'
set interfaces tunnel tun0 parameters ip ttl '255'
set interfaces tunnel tun0 parameters ipv6 hoplimit '255'
set interfaces tunnel tun0 remote 'xxxx'
set interfaces tunnel tun0 source-address 'xxx'

I can add routes via frr manually and without any issue:

ipv6 route xxx:xxx:b046:c000::/56 xxx:xxx:b004:3::2

So, vyos having issues to add the static route entries into frr and not having anything to do with tunnels or interfaces.

Try to touch frr debug to collect more information
https://docs.vyos.io/en/latest/debugging.html#frr

How to get the debug logs? I already enabled debug mode.

Tested configuration:

set interfaces ethernet eth1 address '192.0.2.1/24'
set interfaces ethernet eth1 address 'dead:beef:b004:3::1/64'
set interfaces tunnel tun0 address '2a0f:5707:b004:3::1/64'
set interfaces tunnel tun0 encapsulation 'sit'
set interfaces tunnel tun0 parameters ip ttl '255'
set interfaces tunnel tun0 parameters ipv6 hoplimit '255'
set interfaces tunnel tun0 remote '192.0.2.2'
set interfaces tunnel tun0 source-address '192.0.2.1'
set protocols static route6 cafe:e1f:b046:1::/64 next-hop 2a0f:5707:b004:3::2
set protocols static route6 cafe:e1f:b046:c000::/56 next-hop 2a0f:5707:b004:3::2

Routes and pings:

vyos@r1-roll# run show ipv6 route static | match "tun"
S>* cafe:e1f:b046:1::/64 [1/0] via 2a0f:5707:b004:3::2, tun0, weight 1, 00:01:28
S>* cafe:e1f:b046:c000::/56 [1/0] via 2a0f:5707:b004:3::2, tun0, weight 1, 00:01:28
[edit]
vyos@r1-roll# 

vyos@r1-roll# run ping cafe:e1f:b046:1::1
PING cafe:e1f:b046:1::1(cafe:e1f:b046:1::1) 56 data bytes
64 bytes from cafe:e1f:b046:1::1: icmp_seq=1 ttl=64 time=0.265 ms
64 bytes from cafe:e1f:b046:1::1: icmp_seq=2 ttl=64 time=0.893 ms
^C
--- cafe:e1f:b046:1::1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1049ms
rtt min/avg/max/mdev = 0.265/0.579/0.893/0.314 ms
[edit]
vyos@r1-roll#

Version:

vyos@r1-roll# run show ver

Version:          VyOS 1.4-rolling-202107201147
Release Train:    sagitta

Built by:         [email protected]
Built on:         Tue 20 Jul 2021 19:10 UTC

In T3694#98436, @ernstjo wrote:

How to get the debug logs? I already enabled debug mode.

sudo systemctl stop vyos-configd
sudo touch /tmp/vyos.frr.debug

Just that shitty. When i enable debug mode route install works fine when i disable debugging i won't work.
I have that situation on multiple routers.

run show ipv6 route static 
[edit]
set protocols static route6 xx:xxx:b046::/48 next-hop xxx:xxx:b000::1
[edit]
vyos# commit
[edit]
vyos# run show ipv6 route static 
[edit]

Route not installed..

Enable debug mode:

touch /tmp/vyos.frr.debug
[edit]
# sudo systemctl stop vyos-configd
[edit]
# set protocols static route6 2a0f:5707:b046::/48 next-hop 2a0f:5707:b000::1
[edit]
# commit

Debug Log:

add_before:   add           16 !
add_before:   add           17 ipv6 route xxx:xxx:b040:1::/64 xxx:5707:b004:1::2    
add_before:   add           18 ipv6 route xxx:xxx:b040:2::/64 xxx:5707:b004:2::2    
add_before:   add           19 ipv6 route xxx:xxx:b046::/48 xxx:5707:b000::1    
add_before:   add           20 !
add_before:   add           21 !
add_before:   add           22 
commit_configuration:  Commiting configuration
commit_configuration: new_config   0 !
commit_configuration: new_config   1 frr version 7.5.1-20201222-185-gb3f4ff1d9
commit_configuration: new_config   2 frr defaults traditional
commit_configuration: new_config   3 hostname debian
commit_configuration: new_config   4 log syslog
commit_configuration: new_config   5 log facility local7
commit_configuration: new_config   6 hostname xxxx
commit_configuration: new_config   7 service integrated-vtysh-config
commit_configuration: new_config   8 !
commit_configuration: new_config   9 
commit_configuration: new_config  10 
commit_configuration: new_config  11 !
commit_configuration: new_config  12 vrf mgmt
commit_configuration: new_config  13  ip route 0.0.0.0/0 xxx.xxx.61.254
commit_configuration: new_config  14  exit-vrf
commit_configuration: new_config  15 !
commit_configuration: new_config  16 !
commit_configuration: new_config  17 ipv6 route xxx:xxx:b040:1::/64 xxx:xxx:b004:1::2    
commit_configuration: new_config  18 ipv6 route xxx:xxx:b040:2::/64 xxx:xxx:b004:2::2    
commit_configuration: new_config  19 ipv6 route xxx:xxx:b046::/48 xxx:xxx:b000::1    
commit_configuration: new_config  20 !
commit_configuration: new_config  21 !
commit_configuration: new_config  22 
commit_configuration: new_config  23 line vty
commit_configuration: new_config  24 !
commit_configuration: new_config  25 end

run show ipv6 route static 
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

S>* xxx:xxx:b040:1::/64 [1/0] via xxx:xxx:b004:1::2, tun01523, weight 1, 00:00:51
S>* xxx:xxx:b040:2::/64 [1/0] via xxx:xxx:b004:2::2, tun3001523, weight 1, 00:00:51
S>  xxx:xxx:b046::/48 [1/0] via xxx:xxx:b000::1 (recursive), weight 1, 00:00:06
  *                             via fe80::9459:a8ff:fe1c:c073, l2tpeth1, weight 1, 00:00:06
[edit]

All static routes installed,..but only in debug mode.

Interesting. But I never saw it.
Can you check the latest rolling? I have no other ideas yet.

It happens on different versions and not fixed in the latest version.

I can't reproduce it in any version.
Let's check what is different, and why only in your case do you have a problem, but other users never have this issue.

Do you use standard user (vyos) or not?
Which hypervisor?
How do you deploy the instances?
Do you use radius authentication or something like this?
Anything else?

There must be something in common in the configuration that causes your instances to behave like this.

Do you use standard user (vyos) or not?
yes, standard vyos user

Which hypervisor?
VMware, Proxmox and Baremetal

How do you deploy the instances?
Classical via rolling isos

Do you use radius authentication or something like this?
No

Anything else?
I'm using vrf feature for mgmt access / separation. Mgmt VRF has static default route ipv4/ipv6.

show vrf

VRF name          state     mac address        flags                     interfaces
--------          -----     -----------        -----                     ----------

mgmt              up        66:a8:c2:0f:1d:7c  noarp,master,up,lower_up  eth0

# run show ipv6 route vrf mgmt
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

VRF mgmt:
S>* ::/0 [1/0] via xx:xx:1:30::1, eth0, weight 1, 04w0d01h
K * ::/0 [255/8192] unreachable (ICMP unreachable), 04w0d01h
C>* xx:xx:1:30::/64 is directly connected, eth0, 04w0d01h
C>* fe80::/64 is directly connected, eth0, 04w0d01h

# run show ip route vrf mgmt
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

VRF mgmt:
S>* 0.0.0.0/0 [1/0] via xx.xx.23.1, eth0, weight 1, 04w0d01h
K * 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 04w0d01h
C>* xx.xx.23.0/25 is directly connected, eth0, 04w0d01h

I can reproduce this issue on different routers (Baremetal or VMs). Debug mode on, ipv6 routes install. Debug mode off, static route config is missing in frr, thus not installed into the kernel.

All systems using vrf support to separate out of band management. Systems without vrf enabled it works fine without debug mode on.

I can confirm that happens only if you ssh via vrf and set static route6 in default vrf
And it works without debug, if you stop vyos-configd service

sudo systemctl stop vyos-configd

Does disabling of vyos-configd has any impact in production?

PR https://github.com/vyos/vyos-1x/pull/947

Viacheslav changed the task status from Confirmed to Needs testing.Jul 31 2021, 12:59 PM

Viacheslav assigned this task to stepler.

This is causing a configtest failure on 1.4-rolling-202107301807 stemming from not cleaning up after bgp-evpn-l3vpn-pe-router ("Cannot delete default BGP instance. Dependent VRF instances exist"):

vyos@vyos:~$ configure
[edit]
vyos@vyos# save /tmp/config
Saving configuration to '/tmp/config'...
Done
[edit]
vyos@vyos# cat /tmp/config | strip-private
interfaces {
    ethernet eth0 {
        hw-id 52:54:00:5a:c3:d4
    }
    ethernet eth1 {
        hw-id 52:54:00:77:6c:0a
    }
    ethernet eth2 {
        hw-id 52:54:00:c6:24:8f
    }
    ethernet eth3 {
        hw-id 52:54:00:46:32:73
    }
    ethernet eth4 {
        hw-id 52:54:00:be:72:b5
    }
    ethernet eth5 {
        hw-id 52:54:00:35:0d:ad
    }
    ethernet eth6 {
        hw-id 52:54:00:0e:7f:1c
    }
    loopback lo {
    }
}
system {
    config-management {
        commit-revisions 100
    }
    console {
        device ttyS0 {
            speed 115200
        }
    }
    host-name vyos
    login {
        user vyos {
            authentication {
                encrypted-password xxxxxx
                plaintext-password xxxxxx
            }
        }
    }
    ntp {
        server time1.vyos.net {
        }
        server time2.vyos.net {
        }
        server time3.vyos.net {
        }
    }
    syslog {
        global {
            facility all {
                level info
            }
            facility protocols {
                level debug
            }
        }
    }
}


// Warning: Do not remove the following line.
// vyos-config-version: "bgp@1:broadcast-relay@1:cluster@1:config-management@1:conntrack@2:conntrack-sync@2:dhcp-relay@2:dhcp-server@5:dhcpv6-server@1:dns-forwarding@3:firewall@5:https@3:interfaces@23:ipoe-server@1:ipsec@8:isis@1:l2tp@4:lldp@1:mdns@1:nat@5:nat66@1:ntp@1:openconnect@1:policy@1:pppoe-server@5:pptp@2:qos@1:quagga@9:rpki@1:salt@1:snmp@2:ssh@2:sstp@4:system@21:vrf@3:vrrp@2:vyos-accel-ppp@2:wanloadbalance@3:webproxy@2:zone-policy@1"
// Release version: 1.4-rolling-202107301807
[edit]
vyos@vyos# show | commands | strip-private
set interfaces ethernet eth0 hw-id '52:54:00:5a:c3:d4'
set interfaces ethernet eth1 hw-id '52:54:00:77:6c:0a'
set interfaces ethernet eth2 hw-id '52:54:00:c6:24:8f'
set interfaces ethernet eth3 hw-id '52:54:00:46:32:73'
set interfaces ethernet eth4 hw-id '52:54:00:be:72:b5'
set interfaces ethernet eth5 hw-id '52:54:00:35:0d:ad'
set interfaces ethernet eth6 hw-id '52:54:00:0e:7f:1c'
set interfaces loopback lo
set system config-management commit-revisions '100'
set system console device ttyS0 speed '115200'
set system host-name 'vyos'
set system login user vyos authentication encrypted-password xxxxxx
set system login user vyos authentication plaintext-password xxxxxx
set system ntp server time1.vyos.net
set system ntp server time2.vyos.net
set system ntp server time3.vyos.net
set system syslog global facility all level 'info'
set system syslog global facility protocols level 'debug'
[edit]
vyos@vyos# vtysh -c 'show run'
Building configuration...

Current configuration:
!
frr version 7.5.1-20201222-185-gb3f4ff1d9
frr defaults traditional
hostname vyos
log syslog
log facility local7
service integrated-vtysh-config
!
line vty
!
end
[edit]
vyos@vyos# load /usr/libexec/vyos/tests/config/bgp-evpn-l3vpn-pe-router
Loading configuration from '/usr/libexec/vyos/tests/config/bgp-evpn-l3vpn-pe-router'
Load complete. Use 'commit' to make changes effective.
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# load /tmp/config
Loading configuration from '/tmp/config'
Load complete. Use 'commit' to make changes effective.
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# vtysh -c 'show run'
Building configuration...

Current configuration:
!
frr version 7.5.1-20201222-185-gb3f4ff1d9
frr defaults traditional
hostname vyos
log syslog
log facility local7
service integrated-vtysh-config
!
router bgp 100
 bgp router-id 172.29.255.1
 bgp log-neighbor-changes
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 no bgp network import-check
 neighbor ibgp peer-group
 neighbor ibgp remote-as 100
 neighbor ibgp update-source dum0
 neighbor 172.29.255.2 peer-group ibgp
 neighbor 172.29.255.3 peer-group ibgp
 !
 address-family l2vpn evpn
  neighbor ibgp activate
  advertise-all-vni
  advertise ipv4 unicast
 exit-address-family
!
router bgp 100 vrf green
 no bgp ebgp-requires-policy
 no bgp network import-check
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!
router bgp 100 vrf red
 no bgp ebgp-requires-policy
 no bgp network import-check
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!
router bgp 100 vrf blue
 no bgp ebgp-requires-policy
 no bgp network import-check
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!
line vty
!
end
[edit]
vyos@vyos# vtysh

Hello, this is FRRouting (version 7.5.1-20201222-185-gb3f4ff1d9).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

vyos# configure
vyos(config)# no router bgp 100
% Cannot delete default BGP instance. Dependent VRF instances exist
vyos(config)# exit
vyos# exit
[edit]
vyos@vyos# exit
Warning: configuration changes have not been saved.
exit
vyos@vyos:~$ show version

Version:          VyOS 1.4-rolling-202107301807
Release Train:    sagitta

Built by:         [email protected]
Built on:         Sat 31 Jul 2021 01:17 UTC
Build UUID:       f6cf0038-490b-42cf-ba22-b6dd73a834ad
Build Commit ID:  de1d70636feb41

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (Q35 + ICH9, 2009)
Hardware S/N:     
Hardware UUID:    1703deb0-afe4-4959-8dc8-caa3bec59ff9

Copyright:        VyOS maintainers and contributors

Further uses of BGP cause exceptions:

vyos@vyos:~$ configure
[edit]
vyos@vyos# set protocols bgp local-as 65000
[edit]
vyos@vyos# commit
[ protocols bgp ]
VyOS had an issue completing a command.

We are sorry that you encountered a problem while using VyOS.
There are a few things you can do to help us (and yourself):
- Contact us using the online help desk if you have a subscription:
  https://support.vyos.io/
- Make sure you are running the latest version of VyOS available at:
  https://vyos.net/get/
- Consult the community forum to see how to handle this issue:
  https://forum.vyos.io
- Join us on Slack where our users exchange help and advice:
  https://vyos.slack.com

When reporting problems, please include as much information as possible:
- do not obfuscate any data (feel free to contact us privately if your 
  business policy requires it)
- and include all the information presented below

Report Time:      2021-07-31 20:44:17
Image Version:    VyOS 1.4-rolling-202107301807
Release Train:    sagitta

Built by:         [email protected]
Built on:         Sat 31 Jul 2021 01:17 UTC
Build UUID:       f6cf0038-490b-42cf-ba22-b6dd73a834ad
Build Commit ID:  de1d70636feb41

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (Q35 + ICH9, 2009)
Hardware S/N:     
Hardware UUID:    1703deb0-afe4-4959-8dc8-caa3bec59ff9

Traceback (most recent call last):
  File "/usr/libexec/vyos/conf_mode/protocols_bgp.py", line 284, in <module>
    apply(c)
  File "/usr/libexec/vyos/conf_mode/protocols_bgp.py", line 272, in apply
    frr_cfg.commit_configuration(bgp_daemon)
  File "/usr/lib/python3/dist-packages/vyos/frr.py", line 458, in commit_configuration
    reload_configuration('\n'.join(self.config), daemon=daemon)
  File "/usr/lib/python3/dist-packages/vyos/frr.py", line 206, in reload_configuration
    raise CommitError('FRR configuration failed while running commit. Please ' \
vyos.frr.CommitError: FRR configuration failed while running commit. Please enable debugging to examine logs.

To enable debugging run: "touch /tmp/vyos.frr.debug" and "sudo systemctl stop vyos-configd"



[[protocols bgp]] failed
Commit failed

Enabling debugging reveals the same inability to execute no router bgp 100.

I have added a test which does not let you delete the default BGP instance when VRF bound instances exist.

I figured out the order we need to remove things in:

vyos@vyos:~$ configure
[edit]
vyos@vyos# load /usr/libexec/vyos/tests/config/bgp-evpn-l3vpn-pe-router
Loading configuration from '/usr/libexec/vyos/tests/config/bgp-evpn-l3vpn-pe-router'
Load complete. Use 'commit' to make changes effective.
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# vtysh

Hello, this is FRRouting (version 7.5.1-20201222-185-gb3f4ff1d9).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

vyos# configure
vyos(config)# no router bgp 100
% Cannot delete default BGP instance. Dependent VRF instances exist
vyos(config)# no router bgp 100 vrf blue
% Please unconfigure l3vni 2000
vyos(config)# vrf blue
vyos(config-vrf)# no vni 2000
vyos(config-vrf)# exit
vyos(config)# no router bgp 100 vrf blue
vyos(config)# vrf red
vyos(config-vrf)# no vni 3000
vyos(config-vrf)# exit
vyos(config)# no router bgp 100 vrf red
vyos(config)# vrf green
vyos(config-vrf)# no vni 4000
vyos(config-vrf)# exit
vyos(config)# no router bgp 100 vrf green
vyos(config)# no router bgp 100
vyos(config)# exit
vyos# exit
[edit]
vyos@vyos# exit
Warning: configuration changes have not been saved.
exit

That is, first delete vrf vni, then bgp vrf, then bgp.

It doesn't seem to matter which order we add those configurations in.

Also, this would be a good FRR commit to cherry-pick (I ended up partially re-implementing it to debug this): https://github.com/FRRouting/frr/commit/641b032e23e614cec8dcbeb993a61f2ff08dfde2

Hi @stepler,

I tried this locally and you can do no router bgp 100 vrf blue before no vni 2000, my vtysh instance does not scream for an error - it more feels like a programming issue on our side.

Adding touch /tmp/vyos.frr.debug to add more frr-reload output revels that a BGP vrf is deleted - but on subsequent runs it seems it's re-added again.

@stepler I created https://github.com/FRRouting/frr/pull/9281 for "easier" debugging

@stepler interesting - this bahavior changes when running from frr-reload vs. vtysh.

I have commented out the VNI part for now to get the rolling build running again and need to think how to handle multiple priorities in the same subsystem.

sarthurdev added a subscriber: sarthurdev.Aug 6 2021, 5:01 PM

Looks good on 1.4-rolling-202108300430:

vyos@vyos:~$ configure
[edit]
vyos@vyos# set vrf name MGMT table 100
[edit]
vyos@vyos# set interfaces ethernet eth0 vrf MGMT
[edit]
vyos@vyos# set interfaces ethernet eth0 address 10.0.0.2/24
[edit]
vyos@vyos# set interfaces ethernet eth1 address 192.168.251.2/24
[edit]
vyos@vyos# set vrf name MGMT protocols static route 0.0.0.0/0 next-hop 10.0.0.1
[edit]
vyos@vyos# commit
Adapter does not support changing large-receive-offload settings!
Adapter does not support changing large-receive-offload settings!
[edit]
vyos@vyos# set protocols static route 0.0.0.0/0 next-hop 192.168.251.1
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# exit
Warning: configuration changes have not been saved.
exit
vyos@vyos:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

S>* 0.0.0.0/0 [1/0] via 192.168.251.1, eth1, weight 1, 00:00:02
C>* 192.168.251.0/24 is directly connected, eth1, 00:00:05
vyos@vyos:~$ show version

Version:          VyOS 1.4-rolling-202108300430
Release Train:    sagitta

Built by:         [email protected]
Built on:         Mon 30 Aug 2021 04:30 UTC
Build UUID:       1f5fc6a3-24fb-4e92-a360-13b095e279b1
Build Commit ID:  393ad560653dc2

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (Q35 + ICH9, 2009)
Hardware S/N:     
Hardware UUID:    f555f428-c684-4e87-be48-2fa2a7fc42eb

Copyright:        VyOS maintainers and contributors

Not quite there with the VNI configuration on 1.4-rolling-202108300430 (see middle vtysh output: VRF green has a VNI configured, but VRFs blue and red don't):

vyos@vyos:~$ configure
[edit]
vyos@vyos# save /tmp/config
Saving configuration to '/tmp/config'...
Done
[edit]
vyos@vyos# vtysh -c 'show run'
Building configuration...

Current configuration:
!
frr version 7.5.1-20210619-12-g3f8a74e70
frr defaults traditional
hostname vyos
log syslog
log facility local7
service integrated-vtysh-config
!
line vty
!
end
[edit]
vyos@vyos# load /usr/libexec/vyos/tests/config/bgp-evpn-l3vpn-pe-router
Loading configuration from '/usr/libexec/vyos/tests/config/bgp-evpn-l3vpn-pe-router'
Load complete. Use 'commit' to make changes effective.
[edit]
vyos@vyos# commit
Adapter does not support changing large-receive-offload settings!
Adapter does not support changing large-receive-offload settings!
Adapter does not support changing large-receive-offload settings!
Adapter does not support changing large-receive-offload settings!
Adapter does not support changing large-receive-offload settings!
Adapter does not support changing large-receive-offload settings!
Adapter does not support changing large-receive-offload settings!
[edit]
vyos@vyos# vtysh -c 'show run'
Building configuration...

Current configuration:
!
frr version 7.5.1-20210619-12-g3f8a74e70
frr defaults traditional
hostname vyos
log syslog
log facility local7
service integrated-vtysh-config
!
vrf green
 vni 4000
 exit-vrf
!
vrf mgmt
 ip route 0.0.0.0/0 192.0.2.62
 ipv6 route ::/0 2001:db8:ffff::1
 exit-vrf
!
interface eth1
 ip ospf network point-to-point
!
interface eth3
 ip ospf network point-to-point
!
router bgp 100
 bgp router-id 172.29.255.1
 bgp log-neighbor-changes
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 no bgp network import-check
 neighbor ibgp peer-group
 neighbor ibgp remote-as 100
 neighbor ibgp update-source dum0
 neighbor 172.29.255.2 peer-group ibgp
 neighbor 172.29.255.3 peer-group ibgp
 !
 address-family l2vpn evpn
  neighbor ibgp activate
  advertise-all-vni
  advertise ipv4 unicast
 exit-address-family
!
router bgp 100 vrf blue
 no bgp ebgp-requires-policy
 no bgp network import-check
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!
router bgp 100 vrf red
 no bgp ebgp-requires-policy
 no bgp network import-check
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!
router bgp 100 vrf green
 no bgp ebgp-requires-policy
 no bgp network import-check
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!
router ospf
 ospf router-id 172.29.255.1
 log-adjacency-changes detail
 auto-cost reference-bandwidth 100
 timers throttle spf 200 1000 10000
 redistribute connected
 passive-interface default
 no passive-interface eth1
 no passive-interface eth3
 network 172.29.0.2/31 area 0
 network 172.29.0.6/31 area 0
!
line vty
!
end
[edit]
vyos@vyos# load /tmp/config
Loading configuration from '/tmp/config'
Load complete. Use 'commit' to make changes effective.
[edit]
vyos@vyos# commit
Adapter does not support changing large-receive-offload settings!
Adapter does not support changing large-receive-offload settings!
Adapter does not support changing large-receive-offload settings!
Adapter does not support changing large-receive-offload settings!
Adapter does not support changing large-receive-offload settings!
Adapter does not support changing large-receive-offload settings!
Adapter does not support changing large-receive-offload settings!
[edit]
vyos@vyos# vtysh -c 'show run'
Building configuration...

Current configuration:
!
frr version 7.5.1-20210619-12-g3f8a74e70
frr defaults traditional
hostname vyos
log syslog
log facility local7
service integrated-vtysh-config
!
line vty
!
end
[edit]
vyos@vyos# exit
Warning: configuration changes have not been saved.
exit
vyos@vyos:~$ show version

Version:          VyOS 1.4-rolling-202108300430
Release Train:    sagitta

Built by:         [email protected]
Built on:         Mon 30 Aug 2021 04:30 UTC
Build UUID:       1f5fc6a3-24fb-4e92-a360-13b095e279b1
Build Commit ID:  393ad560653dc2

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (Q35 + ICH9, 2009)
Hardware S/N:     
Hardware UUID:    f555f428-c684-4e87-be48-2fa2a7fc42eb

Copyright:        VyOS maintainers and contributors

Looks good on 1.4-rolling-202303170317:

vyos@vyos:~$ configure
[edit]
vyos@vyos# save /tmp/config
Saving configuration to '/tmp/config'...
Done
[edit]
vyos@vyos# vtysh -c 'show running-config'
Building configuration...

Current configuration:
!
frr version 8.4.2
frr defaults traditional
hostname vyos
log syslog
log facility local7
service integrated-vtysh-config
!
rpki
exit
!
end
[edit]
vyos@vyos# load /usr/libexec/vyos/tests/config/bgp-evpn-l3vpn-pe-router
Loading configuration from '/usr/libexec/vyos/tests/config/bgp-evpn-l3vpn-pe-router'
Load complete. Use 'commit' to make changes effective.
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# vtysh -c 'show running-config'
Building configuration...

Current configuration:
!
frr version 8.4.2
frr defaults traditional
hostname vyos
log syslog
log facility local7
service integrated-vtysh-config
!
vrf blue
 vni 2000
exit-vrf
!
vrf green
 vni 4000
exit-vrf
!
vrf red
 vni 3000
exit-vrf
!
vrf mgmt
 ip route 0.0.0.0/0 192.0.2.62
 ipv6 route ::/0 2001:db8:ffff::1
exit-vrf
!
interface eth1
 ip ospf dead-interval 40
 ip ospf network point-to-point
 no ip ospf passive
exit
!
interface eth3
 ip ospf dead-interval 40
 ip ospf network point-to-point
 no ip ospf passive
exit
!
router bgp 100
 bgp router-id 172.29.255.1
 bgp log-neighbor-changes
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 no bgp network import-check
 neighbor ibgp peer-group
 neighbor ibgp remote-as 100
 neighbor ibgp update-source dum0
 neighbor 172.29.255.2 peer-group ibgp
 neighbor 172.29.255.3 peer-group ibgp
 !
 address-family l2vpn evpn
  neighbor ibgp activate
  advertise-all-vni
  advertise ipv4 unicast
 exit-address-family
exit
!
router bgp 100 vrf blue
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 no bgp network import-check
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
exit
!
router bgp 100 vrf red
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 no bgp network import-check
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
exit
!
router bgp 100 vrf green
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 no bgp network import-check
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
exit
!
router ospf
 ospf router-id 172.29.255.1
 log-adjacency-changes detail
 auto-cost reference-bandwidth 100
 timers throttle spf 200 1000 10000
 redistribute connected
 redistribute babel
 passive-interface default
 network 172.29.0.2/31 area 0
 network 172.29.0.6/31 area 0
exit
!
rpki
exit
!
end
[edit]
vyos@vyos# load /tmp/config
Loading configuration from '/tmp/config'
Load complete. Use 'commit' to make changes effective.
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# vtysh -c 'show running-config'
Building configuration...

Current configuration:
!
frr version 8.4.2
frr defaults traditional
hostname vyos
log syslog
log facility local7
service integrated-vtysh-config
!
interface eth1
exit
!
interface eth3
exit
!
rpki
exit
!
end
[edit]
vyos@vyos# exit
Warning: configuration changes have not been saved.
exit
vyos@vyos:~$ show version
Version:          VyOS 1.4-rolling-202303170317
Release train:    current

Built by:         [email protected]
Built on:         Fri 17 Mar 2023 03:17 UTC
Build UUID:       45391302-1240-4cc7-95a8-da8ee6390765
Build commit ID:  e887f582cfd7de

Architecture:     x86_64
Boot via:         installed image
System type:       guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (Q35 + ICH9, 2009)
Hardware S/N:     
Hardware UUID:    229cc511-8f17-49a4-bc7f-763b150284e3

Copyright:        VyOS maintainers and contributors

Static routes not installed into kernel nor frrClosed, ResolvedPublicBUGActions

Description

Details

Event Timeline

Static routes not installed into kernel nor frr
Closed, ResolvedPublicBUG
Actions