When working normally, routes from router 100.64.9.255 show the proper RD of 100.64.9.255:10 and a vpn export RT of 10:100.
This is a VRF (VPNNet) that has 1 ethernet interface in it and one dummy interface for demonstration. The advertising or local router is dfw01-vpnpe-01 and the remote is sea01-vpnpe-01.
From remote router:
vyos@sea01-vpnpe-01:~$ show bgp ipv4 vpn
BGP table version is 5, local router ID is 100.64.1.255, vrf id 0
Default local pref 100, local AS 4211000000
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
~~~truncated~~~
Route Distinguisher: 100.64.9.255:10
*>i192.168.84.0/22 100.64.9.255 0 100 0 i
UN=100.64.9.255 EC{10:100} label=80 type=bgp, subtype=0
*>i192.168.85.0/24 100.64.9.255 0 100 0 i
UN=100.64.9.255 EC{10:100} label=80 type=bgp, subtype=0
*>i192.168.86.1/32 100.64.9.255 0 100 0 i
UN=100.64.9.255 EC{10:100} label=80 type=bgp, subtype=0From local router:
vyos@dfw01-vpnpe-01:~$ show bgp ipv4 vpn
BGP table version is 5, local router ID is 100.64.9.255, vrf id 0
Default local pref 100, local AS 4211000000
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 100.64.1.254:10
~~~truncated~~~
Route Distinguisher: 100.64.9.255:10
*> 192.168.84.0/22 0.0.0.0@5< 0 32768 i
UN=0.0.0.0 EC{10:100} label=80 type=bgp, subtype=5
0.0.0.0@5< 0 32768 i
UN=0.0.0.0 EC{10:100} label=80 type=bgp, subtype=5
*> 192.168.85.0/24 0.0.0.0@5< 0 32768 i
UN=0.0.0.0 EC{10:100} label=80 type=bgp, subtype=5
*> 192.168.86.1/32 0.0.0.0@5< 0 32768 i
UN=0.0.0.0 EC{10:100} label=80 type=bgp, subtype=5if the interfaces (eth2 and dum66) are deleted, the respective routes (192.168.85.0/24 and 192.168.86.1/32) are removed as expected:
From remote router:
vyos@sea01-vpnpe-01:~$ show bgp ipv4 vpn
BGP table version is 5, local router ID is 100.64.1.255, vrf id 0
Default local pref 100, local AS 4211000000
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 100.64.1.254:10
~~~truncated~~~
Route Distinguisher: 100.64.9.255:10
*>i192.168.84.0/22 100.64.9.255 0 100 0 i
UN=100.64.9.255 EC{10:100} label=80 type=bgp, subtype=0from local router:
vyos@dfw01-vpnpe-01:~$ show bgp ipv4 vpn
BGP table version is 5, local router ID is 100.64.9.255, vrf id 0
Default local pref 100, local AS 4211000000
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
~~~truncated~~~
Route Distinguisher: 100.64.9.255:10
*> 192.168.84.0/22 0.0.0.0@5< 0 32768 i
UN=0.0.0.0 EC{10:100} label=80 type=bgp, subtype=5
0.0.0.0@5< 0 32768 i
UN=0.0.0.0 EC{10:100} label=80 type=bgp, subtype=5if the interfaces are added back, they propagate without issue and the RD/RT are unchanged:
From remote router:
vyos@sea01-vpnpe-01:~$ show bgp ipv4 vpn
BGP table version is 5, local router ID is 100.64.1.255, vrf id 0
Default local pref 100, local AS 4211000000
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
~~~truncated~~~
Route Distinguisher: 192.168.85.3:2
*>i192.168.84.0/22 100.64.9.255 0 100 0 i
UN=100.64.9.255 EC{192.168.85.3:2} label=80 type=bgp, subtype=0
*>i192.168.85.0/24 100.64.9.255 0 100 0 i
UN=100.64.9.255 EC{192.168.85.3:2} label=80 type=bgp, subtype=0but if the interfaces are deleted/non-existent at boot and added in (a new server with a new vrf for example)
Local router prior to adding interfaces back:
vyos@dfw01-vpnpe-01:~$ show bgp ipv4 vpn
BGP table version is 5, local router ID is 100.64.9.255, vrf id 0
Default local pref 100, local AS 4211000000
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
~~~truncated~~~
Route Distinguisher: 100.64.9.255:10
*> 192.168.84.0/22 0.0.0.0@5< 0 32768 i
UN=0.0.0.0 EC{10:100} label=80 type=bgp, subtype=5
0.0.0.0@5< 0 32768 i
UN=0.0.0.0 EC{10:100} label=80 type=bgp, subtype=5Local Router after adding interfaces back:
vyos@dfw01-vpnpe-01:~$ show bgp ipv4 vpn
BGP table version is 5, local router ID is 100.64.9.255, vrf id 0
Default local pref 100, local AS 4211000000
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
~~~truncated~~~
Route Distinguisher: 192.168.86.1:2
*> 192.168.84.0/22 0.0.0.0@5< 0 32768 i
UN=0.0.0.0 EC{192.168.86.1:2} label=80 type=bgp, subtype=5
0.0.0.0@5< 0 32768 i
UN=0.0.0.0 EC{192.168.86.1:2} label=80 type=bgp, subtype=5
*> 192.168.85.0/24 0.0.0.0@5< 0 32768 i
UN=0.0.0.0 EC{192.168.86.1:2} label=80 type=bgp, subtype=5
*> 192.168.86.1/32 0.0.0.0@5< 0 32768 i
UN=0.0.0.0 EC{192.168.86.1:2} label=80 type=bgp, subtype=5After adding the interfaces to the VRF with no other interfaces directly in it causes the RD and RT to change to the IP address of one of the added interfaces.
In this case it assumed 192.168.86.1 which is the interface of dum66
I also noticed that the MPLS configuration goes missing from the router after a reboot but that seems related to wireguard interfaces and will be a different bug report
config (templatized format, so similar to others):
set protocols mpls interface 'wg1000' set protocols mpls interface 'wg1001' set protocols mpls interface 'eth1' set protocols mpls interface 'dum0' set protocols mpls ldp discovery transport-ipv4-address '100.64.9.255' set protocols mpls ldp interface 'eth1' set protocols mpls ldp interface 'wg1000' set protocols mpls ldp interface 'wg1001' set protocols mpls ldp interface 'dum0' set protocols mpls ldp router-id '100.64.9.255' set vrf name VPNNet protocols bgp address-family ipv4-unicast export vpn set vrf name VPNNet protocols bgp address-family ipv4-unicast import vpn set vrf name VPNNet protocols bgp address-family ipv4-unicast label vpn export 'auto' set vrf name VPNNet protocols bgp address-family ipv4-unicast maximum-paths ebgp '2' set vrf name VPNNet protocols bgp address-family ipv4-unicast maximum-paths ibgp '2' set vrf name VPNNet protocols bgp address-family ipv4-unicast rd vpn export '100.64.9.255:10' set vrf name VPNNet protocols bgp address-family ipv4-unicast redistribute connected route-map 'BGP-VPNNET-TAG-LOCAL' set vrf name VPNNet protocols bgp address-family ipv4-unicast redistribute static route-map 'BGP-VPNNET-TAG-LOCAL' set vrf name VPNNet protocols bgp address-family ipv4-unicast route-target vpn export '10:100' set vrf name VPNNet protocols bgp address-family ipv4-unicast route-target vpn import '10:100 10:200' set vrf name VPNNet protocols bgp parameters cluster-id '192.168.84.0' set vrf name VPNNet protocols bgp parameters log-neighbor-changes set vrf name VPNNet protocols bgp peer-group EBGP_VPNNET_EDGE_V4 address-family ipv4-unicast route-map export 'BGP-VPNNET-EDGE-OUT-V4' set vrf name VPNNet protocols bgp peer-group EBGP_VPNNET_EDGE_V4 address-family ipv4-unicast route-map import 'BGP-VPNNET-EDGE-IN-V4' set vrf name VPNNet protocols bgp peer-group EBGP_VPNNET_EDGE_V4 address-family ipv4-unicast soft-reconfiguration inbound set vrf name VPNNet protocols bgp peer-group EBGP_VPNNET_EDGE_V4 ebgp-multihop '255' set vrf name VPNNet protocols bgp peer-group EBGP_VPNNET_PEER_V4 address-family ipv4-unicast route-map export 'BGP-VPNNET-PEER-OUT-V4' set vrf name VPNNet protocols bgp peer-group EBGP_VPNNET_PEER_V4 address-family ipv4-unicast route-map import 'BGP-VPNNET-PEER-IN-V4' set vrf name VPNNet protocols bgp peer-group EBGP_VPNNET_PEER_V4 address-family ipv4-unicast soft-reconfiguration inbound set vrf name VPNNet protocols bgp peer-group EBGP_VPNNET_PEER_V4 ebgp-multihop '255' set vrf name VPNNet protocols bgp system-as '4211000000' set vrf name VPNNet protocols static route 192.168.84.0/22 blackhole set vrf name VPNNet table '100'