Whenever an interface in a VRF goes down and up on one PE router, the route is not reinstalled on the other PE.
Topology:
The setup is very similar to L3VPN for Hub-and-Spoke connectivity with VyOS. See the diagram:
Steps to reproduce:
- Verify that 10.0.0.0/24 is installed in PE02's routing table. Note the RD.
vyos@PE01:~$ show bgp ipv4 vpn
BGP table version is 2, local router ID is 192.168.255.10, vrf id 0
Default local pref 100, local AS 65002
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 192.168.255.10:200
*> 10.0.10.0/24 0.0.0.0@5< 0 32768 ?
UN=0.0.0.0 EC{65002:200} label=80 type=bgp, subtype=5
Route Distinguisher: 192.168.255.11:200
*>i10.0.0.0/24 192.168.255.11 0 100 0 ?
UN=192.168.255.11 EC{65002:200} label=80 type=bgp, subtype=0
Displayed 2 routes and 2 total paths
vyos@PE01:~$ show ip route vrf BLUE
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF BLUE:
B> 10.0.0.0/24 [200/0] via 192.168.255.11 (vrf default) (recursive), label 80, weight 1, 00:01:07
* via 192.168.10.0, eth0 (vrf default), label 19/80, weight 1, 00:01:07
C>* 10.0.10.0/24 is directly connected, eth1, 00:09:57- Unplug the link between CLIENT02 and PE02
- The route is removed from PE01
vyos@PE01:~$ show bgp ipv4 vpn
BGP table version is 2, local router ID is 192.168.255.10, vrf id 0
Default local pref 100, local AS 65002
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 192.168.255.10:200
*> 10.0.10.0/24 0.0.0.0@5< 0 32768 ?
UN=0.0.0.0 EC{65002:200} label=80 type=bgp, subtype=5
vyos@PE01:~$ show ip route vrf BLUE
Displayed 1 routes and 1 total paths
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF BLUE:
C>* 10.0.10.0/24 is directly connected, eth1, 00:10:46- Plug back in the link between CLIENT02 and PE02
- The route appears under show bgp ipv4 vpn (with the wrong RD! - it should be 192.168.255.11:200, not 10.0.01:2) but is not installed back in the routing table (show ip route vrf BLUE)
vyos@PE01:~$ show bgp ipv4 vpn
BGP table version is 1, local router ID is 192.168.255.10, vrf id 0
Default local pref 100, local AS 65002
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 10.0.0.1:2
*>i10.0.0.0/24 192.168.255.11 0 100 0 ?
UN=192.168.255.11 EC{10.0.0.1:2} label=80 type=bgp, subtype=0
Route Distinguisher: 192.168.255.10:200
*> 10.0.10.0/24 0.0.0.0@5< 0 32768 ?
UN=0.0.0.0 EC{65002:200} label=80 type=bgp, subtype=5
Displayed 2 routes and 2 total paths
vyos@PE01:~$ show ip route vrf BLUE
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF BLUE:
C>* 10.0.10.0/24 is directly connected, eth1, 00:11:23The only way that I've been able to get the route to appear back in PE01's routing table is to reboot PE02. Restarting the routing daemons on either PE (restart all) doesn't seem to work either (the route isn't installed and the wrong RD is used).
I've also reproduced this behaviour switching the sides and with VPNv6.