Page MenuHomeVyOS Platform
Create Task
Maniphest T7009

BGP EVPN VRF Route not Advertised
Open, NormalPublicBUG
Actions

Description

The config below is derived from the evpn blog examples:

This is a minimal bgp/evpn setup with a route reflector and a single client. Each has a static vrf route 172.20.X.0/20 intended to be advertised in the test vrf.

Each node is a fresh setup of 1.5-rolling-202412310006, with the following config commands applied.

Node rtr-A / route-reflector

delete system service
set interfaces bridge br1 enable-vlan
set interfaces bridge br1 member interface vxlan1
set interfaces bridge br1 vif 10 address '172.20.128.1/24'
set interfaces bridge br1 vif 10 vrf 'test'
set interfaces bridge br1 vif 4094 vrf 'test'
set interfaces ethernet eth0 address '10.0.0.1/24'
set interfaces vxlan vxlan1 parameters external
set interfaces vxlan vxlan1 parameters nolearning
set interfaces vxlan vxlan1 source-address '10.0.0.1'
set interfaces vxlan vxlan1 vlan-to-vni 10 vni '10001'
set interfaces vxlan vxlan1 vlan-to-vni 4094 vni '10000'
set protocols bgp address-family l2vpn-evpn advertise ipv4 unicast
set protocols bgp address-family l2vpn-evpn advertise-all-vni
set protocols bgp address-family l2vpn-evpn advertise-svi-ip
set protocols bgp listen range 10.0.0.0/24 peer-group 'net'
set protocols bgp parameters log-neighbor-changes
set protocols bgp parameters router-id '172.20.8.1'
set protocols bgp peer-group net address-family ipv4-unicast route-reflector-client
set protocols bgp peer-group net address-family l2vpn-evpn route-reflector-client
set protocols bgp peer-group net capability dynamic
set protocols bgp peer-group net remote-as 'internal'
set protocols bgp system-as '4242421989'
set system host-name 'rtr-A'
set vrf name test protocols bgp address-family ipv4-unicast network 172.20.128.0/20
set vrf name test protocols bgp address-family l2vpn-evpn advertise ipv4 unicast
set vrf name test protocols bgp system-as '4242421989'
set vrf name test protocols static route 172.20.128.0/20 blackhole distance '255'
set vrf name test table '100'
set vrf name test vni '10000'

Node rtr-B / route client

delete system service
set interfaces bridge br1 enable-vlan
set interfaces bridge br1 member interface vxlan1
set interfaces bridge br1 vif 10 address '172.20.80.1/24'
set interfaces bridge br1 vif 10 vrf 'test'
set interfaces bridge br1 vif 4094 vrf 'test'
set interfaces ethernet eth0 address '10.0.0.10/24'
set interfaces vxlan vxlan1 parameters external
set interfaces vxlan vxlan1 parameters nolearning
set interfaces vxlan vxlan1 source-address '10.0.0.10'
set interfaces vxlan vxlan1 vlan-to-vni 10 vni '10002'
set interfaces vxlan vxlan1 vlan-to-vni 4094 vni '10000'
set protocols bgp address-family l2vpn-evpn advertise ipv4 unicast
set protocols bgp address-family l2vpn-evpn advertise-all-vni
set protocols bgp address-family l2vpn-evpn advertise-svi-ip
set protocols bgp neighbor 10.0.0.1 peer-group 'net'
set protocols bgp parameters log-neighbor-changes
set protocols bgp parameters router-id '172.20.8.10'
set protocols bgp peer-group net address-family ipv4-unicast nexthop-self
set protocols bgp peer-group net address-family l2vpn-evpn nexthop-self
set protocols bgp peer-group net capability dynamic
set protocols bgp peer-group net remote-as 'internal'
set protocols bgp system-as '4242421989'
set system host-name 'rtr-B'
set vrf name test protocols bgp address-family ipv4-unicast network 172.20.80.0/20
set vrf name test protocols bgp address-family l2vpn-evpn advertise ipv4 unicast
set vrf name test protocols bgp system-as '4242421989'
set vrf name test protocols static route 172.20.80.0/20 blackhole distance '255'
set vrf name test table '100'
set vrf name test vni '10000'

After rebooting both nodes, a bgp session is established, but the static /20 L3-vni routes are not advertised.

Node rtr-A / route-reflector

vyos@rtr-A:~$ show system image 
Name                      Default boot    Running
------------------------  --------------  ---------
1.5-rolling-202412310006  Yes             Yes


vyos@rtr-A:~$ show ip route vrf all
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF default:
C>* 10.0.0.0/24 is directly connected, eth0, weight 1, 00:00:28
L>* 10.0.0.1/32 is directly connected, eth0, weight 1, 00:00:28

VRF test:
K>* 127.0.0.0/8 [0/0] is directly connected, test, weight 1, 00:00:29
B>* 172.20.80.1/32 [200/0] via 10.0.0.10, br1.4094 onlink, weight 1, 00:00:20
S   172.20.128.0/20 [255/0] unreachable (blackhole), weight 1, 00:00:26
C>* 172.20.128.0/24 is directly connected, br1.10, weight 1, 00:00:27
L>* 172.20.128.1/32 is directly connected, br1.10, weight 1, 00:00:27


vyos@rtr-A:~$ show bgp l2vpn evpn neighbors 10.0.0.10 advertised-routes 
BGP table version is 0, local router ID is 172.20.8.1
Default local pref 100, local AS 4242421989
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 172.20.8.1:3
 *> [2]:[0]:[48]:[9a:9f:6b:8a:ab:81]:[32]:[172.20.128.1]
                                  100  32768 i
 *> [2]:[0]:[48]:[9a:9f:6b:8a:ab:81]:[128]:[fe80::989f:6bff:fe8a:ab81]
                                  100  32768 i
 *> [3]:[0]:[32]:[10.0.0.1]
                                  100  32768 i
Route Distinguisher: 172.20.8.10:3
 *> [2]:[0]:[48]:[0e:86:bf:a0:b1:9c]:[32]:[172.20.80.1]
                             0    100      0 i
 *> [2]:[0]:[48]:[0e:86:bf:a0:b1:9c]:[128]:[fe80::c86:bfff:fea0:b19c]
                             0    100      0 i
 *> [3]:[0]:[32]:[10.0.0.10]
                             0    100      0 i

Total number of prefixes 6

Node rtr-B / route client

vyos@rtr-B:~$ show system image 
Name                      Default boot    Running
------------------------  --------------  ---------
1.5-rolling-202412310006  Yes             Yes


vyos@rtr-B:~$ show ip route vrf all
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF default:
C>* 10.0.0.0/24 is directly connected, eth0, weight 1, 00:00:46
L>* 10.0.0.10/32 is directly connected, eth0, weight 1, 00:00:46

VRF test:
K>* 127.0.0.0/8 [0/0] is directly connected, test, weight 1, 00:00:47
S   172.20.80.0/20 [255/0] unreachable (blackhole), weight 1, 00:00:44
C>* 172.20.80.0/24 is directly connected, br1.10, weight 1, 00:00:45
L>* 172.20.80.1/32 is directly connected, br1.10, weight 1, 00:00:45
B>* 172.20.128.1/32 [200/0] via 10.0.0.1, br1.4094 onlink, weight 1, 00:00:42


vyos@rtr-B:~$ show bgp l2vpn evpn neighbors 10.0.0.1 advertised-routes 
BGP table version is 0, local router ID is 172.20.8.10
Default local pref 100, local AS 4242421989
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 172.20.8.10:3
 *> [2]:[0]:[48]:[0e:86:bf:a0:b1:9c]:[32]:[172.20.80.1]
                                  100  32768 i
 *> [2]:[0]:[48]:[0e:86:bf:a0:b1:9c]:[128]:[fe80::c86:bfff:fea0:b19c]
                                  100  32768 i
 *> [3]:[0]:[32]:[10.0.0.10]
                                  100  32768 i

Total number of prefixes 3

When the same config is applied to a fresh install of `1.5-rolling-202408230022, the /20 L3-vni routes are advertised as expected.

Node rtr-A / route-reflector

vyos@rtr-A:~$ show system image 
Name                      Default boot    Running
------------------------  --------------  ---------
1.5-rolling-202408230022  Yes             Yes


vyos@rtr-A:~$ show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF default:
C>* 10.0.0.0/24 is directly connected, eth0, 00:02:06

VRF test:
B>* 172.20.80.0/20 [200/0] via 10.0.0.10, br1.4094 onlink, weight 1, 00:00:12
B>* 172.20.80.1/32 [200/0] via 10.0.0.10, br1.4094 onlink, weight 1, 00:00:12
S   172.20.128.0/20 [255/0] unreachable (blackhole), weight 1, 00:02:04
C>* 172.20.128.0/24 is directly connected, br1.10, 00:02:05


vyos@rtr-A:~$ show bgp l2vpn evpn neighbors 10.0.0.10 advertised-routes 
BGP table version is 0, local router ID is 172.20.8.1
Default local pref 100, local AS 4242421989
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 172.20.8.1:3
 *> [2]:[0]:[48]:[9a:9f:6b:8a:ab:81]:[32]:[172.20.128.1]
                                  100  32768 i
 *> [2]:[0]:[48]:[9a:9f:6b:8a:ab:81]:[128]:[fe80::989f:6bff:fe8a:ab81]
                                  100  32768 i
 *> [3]:[0]:[32]:[10.0.0.1]
                                  100  32768 i
Route Distinguisher: 172.20.8.10:3
 *> [2]:[0]:[48]:[0e:86:bf:a0:b1:9c]:[32]:[172.20.80.1]
                             0    100      0 i
 *> [2]:[0]:[48]:[0e:86:bf:a0:b1:9c]:[128]:[fe80::c86:bfff:fea0:b19c]
                             0    100      0 i
 *> [3]:[0]:[32]:[10.0.0.10]
                             0    100      0 i
Route Distinguisher: 172.20.80.1:2
 *> [5]:[0]:[20]:[172.20.80.0]
                             0    100      0 i
Route Distinguisher: 172.20.128.1:2
 *> [5]:[0]:[20]:[172.20.128.0]
                             0    100  32768 i

Total number of prefixes 8

Node rtr-B / route client

vyos@rtr-B:~$ show system image 
Name                      Default boot    Running
------------------------  --------------  ---------
1.5-rolling-202408230022  Yes             Yes


vyos@rtr-B:~$ show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF default:
C>* 10.0.0.0/24 is directly connected, eth0, 00:02:48

VRF test:
S   172.20.80.0/20 [255/0] unreachable (blackhole), weight 1, 00:02:46
C>* 172.20.80.0/24 is directly connected, br1.10, 00:02:47
B>* 172.20.128.0/20 [200/0] via 10.0.0.1, br1.4094 onlink, weight 1, 00:00:40
B>* 172.20.128.1/32 [200/0] via 10.0.0.1, br1.4094 onlink, weight 1, 00:00:40


vyos@rtr-B:~$ show bgp l2vpn evpn neighbors 10.0.0.1 advertised-routes 
BGP table version is 0, local router ID is 172.20.8.10
Default local pref 100, local AS 4242421989
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 172.20.8.10:3
 *> [2]:[0]:[48]:[0e:86:bf:a0:b1:9c]:[32]:[172.20.80.1]
                                  100  32768 i
 *> [2]:[0]:[48]:[0e:86:bf:a0:b1:9c]:[128]:[fe80::c86:bfff:fea0:b19c]
                                  100  32768 i
 *> [3]:[0]:[32]:[10.0.0.10]
                                  100  32768 i
Route Distinguisher: 172.20.80.1:2
 *> [5]:[0]:[20]:[172.20.80.0]
                             0    100  32768 i

Total number of prefixes 4

Details

Version
1.5-rolling-202412310006
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Related Objects

Event Timeline

kevinrausch created this task.Jan 4 2025, 6:53 PM
kevinrausch updated the task description. (Show Details)Jan 4 2025, 6:57 PM
kevinrausch added a comment.EditedJan 4 2025, 8:49 PM

I tested the following builds using the same config noted above:

vyos-1.5-rolling-202412050007-generic-amd64.iso - works
vyos-1.5-rolling-202412060007-generic-amd64.iso - works
vyos-1.5-rolling-202412160007-generic-amd64.iso - works
vyos-1.5-rolling-202412310006-generic-amd64.iso - fails
vyos-1.5-rolling-202501031241-generic-amd64.iso - fails

I noticed the builds that fail show new Kernel and Local routes in the vrf table, which were not shown in working builds:

VRF test:
K>* 127.0.0.0/8 [0/0] is directly connected, test, weight 1, 00:00:47
L>* 172.20.X.1/32 is directly connected, br1.10, weight 1, 00:00:45
...
pasik subscribed.Jan 5 2025, 9:49 AM
Viacheslav triaged this task as Normal priority.Jan 6 2025, 11:17 AM
Viacheslav subscribed.Jan 6 2025, 11:21 AM

@kevinrausch You see the kernel and local routes as the newest version uses FRR 10.2.x T6747