FRR Misconfig when using multiple VRF VNI
Closed, ResolvedPublicBUG
Actions

Assigned To

Authored By

	ssasso
	Nov 2 2021, 2:14 PM

Description

When configuring multiple VRF VNI, only one is configured on frr.conf:

i.e. This configuration:

set vrf name blue table '2000'
set vrf name blue vni '2000'
set vrf name red table '3000'
set vrf name red vni '3000'

Is saved on frr.conf as:

!
vrf blue
 vni 2000
 exit-vrf
!

Verified with:

vyos@LEAF-1:~$ show bgp l2vpn evpn vni
Advertise Gateway Macip: Disabled
Advertise SVI Macip: Disabled
Advertise All VNI flag: Enabled
BUM flooding: Head-end replication
Number of L2 VNIs: 1
Number of L3 VNIs: 1
Flags: * - Kernel
  VNI        Type RD                    Import RT                 Export RT                 Tenant VRF
* 100        L2   10.255.255.1:2        100:100                   100:100                  default
* 2000       L3   192.168.1.1:5         100:2000                  100:2000                 blue

Reporting full configuration and sh ver below.
System config:

set interfaces bridge br100 member interface eth3
set interfaces bridge br100 member interface vxlan100
set interfaces bridge br2000 description 'customer blue'
set interfaces bridge br2000 member interface vxlan2000
set interfaces bridge br2000 vrf 'blue'
set interfaces bridge br3000 description 'customer red'
set interfaces bridge br3000 member interface vxlan3000
set interfaces bridge br3000 vrf 'red'
set interfaces dummy dum0 address '10.255.255.1/32'
set interfaces dummy dum3000 address '172.16.1.1/32'
set interfaces dummy dum3000 description 'customer red'
set interfaces dummy dum3000 vrf 'red'
set interfaces ethernet eth0 address '172.23.131.101/24'
set interfaces ethernet eth0 description 'out-of-band management'
set interfaces ethernet eth0 hw-id '0c:8a:66:c0:f8:00'
set interfaces ethernet eth0 vrf 'management'
set interfaces ethernet eth1 address '10.248.1.0/31'
set interfaces ethernet eth1 description 'link to leaf 2'
set interfaces ethernet eth1 hw-id '0c:8a:66:c0:f8:01'
set interfaces ethernet eth1 mtu '1600'
set interfaces ethernet eth2 address '10.248.2.0/31'
set interfaces ethernet eth2 description 'link to leaf 3'
set interfaces ethernet eth2 hw-id '0c:8a:66:c0:f8:02'
set interfaces ethernet eth2 mtu '1600'
set interfaces ethernet eth3 hw-id '0c:8a:66:c0:f8:03'
set interfaces ethernet eth4 address '192.168.1.1/24'
set interfaces ethernet eth4 description 'customer blue'
set interfaces ethernet eth4 hw-id '0c:8a:66:c0:f8:04'
set interfaces ethernet eth4 vrf 'blue'
set interfaces ethernet eth5 hw-id '0c:8a:66:c0:f8:05'
set interfaces ethernet eth6 hw-id '0c:8a:66:c0:f8:06'
set interfaces ethernet eth7 hw-id '0c:8a:66:c0:f8:07'
set interfaces loopback lo
set interfaces vxlan vxlan100 mtu '1500'
set interfaces vxlan vxlan100 parameters nolearning
set interfaces vxlan vxlan100 port '4789'
set interfaces vxlan vxlan100 source-address '10.255.255.1'
set interfaces vxlan vxlan100 vni '100'
set interfaces vxlan vxlan2000 mtu '1500'
set interfaces vxlan vxlan2000 parameters nolearning
set interfaces vxlan vxlan2000 port '4789'
set interfaces vxlan vxlan2000 source-address '10.255.255.1'
set interfaces vxlan vxlan2000 vni '2000'
set interfaces vxlan vxlan3000 mtu '1500'
set interfaces vxlan vxlan3000 parameters nolearning
set interfaces vxlan vxlan3000 port '4789'
set interfaces vxlan vxlan3000 source-address '10.255.255.1'
set interfaces vxlan vxlan3000 vni '3000'
set protocols bgp address-family l2vpn-evpn advertise ipv4 unicast
set protocols bgp address-family l2vpn-evpn advertise-all-vni
set protocols bgp local-as '100'
set protocols bgp neighbor 10.255.255.2 peer-group 'ibgp'
set protocols bgp neighbor 10.255.255.3 peer-group 'ibgp'
set protocols bgp parameters default
set protocols bgp parameters log-neighbor-changes
set protocols bgp parameters router-id '10.255.255.1'
set protocols bgp peer-group ibgp address-family l2vpn-evpn nexthop-self
set protocols bgp peer-group ibgp remote-as '100'
set protocols bgp peer-group ibgp update-source 'dum0'
set protocols ospf area 0 network '10.248.1.0/31'
set protocols ospf area 0 network '10.248.2.0/31'
set protocols ospf interface eth1 network 'point-to-point'
set protocols ospf interface eth2 network 'point-to-point'
set protocols ospf log-adjacency-changes detail
set protocols ospf parameters abr-type 'cisco'
set protocols ospf parameters router-id '10.255.255.1'
set protocols ospf passive-interface 'default'
set protocols ospf passive-interface-exclude 'eth1'
set protocols ospf passive-interface-exclude 'eth2'
set protocols ospf redistribute connected
set service ssh vrf 'management'
set system config-management commit-revisions '100'
set system conntrack modules ftp
set system conntrack modules h323
set system conntrack modules nfs
set system conntrack modules pptp
set system conntrack modules sip
set system conntrack modules sqlnet
set system conntrack modules tftp
set system console device ttyS0 speed '115200'
set system host-name 'LEAF-1'
set system login user vyos authentication encrypted-password '$6$31IXVLt6kK5bfoyF$QyBZeq5U6zWjCwnnfKb7.2ahoZqnliYjIXJlX..WKYK48vvigOZldZOhPz/iNIJjkNSNAZTOUtt0ZAMx7YMjz0'
set system login user vyos authentication plaintext-password ''
set system name-server '172.17.6.11'
set system ntp listen-address '172.23.131.101'
set system ntp server 172.17.140.222
set system ntp server time1.vyos.net
set system ntp server time2.vyos.net
set system ntp server time3.vyos.net
set system ntp vrf 'management'
set system syslog global facility all level 'info'
set system syslog global facility protocols level 'debug'
set vrf name blue protocols bgp address-family ipv4-unicast redistribute connected
set vrf name blue protocols bgp address-family l2vpn-evpn advertise ipv4 unicast
set vrf name blue protocols bgp local-as '100'
set vrf name blue table '2000'
set vrf name blue vni '2000'
set vrf name management protocols static route 0.0.0.0/0 next-hop 172.23.131.1
set vrf name management table '1000'
set vrf name red protocols bgp address-family ipv4-unicast redistribute connected
set vrf name red protocols bgp address-family l2vpn-evpn advertise ipv4 unicast
set vrf name red protocols bgp local-as '100'
set vrf name red table '3000'
set vrf name red vni '3000'

Full "generated" frr.conf:

frr version 7.5.1-20210619-12-g3f8a74e70
frr defaults traditional
hostname LEAF-1
log syslog
log facility local7
service integrated-vtysh-config
!
vrf blue
 vni 2000
 exit-vrf
!
vrf management
 ip route 0.0.0.0/0 172.23.131.1
 exit-vrf
!
interface eth1
 ip ospf network point-to-point
!
interface eth2
 ip ospf network point-to-point
!
router bgp 100
 bgp router-id 10.255.255.1
 bgp log-neighbor-changes
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 no bgp network import-check
 neighbor ibgp peer-group
 neighbor ibgp remote-as 100
 neighbor ibgp update-source dum0
 neighbor 10.255.255.2 peer-group ibgp
 neighbor 10.255.255.3 peer-group ibgp
 !
 address-family l2vpn evpn
  neighbor ibgp activate
  neighbor ibgp next-hop-self
  advertise-all-vni
  advertise ipv4 unicast
 exit-address-family
!
router bgp 100 vrf blue
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 no bgp network import-check
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!
router bgp 100 vrf red
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 no bgp network import-check
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!
router ospf
 ospf router-id 10.255.255.1
 log-adjacency-changes detail
 auto-cost reference-bandwidth 100
 timers throttle spf 200 1000 10000
 redistribute connected
 passive-interface default
 no passive-interface eth1
 no passive-interface eth2
 network 10.248.1.0/31 area 0
 network 10.248.2.0/31 area 0
!
line vty
!

VERSION:

Version:          VyOS 1.4-rolling-202110240217
Release train:    sagitta

Built by:         [email protected]
Built on:         Sun 24 Oct 2021 02:17 UTC
Build UUID:       08131379-f272-430c-9969-ee0ffcad495a
Build commit ID:  f6c1a927ca63da

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (i440FX + PIIX, 1996)
Hardware S/N:
Hardware UUID:    0ab18ea3-a867-43c3-8fc1-23912194c0f8

Copyright:        VyOS maintainers and contributors

Details

Difficulty level: Normal (likely a few hours)
Version: 1.4-rolling-202110240217
Why the issue appeared?: Design mistake
Is it a breaking change?: Perfectly compatible
Issue type: Bug (incorrect behavior)

Event Timeline

ssasso created this task.Nov 2 2021, 2:14 PM

Additional note: this, of course, breaks the 'red' vrf connectivity.
If I manually add, using vtysh, the

vrf red
 vni 3000

everything on the 'red' vrf works fine.

pasik added a subscriber: pasik.Nov 2 2021, 4:34 PM

Possibly unrelated, but I'll leave it here

frr-reload output:   0 /usr/lib/frr/frr-reload.py:851: SyntaxWarning: "is not" with a literal. Did you mean "!="?
frr-reload output:   1   if line is not "exit-vrf":
frr-reload output:   2 2021-11-02 18:33:34,225  INF

zsdc assigned this task to m.korobeinikov.Nov 3 2021, 11:13 AM

c-po moved this task from Need Triage to Backlog on the VyOS 1.4 Sagitta board.Nov 3 2021, 5:40 PM

If you add configuration to "vtysh" will you see config like this or another:

frr version 7.5.1-20210619-12-g3f8a74e70
frr defaults traditional
hostname LEAF-1
log syslog
log facility local7
service integrated-vtysh-config
!
vrf blue
vni 2000
exit-vrf
!
vrf red
vni 3000
exit-vrf
!

When I manually add the "vrf red" with vtysh, then the frr config is like the one you reported.

Zebra configuration:

root@r11-roll:/home/vyos# cat foo.txt 
!
frr version 7.5.1-20210619-12-g3f8a74e70
frr defaults traditional
hostname r11-roll
log syslog
log facility local7
service integrated-vtysh-config
!
vrf blue
 vni 2000
 exit-vrf
!
vrf red
 vni 3000
 exit-vrf
!
line vty
!
end

DEBUG:

root@r11-roll:/home/vyos# sudo python3 /usr/lib/frr/frr-reload.py --stdout --debug --test --daemon zebra foo.txt 
/usr/lib/frr/frr-reload.py:851: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if line is not "exit-vrf":
2021-11-15 12:08:01,947  INFO: Called via "Namespace(input=None, reload=False, test=True, debug=True, log_level='info', stdout=True, pathspace=None, filename='foo.txt', overwrite=False, bindir='/usr/bin', confdir='/etc/frr', rundir='/var/run/frr', vty_socket=None, daemon='zebra')"
2021-11-15 12:08:01,948  INFO: Loading Config object from file foo.txt
2021-11-15 12:08:02,006 DEBUG: LINE frr version 7.5.1-20210619-12-g3f8a74e70          : entering new context, ['frr version 7.5.1-20210619-12-g3f8a74e70']      
2021-11-15 12:08:02,007 DEBUG: LINE frr defaults traditional                          : entering new context, ['frr defaults traditional']                      
2021-11-15 12:08:02,007 DEBUG: LINE hostname r11-roll                                 : entering new context, ['hostname r11-roll']                             
2021-11-15 12:08:02,007 DEBUG: LINE log syslog                                        : entering new context, ['log syslog']                                    
2021-11-15 12:08:02,008 DEBUG: LINE log facility local7                               : entering new context, ['log facility local7']                           
2021-11-15 12:08:02,008 DEBUG: LINE service integrated-vtysh-config                   : entering new context, ['service integrated-vtysh-config']               
2021-11-15 12:08:02,008 DEBUG: LINE vrf blue                                          : entering new context, ['vrf blue']                                      
2021-11-15 12:08:02,008 DEBUG: LINE vni 2000                                          : append to current_context_lines, ['vrf blue']                                      
2021-11-15 12:08:02,008 DEBUG: LINE exit-vrf                                          : append to current_context_lines, ['vrf blue']                                      
2021-11-15 12:08:02,008 DEBUG: LINE end                                               : exiting old context, []                                                
2021-11-15 12:08:02,008 DEBUG: LINE vrf red                                           : entering new context, ['vrf red']                                       
2021-11-15 12:08:02,008 DEBUG: LINE vni 3000                                          : append to current_context_lines, ['vrf red']                                       
2021-11-15 12:08:02,008 DEBUG: LINE exit-vrf                                          : append to current_context_lines, ['vrf red']                                       
2021-11-15 12:08:02,008 DEBUG: LINE end                                               : exiting old context, []                                                
2021-11-15 12:08:02,008 DEBUG: LINE line vty                                          : entering new context, ['line vty']                                      
2021-11-15 12:08:02,008 DEBUG: LINE end                                               : exiting old context, ['line vty']                                      
2021-11-15 12:08:02,008  INFO: Loading Config object from vtysh show running
2021-11-15 12:08:02,074 DEBUG: LINE frr version 7.5.1-20210619-12-g3f8a74e70          : entering new context, ['frr version 7.5.1-20210619-12-g3f8a74e70']      
2021-11-15 12:08:02,074 DEBUG: LINE frr defaults traditional                          : entering new context, ['frr defaults traditional']                      
2021-11-15 12:08:02,074 DEBUG: LINE hostname r11-roll                                 : entering new context, ['hostname r11-roll']                             
2021-11-15 12:08:02,074 DEBUG: LINE log syslog                                        : entering new context, ['log syslog']                                    
2021-11-15 12:08:02,074 DEBUG: LINE log facility local7                               : entering new context, ['log facility local7']                           
2021-11-15 12:08:02,075 DEBUG: LINE service integrated-vtysh-config                   : entering new context, ['service integrated-vtysh-config']               
2021-11-15 12:08:02,075 DEBUG: LINE line vty                                          : entering new context, ['line vty']                                      
2021-11-15 12:08:02,075 DEBUG: LINE end                                               : exiting old context, ['line vty']                                      

Lines To Add
============
vrf blue
vrf blue
 vni 2000
vrf blue
 exit-vrf
vrf red
vrf red
 vni 3000
root@r11-roll:/home/vyos#

Viacheslav reassigned this task from m.korobeinikov to c-po.Nov 15 2021, 10:29 AM

Viacheslav added a subscriber: m.korobeinikov.

c-po triaged this task as Normal priority.Nov 15 2021, 6:37 PM

c-po changed Difficulty level from Unknown (require assessment) to Normal (likely a few hours).

c-po changed Why the issue appeared? from Will be filled on close to Design mistake.

c-po changed Is it a breaking change? from Unspecified (possibly destroys the router) to Perfectly compatible.

c-po closed this task as Resolved.Nov 15 2021, 7:55 PM

c-po moved this task from Backlog to Finished on the VyOS 1.4 Sagitta board.Nov 24 2021, 7:38 AM

FRR Misconfig when using multiple VRF VNIClosed, ResolvedPublicBUGActions

Description

Details

Event Timeline

FRR Misconfig when using multiple VRF VNI
Closed, ResolvedPublicBUG
Actions