Page MenuHomeVyOS Platform
Create Task
Maniphest T9095

EVPN Multihoming: ARP synchronization is not working
Open, NormalPublicBUG
Actions

Description

ARP synchronization is not working between VTEPs in the same ES.

Topology:

L2Multihoming.jpg (955×662 px, 137 KB)

Underlay: OSPF. The main function is to distribute Loopback IPs.
Overlay:
Control Plane: MP-iBGP. The main function is to distribute EVPN routes.
Data Plane: VXLAN
RR is a router reflector.
VTEP-1A and VTEP-1B contain the same ES via interface eth1
HOST-1 and HOST-2 are in different networks. 192.168.101.11/24 and 192.168.201.20/24 respectively.
Topology represents the Symmetric IRB model.
All VTEPS use the Anycast Gateway feature.

Configurations:

HOST-1

set interfaces bonding bond1 address '192.168.101.11/24'
set interfaces bonding bond1 member interface 'eth0'
set interfaces bonding bond1 member interface 'eth1'

HOST-2

set interfaces ethernet eth0 address '192.168.201.20/24'

VTEP-1A

set interfaces bonding bond0 evpn es-id '111'
set interfaces bonding bond0 evpn es-sys-mac '00:00:00:00:00:01'
set interfaces bonding bond0 member interface 'eth1'
set interfaces bonding bond0 system-mac '00:00:00:00:00:01'
set interfaces bridge br0 enable-vlan
set interfaces bridge br0 member interface bond0 allowed-vlan '100'
set interfaces bridge br0 member interface bond0 native-vlan '100'
set interfaces bridge br0 member interface vxlan1 disable-learning
set interfaces bridge br0 vif 100 vrf 'RED'
set interfaces bridge br0 vif 1000 vrf 'RED'
set interfaces dummy dum0 address '1.1.1.1/32'
set interfaces ethernet eth0 address '10.0.0.1/24'
set interfaces pseudo-ethernet peth100 address '192.168.101.1/24'
set interfaces pseudo-ethernet peth100 anycast-gateway
set interfaces pseudo-ethernet peth100 ip enable-arp-accept
set interfaces pseudo-ethernet peth100 mac '00:aa:aa:aa:aa:aa'
set interfaces pseudo-ethernet peth100 source-interface 'br0.100'
set interfaces pseudo-ethernet peth100 vrf 'RED'
set interfaces vxlan vxlan1 parameters external
set interfaces vxlan vxlan1 parameters nolearning
set interfaces vxlan vxlan1 parameters vni-filter
set interfaces vxlan vxlan1 source-address '1.1.1.1'
set interfaces vxlan vxlan1 vlan-to-vni 100 vni '10100'
set interfaces vxlan vxlan1 vlan-to-vni 1000 vni '1000'
set protocols bgp address-family l2vpn-evpn advertise-all-vni
set protocols bgp address-family l2vpn-evpn vni 10100 rd '1.1.1.1:10100'
set protocols bgp address-family l2vpn-evpn vni 10100 route-target both '65000:10100'
set protocols bgp neighbor 10.10.10.10 address-family l2vpn-evpn nexthop-self
set protocols bgp neighbor 10.10.10.10 remote-as '65000'
set protocols bgp neighbor 10.10.10.10 update-source 'dum0'
set protocols bgp parameters router-id '1.1.1.1'
set protocols bgp system-as '65000'
set protocols ospf interface dum0 area '0'
set protocols ospf interface eth0 area '0'
set protocols ospf parameters router-id '1.1.1.1'
set system host-name 'VTEP-1A'
set vrf name RED protocols bgp address-family ipv4-unicast redistribute connected
set vrf name RED protocols bgp address-family l2vpn-evpn advertise ipv4 unicast
set vrf name RED protocols bgp address-family l2vpn-evpn rd '1.1.1.1:1000'
set vrf name RED protocols bgp address-family l2vpn-evpn route-target both '65000:1000'
set vrf name RED protocols bgp system-as '65000'
set vrf name RED table '111'

VTEP-1B

set interfaces bonding bond0 evpn es-id '111'
set interfaces bonding bond0 evpn es-sys-mac '00:00:00:00:00:01'
set interfaces bonding bond0 member interface 'eth1'
set interfaces bonding bond0 system-mac '00:00:00:00:00:01'
set interfaces bridge br0 enable-vlan
set interfaces bridge br0 member interface bond0 allowed-vlan '100'
set interfaces bridge br0 member interface bond0 native-vlan '100'
set interfaces bridge br0 member interface vxlan1 disable-learning
set interfaces bridge br0 vif 100 vrf 'RED'
set interfaces bridge br0 vif 1000 vrf 'RED'
set interfaces dummy dum0 address '2.2.2.2/32'
set interfaces ethernet eth0 address '10.0.0.2/24'
set interfaces pseudo-ethernet peth100 address '192.168.101.1/24'
set interfaces pseudo-ethernet peth100 anycast-gateway
set interfaces pseudo-ethernet peth100 ip enable-arp-accept
set interfaces pseudo-ethernet peth100 mac '00:aa:aa:aa:aa:aa'
set interfaces pseudo-ethernet peth100 source-interface 'br0.100'
set interfaces pseudo-ethernet peth100 vrf 'RED'
set interfaces vxlan vxlan1 parameters external
set interfaces vxlan vxlan1 parameters nolearning
set interfaces vxlan vxlan1 parameters vni-filter
set interfaces vxlan vxlan1 source-address '2.2.2.2'
set interfaces vxlan vxlan1 vlan-to-vni 100 vni '10100'
set interfaces vxlan vxlan1 vlan-to-vni 1000 vni '1000'
set protocols bgp address-family l2vpn-evpn advertise-all-vni
set protocols bgp address-family l2vpn-evpn vni 10100 rd '2.2.2.2:10100'
set protocols bgp address-family l2vpn-evpn vni 10100 route-target both '65000:10100'
set protocols bgp neighbor 10.10.10.10 address-family l2vpn-evpn nexthop-self
set protocols bgp neighbor 10.10.10.10 remote-as '65000'
set protocols bgp neighbor 10.10.10.10 update-source 'dum0'
set protocols bgp parameters router-id '2.2.2.2'
set protocols bgp system-as '65000'
set protocols ospf interface dum0 area '0'
set protocols ospf interface eth0 area '0'
set protocols ospf parameters router-id '2.2.2.2'
set system host-name 'VTEP-1B'
set vrf name RED protocols bgp address-family ipv4-unicast redistribute connected
set vrf name RED protocols bgp address-family l2vpn-evpn advertise ipv4 unicast
set vrf name RED protocols bgp address-family l2vpn-evpn rd '2.2.2.2:1000'
set vrf name RED protocols bgp address-family l2vpn-evpn route-target both '65000:1000'
set vrf name RED protocols bgp system-as '65000'
set vrf name RED table '111'
set vrf name RED vni '1000'

VTEP-2

set interfaces bridge br0 enable-vlan
set interfaces bridge br0 member interface eth1 allowed-vlan '200'
set interfaces bridge br0 member interface eth1 native-vlan '200'
set interfaces bridge br0 member interface vxlan1 disable-learning
set interfaces bridge br0 vif 200 vrf 'RED'
set interfaces bridge br0 vif 1000 vrf 'RED'
set interfaces dummy dum0 address '3.3.3.3/32'
set interfaces ethernet eth0 address '10.0.0.3/24'
set interfaces pseudo-ethernet peth200 address '192.168.201.1/24'
set interfaces pseudo-ethernet peth200 anycast-gateway
set interfaces pseudo-ethernet peth200 ip enable-arp-accept
set interfaces pseudo-ethernet peth200 mac '00:aa:aa:aa:aa:aa'
set interfaces pseudo-ethernet peth200 source-interface 'br0.200'
set interfaces pseudo-ethernet peth200 vrf 'RED'
set interfaces vxlan vxlan1 parameters external
set interfaces vxlan vxlan1 parameters nolearning
set interfaces vxlan vxlan1 parameters vni-filter
set interfaces vxlan vxlan1 source-address '3.3.3.3'
set interfaces vxlan vxlan1 vlan-to-vni 200 vni '10200'
set interfaces vxlan vxlan1 vlan-to-vni 1000 vni '1000'
set protocols bgp address-family l2vpn-evpn advertise-all-vni
set protocols bgp address-family l2vpn-evpn vni 200 rd '3.3.3.3:10200'
set protocols bgp address-family l2vpn-evpn vni 200 route-target both '65000:10200'
set protocols bgp neighbor 10.10.10.10 address-family l2vpn-evpn nexthop-self
set protocols bgp neighbor 10.10.10.10 remote-as '65000'
set protocols bgp neighbor 10.10.10.10 update-source 'dum0'
set protocols bgp parameters router-id '3.3.3.3'
set protocols bgp system-as '65000'
set protocols ospf interface dum0 area '0'
set protocols ospf interface eth0 area '0'
set protocols ospf parameters router-id '3.3.3.3'
set system host-name 'VTEP-2'
set vrf name RED protocols bgp address-family ipv4-unicast maximum-paths ibgp '4'
set vrf name RED protocols bgp address-family ipv4-unicast redistribute connected
set vrf name RED protocols bgp address-family l2vpn-evpn advertise ipv4 unicast
set vrf name RED protocols bgp address-family l2vpn-evpn rd '3.3.3.3:1000'
set vrf name RED protocols bgp address-family l2vpn-evpn route-target both '65000:1000'
set vrf name RED protocols bgp system-as '65000'
set vrf name RED table '111'
set vrf name RED vni '1000'

RR

set interfaces dummy dum1 address '10.10.10.10/32'
set interfaces ethernet eth0 address '10.0.0.10/24'
set protocols bgp neighbor 1.1.1.1 peer-group 'VTEP'
set protocols bgp neighbor 2.2.2.2 peer-group 'VTEP'
set protocols bgp neighbor 3.3.3.3 peer-group 'VTEP'
set protocols bgp parameters router-id '10.10.10.10'
set protocols bgp peer-group VTEP address-family l2vpn-evpn route-reflector-client
set protocols bgp peer-group VTEP remote-as '65000'
set protocols bgp peer-group VTEP update-source 'dum1'
set protocols bgp system-as '65000'
set protocols ospf parameters router-id '10.10.10.10'
set protocols ospf interface dum1 area '0'
set protocols ospf interface eth0 area '0'
set system host-name 'RR'

The output of the ARP tables
VTEP-1A

vyos@vyos:~$ show arp
Address          Interface    Link layer address    State
---------------  -----------  --------------------  ---------
10.0.0.2         eth0         0c:20:48:6e:00:00     STALE
3.3.3.3          vxlan1       da:03:64:6b:13:93     NOARP
10.0.0.10        eth0         0c:fa:6a:21:13:dd     DELAY
10.0.0.3         eth0         0c:4d:79:5e:00:00     REACHABLE
3.3.3.3          br0.1000     da:03:64:6b:13:93     NOARP
192.168.101.11   peth100      0c:d0:31:18:00:00     REACHABLE

VTEP-1B

vyos@vyos:~$ show arp
Address          Interface    Link layer address    State
---------------  -----------  --------------------  -------
10.0.0.10        eth0         0c:fa:6a:21:13:dd     STALE
10.0.0.1         eth0         0c:24:a5:f7:00:00     STALE
3.3.3.3          vxlan1       da:03:64:6b:13:93     NOARP
10.0.0.3         eth0         0c:4d:79:5e:00:00     STALE
192.168.101.11   peth100                            FAILED
3.3.3.3          br0.1000     da:03:64:6b:13:93     NOARP

As we see VTEP-1B does not know about 192.168.101.11.
It can happend in the following situation.
The next steps are not fully describe the whole packet flow but they contain main idea.
Steps:

  1. HOST-2 send ICMP to HOST1.
  2. The traffic goes through VTEP-2
  3. VTEP-2 select the route via VTEP-1B.
  4. VTEP-1B sends ARP request to HOST-1.
  5. HOST-1 replies but via link to VTEP-1A.

5, VTEP-1A receive ARP reply and add ARP entry to its ARP table.

  1. VTEP-1A generate BGP NLRI EVPN route type-2 with MAC/IP of 192.168.101.11 and sends it to all VTEPS.
  2. VTEP-1B receives this NLRI and generates for this MAC/IP BGP NLRI EVPN route type-2 with proxy bit and sends it to all VTEPS.
  3. VTEP-1B does not add this MAC/IP to its ARP table and... ICMP traffic is blocked. (THIS IS NOT CORRECT)

Details

Version
VyOS 2026.07.11-0033-rolling
Is it a breaking change?
Perfectly compatible
Issue type
Bug (incorrect behavior)

Event Timeline

a.apostoliuk created this task.Jul 16 2026, 10:54 AM
a.apostoliuk triaged this task as Normal priority.
pasik subscribed.Jul 17 2026, 9:25 AM