Page MenuHomeVyOS Platform

VPP: VRRP rfc3768 (vmac) on VPP interface
In progress, NormalPublicBUG

Description

VRRP rfc3768 on VPP interfaces, VIP is dead.
Keepalived creates a separate vmac interface (macvlan) for the VIP. Everything looks correct in the kernel, show vrrp says healthy, but VPP has no idea the interface or address exist, because there's no way to make VPP aware of it. As a result, VPP drops every ARP request for the VIP 100% loss.
It only mirrors interfaces VPP itself explicitly owns, and by design never scans for whatever an external tool adds (as far as I know).
While this is a specific and rare case, it may not be obvious, as acceptable by configuration.

Steps to reproduce

# both nodes
set vpp settings interface eth1
set interfaces ethernet eth1 address '192.0.2.<1|2>/24'
set high-availability vrrp group VRRP2 interface 'eth1'
set high-availability vrrp group VRRP2 vrid '2'
set high-availability vrrp group VRRP2 priority '<200|100>'
set high-availability vrrp group VRRP2 address '192.0.2.100/24'
set high-availability vrrp group VRRP2 peer-address '<other node's address>'
set high-availability vrrp group VRRP2 hello-source-address '<own address>'
set high-availability vrrp group VRRP2 rfc3768-compatibility
commit
vppctl show interface addr
eth1 (up):
  L3 10.130.2.1/24
  L3 192.0.2.1/24
local0 (dn):
tap4096 (up):
vppctl show ip fib 192.0.2.100/32
ipv4-VRF:0, fib_index:0, flow hash:[src dst sport dport proto flowlabel ] epoch:0 flags:none locks:[adjacency:1, default-route:1, lcp-rt:1, ]
192.0.2.0/24 fib:0 index:13 locks:3
  interface refs:1 entry-flags:connected,attached, src-flags:added,contributing,active, cover:-1
    path-list:[25] locks:2 uPRF-list:17 len:1 itfs:[1, ]
      path:[28] pl-index:25 ip4 weight=1 pref=0 attached:  oper-flags:resolved, cfg-flags:glean,
         eth1

  lcp-rt refs:1 entry-flags:local, src-flags:added,
    path-list:[30] locks:1 flags:local, uPRF-list:24 len:0 itfs:[]
      path:[33] pl-index:30 ip4 weight=1 pref=0 receive:  oper-flags:resolved, cfg-flags:local,
        [@0]: dpo-receive: 0.0.0.0 on local0

 forwarding:   unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:14 buckets:1 uRPF:17 to:[0:0]]
    [0] [@4]: ipv4-glean: [src:192.0.2.0/24] eth1: mtu:2026 next:1 flags:[] ffffffffffff5254000101000806

No specific /32 entry anywhere for 192.0.2.100
VPP does not know the VIP exists

There is a proposal to consider documenting this limitation and also to prohibit such configurations in the CLI.

Details

Version
VyOS 1.5-rolling-202610191432
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

Hi,
I can fully confirm this analysis. The root cause is a fundamental architectural blind spot in the current integration between the Linux Control Plane (LCP) and VPP.
When Keepalived is configured with rfc3768-compatibility, it dynamically spawns a separate macvlan interface inside the Linux kernel to handle the Virtual MAC (VMAC) and VIP. Because VPP only mirrors interfaces it explicitly owns and initializes, its Netlink listener completely ignores these dynamic sub-interfaces added by external tools. Since VPP acts as the exclusive gatekeeper for the physical hardware interface (eth1), it looks up its local FIB, finds no /32 entry for the VIP, and drops 100% of incoming ARP requests for it before they can ever reach the Linux TAP/Keepalived daemon.
Even workarounds like Keepalived's vmac_xmit_base fail here because they only affect outbound advertisements, leaving inbound ARP resolution completely broken.

Suggested Solution Path

To handle this cleanly without restricting users who want high-performance HA, we should consider a two-phase approach:

  1. Short-Term Mitigation (Quick Fix for 1.5-rolling)

As proposed in the description, we should implement a strict configuration validator in the CLI schema. If an interface is bound to VPP, committing rfc3768-compatibility must throw a validation error. This prevents users from silently breaking their data plane.

  1. Long-Term Feature (Native VPP VRRP Backend Integration)

Instead of forcing all VRRP configurations through Keepalived, VyOS could introduce a configurable VRRP provider backend. This would allow users to choose between the legacy Keepalived daemon and VPP's native, high-performance VRRP plugin.
An idealized CLI structure could look like this:
text

Global or group-specific HA backend selection

set high-availability vrrp provider ['keepalived' | 'vpp']

Scenario A: Legacy Kernel Path (Default)

set high-availability vrrp provider 'keepalived'
set high-availability vrrp group VRRP2 interface 'eth1'
set high-availability vrrp group VRRP2 rfc3768-compatibility # ALLOWED (on non-VPP interfaces)

Scenario B: Native VPP Dataplane Path

set high-availability vrrp provider 'vpp'
set high-availability vrrp group VRRP2 interface 'eth1' # MUST be a VPP interface
set high-availability vrrp group VRRP2 rfc3768-compatibility # ALLOWED (handled natively by VPP plugin)
Verwende Code mit Vorsicht.
Why this makes sense:
• Validation Safety: The vyos-configd engine can easily enforce constraints (e.g., throw an error if provider keepalived + vpp interface + rfc3768-compatibility are combined).
• No Feature Loss: Users relying on complex Keepalived tracking scripts can still use them on standard kernel interfaces.
• True VPP Performance: Users requiring sub-millisecond RFC 3768 failover on VPP interfaces get a native, hardware-fast solution without any LCP synchronization bottlenecks.
I highly support prohibiting the broken configuration right now, but we should definitely open a follow-up feature request to implement VPP's native VRRP plugin as an alternative HA backend.

Viacheslav changed the task status from Open to In progress.Tue, Nov 3, 4:00 PM
Viacheslav assigned this task to rherold.

The change is up as a pull request and ready for review: https://github.com/vyos/vyos-1x/pull/5596

It is the short-term step until VPP has its own VRRP support. With rfc3768-compatibility keepalived puts the virtual IP address on a macvlan interface with the virtual MAC address. VPP does not know that interface, so the FIB has no route for the address and it is not reachable through VPP. The option is now refused for a VRRP group whose interface is handled by VPP (vpp settings interface, an interfaces vpp interface such as a bonding, or a VLAN of those), in the VRRP verification, the VPP verification and the verification of each interfaces vpp type, so the order of the commits does not get around it. Disabled groups are ignored.

A saved configuration with the option on such an interface would not load after the upgrade (the boot commit fails and VPP does not start, measured on the lab router). A config migration (vrrp 4 to 5) removes the option from those groups, the virtual IP address is reachable afterwards.

Checked on a current rolling image: test_vpp.py 49 of 49 including three new tests (group on a VPP interface, bonding, a VPP interface created after the group), the VRRP smoketest, a migration unit test and a config test. The negative controls fail as expected. The CodeRabbit findings are fixed. The pull request is waiting for the workflow approval.