Page MenuHomeVyOS Platform
Create Task
Maniphest T9521

VPP: VRRP rfc3768 (vmac) on VPP interface
Open, NormalPublicBUG
Actions

Description

VRRP rfc3768 on VPP interfaces, VIP is dead.
Keepalived creates a separate vmac interface (macvlan) for the VIP. Everything looks correct in the kernel, show vrrp says healthy, but VPP has no idea the interface or address exist, because there's no way to make VPP aware of it. As a result, VPP drops every ARP request for the VIP 100% loss.
It only mirrors interfaces VPP itself explicitly owns, and by design never scans for whatever an external tool adds (as far as I know).
While this is a specific and rare case, it may not be obvious, as acceptable by configuration.

Steps to reproduce

# both nodes
set vpp settings interface eth1
set interfaces ethernet eth1 address '192.0.2.<1|2>/24'
set high-availability vrrp group VRRP2 interface 'eth1'
set high-availability vrrp group VRRP2 vrid '2'
set high-availability vrrp group VRRP2 priority '<200|100>'
set high-availability vrrp group VRRP2 address '192.0.2.100/24'
set high-availability vrrp group VRRP2 peer-address '<other node's address>'
set high-availability vrrp group VRRP2 hello-source-address '<own address>'
set high-availability vrrp group VRRP2 rfc3768-compatibility
commit
vppctl show interface addr
eth1 (up):
  L3 10.130.2.1/24
  L3 192.0.2.1/24
local0 (dn):
tap4096 (up):
vppctl show ip fib 192.0.2.100/32
ipv4-VRF:0, fib_index:0, flow hash:[src dst sport dport proto flowlabel ] epoch:0 flags:none locks:[adjacency:1, default-route:1, lcp-rt:1, ]
192.0.2.0/24 fib:0 index:13 locks:3
  interface refs:1 entry-flags:connected,attached, src-flags:added,contributing,active, cover:-1
    path-list:[25] locks:2 uPRF-list:17 len:1 itfs:[1, ]
      path:[28] pl-index:25 ip4 weight=1 pref=0 attached:  oper-flags:resolved, cfg-flags:glean,
         eth1

  lcp-rt refs:1 entry-flags:local, src-flags:added,
    path-list:[30] locks:1 flags:local, uPRF-list:24 len:0 itfs:[]
      path:[33] pl-index:30 ip4 weight=1 pref=0 receive:  oper-flags:resolved, cfg-flags:local,
        [@0]: dpo-receive: 0.0.0.0 on local0

 forwarding:   unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:14 buckets:1 uRPF:17 to:[0:0]]
    [0] [@4]: ipv4-glean: [src:192.0.2.0/24] eth1: mtu:2026 next:1 flags:[] ffffffffffff5254000101000806

No specific /32 entry anywhere for 192.0.2.100
VPP does not know the VIP exists

There is a proposal to consider documenting this limitation and also to prohibit such configurations in the CLI.

Details

Version
VyOS 1.5-rolling-202610191432
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

a.pidnebesny created this task.Tue, Oct 27, 7:32 PM
pasik subscribed.Tue, Oct 27, 8:03 PM
rherold subscribed.Tue, Oct 27, 10:08 PM

Hi,
I can fully confirm this analysis. The root cause is a fundamental architectural blind spot in the current integration between the Linux Control Plane (LCP) and VPP.
When Keepalived is configured with rfc3768-compatibility, it dynamically spawns a separate macvlan interface inside the Linux kernel to handle the Virtual MAC (VMAC) and VIP. Because VPP only mirrors interfaces it explicitly owns and initializes, its Netlink listener completely ignores these dynamic sub-interfaces added by external tools. Since VPP acts as the exclusive gatekeeper for the physical hardware interface (eth1), it looks up its local FIB, finds no /32 entry for the VIP, and drops 100% of incoming ARP requests for it before they can ever reach the Linux TAP/Keepalived daemon.
Even workarounds like Keepalived's vmac_xmit_base fail here because they only affect outbound advertisements, leaving inbound ARP resolution completely broken.

Suggested Solution Path

To handle this cleanly without restricting users who want high-performance HA, we should consider a two-phase approach:

  1. Short-Term Mitigation (Quick Fix for 1.5-rolling)

As proposed in the description, we should implement a strict configuration validator in the CLI schema. If an interface is bound to VPP, committing rfc3768-compatibility must throw a validation error. This prevents users from silently breaking their data plane.

  1. Long-Term Feature (Native VPP VRRP Backend Integration)

Instead of forcing all VRRP configurations through Keepalived, VyOS could introduce a configurable VRRP provider backend. This would allow users to choose between the legacy Keepalived daemon and VPP's native, high-performance VRRP plugin.
An idealized CLI structure could look like this:
text

Global or group-specific HA backend selection

set high-availability vrrp provider ['keepalived' | 'vpp']

Scenario A: Legacy Kernel Path (Default)

set high-availability vrrp provider 'keepalived'
set high-availability vrrp group VRRP2 interface 'eth1'
set high-availability vrrp group VRRP2 rfc3768-compatibility # ALLOWED (on non-VPP interfaces)

Scenario B: Native VPP Dataplane Path

set high-availability vrrp provider 'vpp'
set high-availability vrrp group VRRP2 interface 'eth1' # MUST be a VPP interface
set high-availability vrrp group VRRP2 rfc3768-compatibility # ALLOWED (handled natively by VPP plugin)
Verwende Code mit Vorsicht.
Why this makes sense:
• Validation Safety: The vyos-configd engine can easily enforce constraints (e.g., throw an error if provider keepalived + vpp interface + rfc3768-compatibility are combined).
• No Feature Loss: Users relying on complex Keepalived tracking scripts can still use them on standard kernel interfaces.
• True VPP Performance: Users requiring sub-millisecond RFC 3768 failover on VPP interfaces get a native, hardware-fast solution without any LCP synchronization bottlenecks.
I highly support prohibiting the broken configuration right now, but we should definitely open a follow-up feature request to implement VPP's native VRRP plugin as an alternative HA backend.

rherold added a comment.Wed, Oct 28, 9:42 AM

PR for Quickfix: https://github.com/vyos/vyos-1x/pull/5596

Viacheslav triaged this task as Normal priority.Fri, Oct 30, 12:11 PM