Page MenuHomeVyOS Platform

vpp: PCI interfaces using the "normal" dpdk driver (ixgbe, i40e, etc.) are never bound to vfio-pci
In progress, HighPublic

Description

Summary

Configuring a "normal" PCI Ethernet NIC (Intel ixgbe, and presumably
i40e/igb/virtio-net/etc. — anything not in the small override_drivers
allow-list) under vpp settings interface <name> reliably fails to commit.
VPP starts, but never creates the DPDK interface, and the conf_mode script
crashes with:

inside lcp_pair_add() -> check_retval_wrapper(), because VPP's own
DPDK/EAL layer silently skips the PCI device at startup rather than binding
it to vfio-pci.

This looks like a regression from the removal of AF_XDP support: `driver
dpdk` + vfio-pci is now the only path for interfaces that aren't in the
hard-coded vendor allow-list (override_drivers in src/conf_mode/vpp.py,
currently just hv_netvsc and ena), but nothing binds the device to
vfio-pci for any other driver.

Environment

  • VyOS rolling (2026.09.09-0029-rolling)
  • NIC: Intel X552/X557-AT 10GBASE-T, ixgbe driver, singleton IOMMU group
  • IOMMU/VFIO modules loaded, hugepages reserved, allow-unsupported-nics set
  • Reproduced inside a privileged Incus/LXC container with the NIC passed through via nictype=physical, but the core bug is in VPP's own conf_mode script and is not specific to that deployment method

Configuration to reproduce

set interfaces ethernet eth0 hw-id '<mac of the NIC>'
set vpp settings allow-unsupported-nics
set vpp settings interface-rx-mode interrupt
set vpp settings interface eth0
commit

Expected behavior

VPP creates a DPDK interface for eth0, LCP-pairs it to a kernel-side
mirror interface, and the commit succeeds.

Actual behavior

Traceback (most recent call last):
File "/usr/libexec/vyos/services/vyos-configd", line 109, in run_script
config_manager.apply(script_name, c)
File "/usr/lib/python3/dist-packages/vyos/configmanager.py", line 136, in apply
mod.apply(config_dict)
File "/usr/libexec/vyos/conf_mode/vpp.py", line 875, in apply
vpp_control.lcp_pair_add(iface, iface)
File "/usr/lib/python3/dist-packages/vyos/vpp/control_vpp.py", line 83, in check_retval_wrapper
if not return_value.retval == 0:
AttributeError: 'NoneType' object has no attribute 'retval'
vpp failed
Commit failed

vppctl show interface shows only local0. VPP's own log explains why:
EAL: VFIO support initialized
EAL: Failed to open VFIO group 27
EAL: 0000:03:00.1 not managed by VFIO driver, skipping

Root cause

lcp_pair_add() calls get_sw_if_index(iface); when VPP never created the
interface this returns 0/None, and lcp_pair_add() has no error branch
for that — it implicitly returns None, and the @_Decorators.check_retval
wrapper then does return_value.retval, producing the AttributeError
above. That part is a secondary symptom (see "Suggested hardening" below);
the real root cause is upstream of it.

VPP's DPDK/EAL layer does not bind a PCI device to vfio-pci itself
when IOMMU is active — vlib_pci_bind_to_uio() expects the device to
already be bound to vfio-pci before VPP starts, and just skips it with a
log line otherwise. In src/conf_mode/vpp.py, apply() only calls
control_host.override_driver() for interfaces whose kernel_module is in
override_drivers (currently hv_netvsc, ena). For every other driver
(ixgbe, i40e, igb, virtio-net, ...) no code path binds the device to
vfio-pci at all before VPP is (re)started.

Fix (tested locally)

In apply(), src/conf_mode/vpp.py, add an elif branch that explicitly
binds the device to vfio-pci via the same control_host.override_driver()
helper already used for the allow-listed case, skipping it if the device is
already on vfio-pci (idempotent re-commits):

diff
                     if iface_config['kernel_module'] in override_drivers:
                         control_host.override_driver(
                             config['persist_config'][iface]['bus_id'],
                             config['persist_config'][iface]['dev_id'],
                             override_drivers[iface_config['kernel_module']],
                         )
+                    elif iface_config['kernel_module'] not in not_pci_drv:
+                        # VPP's own EAL (vlib_pci_bind_to_uio) does not bind
+                        # a PCI device to vfio-pci itself when IOMMU is
+                        # active; it silently skips the device, so no DPDK
+                        # interface is ever created and lcp_pair_add() fails
+                        # with a confusing AttributeError instead of a clear
+                        # error. override_drivers only covers a small
+                        # allow-list (hv_netvsc, ena); "normal" PCI NICs
+                        # (ixgbe, i40e, virtio, ...) going through the
+                        # mandatory dpdk/vfio-pci path need the same
+                        # explicit rebind. Skip if already on vfio-pci.
+                        dev_id = config['persist_config'][iface]['dev_id']
+                        current_drv = None
+                        driver_link = Path(f'/sys/bus/pci/devices/{dev_id}/driver')
+                        if driver_link.exists():
+                            current_drv = driver_link.resolve().name
+                        if current_drv != 'vfio-pci':
+                            control_host.override_driver(
+                                config['persist_config'][iface]['bus_id'],
+                                dev_id,
+                                'vfio-pci',
+                            )

Verified stable across multiple restart cycles with this fix in place: each
restart correctly re-runs apply(), re-binds the NIC to vfio-pci, and VPP
comes up with the interface automatically with no manual intervention.

Suggested additional hardening (not implemented)

lcp_pair_add() in control_vpp.py silently returns None when
get_sw_if_index() is falsy. Raising a clear VPPValueError there (e.g.
"interface <name> does not exist in VPP") would make this whole class of
bug much faster to diagnose in the future instead of surfacing as an
unrelated-looking AttributeError.

Details

Version
2026.09.09-0029-rolling
Is it a breaking change?
Perfectly compatible
Issue type
Bug (incorrect behavior)

Event Timeline

forbushman triaged this task as High priority.
forbushman created this object in space S1 VyOS Public.