Page MenuHomeVyOS Platform
Create Task
Maniphest T7819

VPP: enable vpp for two network interfaces fails on AWS
Closed, ResolvedPublicBUG
Actions

Description

vyos@VyOS-for-Smoke-Tests# commit
[ vpp ]
FileExistsError: [Errno 17] File exists

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/libexec/vyos/services/vyos-configd", line 156, in run_script
    script.apply(c)
  File "/usr/libexec/vyos/conf_mode/vpp.py", line 613, in apply
    control_host.override_driver(
  File "/usr/lib/python3/dist-packages/vyos/vpp/control_host.py", line 138, in override_driver
    Path('/sys/module/vfio_pci/drivers/pci:vfio-pci/new_id').write_text(
  File "/usr/lib/python3.11/pathlib.py", line 1079, in write_text
    with self.open(mode='w', encoding=encoding, errors=errors, newline=newline) as f:
FileExistsError: [Errno 17] File exists

[[vpp]] failed
Commit failed

Steps to reproduce: 

set vpp settings interface eth0 driver 'dpdk'
commit
set vpp settings interface eth1 driver 'dpdk'
commit

Version:

Version:          VyOS 1.5-rolling-202509131155
Release train:    current
Release flavor:   aws

Built by:         a.kudientsov@vyos.io
Built on:         Sat 13 Sep 2025 11:55 UTC
Build UUID:       3500803b-0bc0-4cc8-b8dd-41f4825496a7
Build commit ID:  1a6747b038ea49

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest
Secure Boot:      n/a (BIOS)

Hardware vendor:  Amazon EC2
Hardware model:   c5.2xlarge
Hardware S/N:     ec2fab01-abc5-3825-d2e9-9a79a1a404c7
Hardware UUID:    ec2fab01-abc5-3825-d2e9-9a79a1a404c7
vyos@VyOS-for-Smoke-Tests:~$ lspci -nn
00:00.0 Host bridge [0600]: Intel Corporation 440FX - 82441FX PMC [Natoma] [8086:1237]
00:01.0 ISA bridge [0601]: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] [8086:7000]
00:01.3 Non-VGA unclassified device [0000]: Intel Corporation 82371AB/EB/MB PIIX4 ACPI [8086:7113] (rev 08)
00:03.0 VGA compatible controller [0300]: Amazon.com, Inc. Device [1d0f:1111]
00:04.0 Non-Volatile memory controller [0108]: Amazon.com, Inc. NVMe EBS Controller [1d0f:8061]
00:05.0 Ethernet controller [0200]: Amazon.com, Inc. Elastic Network Adapter (ENA) [1d0f:ec20]
00:06.0 Ethernet controller [0200]: Amazon.com, Inc. Elastic Network Adapter (ENA) [1d0f:ec20]

vyos@VyOS-for-Smoke-Tests:~$ lspci -vvv -s 00:05.0
00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA)
        Physical Slot: 5
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Region 0: Memory at febf4000 (32-bit, non-prefetchable) [size=16K]
        Region 2: Memory at fe800000 (32-bit, prefetchable) [size=1M]
        Region 4: Memory at febd0000 (32-bit, non-prefetchable) [size=64K]
        Capabilities: <access denied>
        Kernel driver in use: ena
        Kernel modules: ena

vyos@VyOS-for-Smoke-Tests:~$ lspci -vvv -s 00:06.0
00:06.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA)
        Physical Slot: 6
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Region 0: Memory at febf8000 (32-bit, non-prefetchable) [size=16K]
        Region 2: Memory at fe900000 (32-bit, prefetchable) [size=1M]
        Region 4: Memory at febe0000 (32-bit, non-prefetchable) [size=64K]
        Capabilities: <access denied>
        Kernel driver in use: ena
        Kernel modules: ena

journalctl:

Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]: Received message: {"type": "init"}
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]: config session pid is 3335
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]: config session sudo_user is vyos
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]: commit_scripts: ['vpp']
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]: Received message: {"type": "node", "last": true, "data": "/usr/libexec/vyos/conf_mode/vpp.py"}
Sep 14 05:12:52 VyOS-for-Smoke-Tests vpp.py[776]: Module 'vfio_iommu_type1' is alrady loaded
Sep 14 05:12:52 VyOS-for-Smoke-Tests vpp.py[776]: Module 'vfio_pci' is alrady loaded
Sep 14 05:12:52 VyOS-for-Smoke-Tests vpp.py[776]: Module 'vfio_pci_core' is alrady loaded
Sep 14 05:12:52 VyOS-for-Smoke-Tests vpp.py[776]: Module 'vfio' is alrady loaded
Sep 14 05:12:52 VyOS-for-Smoke-Tests netplugd[2703]: eth1: state ACTIVE flags 0x00011043 UP,BROADCAST,RUNNING,MULTICAST,10000 -> 0x00001002 BROADCAST,MULTICAST
Sep 14 05:12:52 VyOS-for-Smoke-Tests dhclient[2840]: receive_packet failed on eth1: Network is down
Sep 14 05:12:52 VyOS-for-Smoke-Tests dhclient[2840]: receive_packet failed on eth1: Network is down
Sep 14 05:12:52 VyOS-for-Smoke-Tests netplugd[3860]: /etc/netplug/netplug eth1 probe -> pid 3860
Sep 14 05:12:52 VyOS-for-Smoke-Tests netplugd[2703]: eth1: can't get flags: No such device
Sep 14 05:12:52 VyOS-for-Smoke-Tests netplugd[2703]: eth1: state PROBING pid 3860 exited status 256
Sep 14 05:12:52 VyOS-for-Smoke-Tests netplugd[2703]: Could not bring eth1 back up
Sep 14 05:12:52 VyOS-for-Smoke-Tests netplugd[2703]: eth1: can't get flags: No such device
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]: FileExistsError: [Errno 17] File exists
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]: During handling of the above exception, another exception occurred:
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]: Traceback (most recent call last):
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]:   File "/usr/libexec/vyos/services/vyos-configd", line 156, in run_script
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]:     script.apply(c)
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]:   File "/usr/libexec/vyos/conf_mode/vpp.py", line 613, in apply
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]:     control_host.override_driver(
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]:   File "/usr/lib/python3/dist-packages/vyos/vpp/control_host.py", line 138, in override_driver
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]:     Path('/sys/module/vfio_pci/drivers/pci:vfio-pci/new_id').write_text(
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]:   File "/usr/lib/python3.11/pathlib.py", line 1079, in write_text
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]:     with self.open(mode='w', encoding=encoding, errors=errors, newline=newline) as f:
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]: FileExistsError: [Errno 17] File exists
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]: Sending reply: ERROR_COMMIT_APPLY with output
Sep 14 05:12:52 VyOS-for-Smoke-Tests vyos-configd[776]: scripts_called: ['vpp']

dmesg:

[  296.321796] VFIO - User Level meta-driver version: 0.3
[  296.384469] vfio-pci 0000:00:05.0: Adding to iommu group 0
[  296.384530] vfio-pci 0000:00:05.0: Adding kernel taint for vfio-noiommu group on device
[  299.045669] Initializing XFRM netlink socket
[  299.389854] vfio-pci 0000:00:05.0: vfio-noiommu device opened by user (vpp:3506)
[  299.724357] tun: Universal TUN/TAP device driver, 1.6
[  336.527579] vfio-pci 0000:00:05.0: Removing from iommu group 0
[  336.558803] pci 0000:00:05.0: [1d0f:ec20] type 00 class 0x020000
[  336.560253] pci 0000:00:05.0: reg 0x10: [mem 0xfebf4000-0xfebf7fff]
[  336.562377] pci 0000:00:05.0: reg 0x18: [mem 0xfe800000-0xfe8fffff pref]
[  336.564427] pci 0000:00:05.0: reg 0x20: [mem 0xfebd0000-0xfebdffff]
[  336.567070] pci 0000:00:05.0: BAR 2: assigned [mem 0xc0000000-0xc00fffff pref]
[  336.568776] pci 0000:00:05.0: BAR 4: assigned [mem 0xc0100000-0xc010ffff]
[  336.568850] pci 0000:00:05.0: BAR 0: assigned [mem 0xc0110000-0xc0113fff]
[  336.570753] ena 0000:00:05.0: ENA device version: 0.10
[  336.570755] ena 0000:00:05.0: ENA controller version: 0.0.1 implementation version 1
[  336.588001] ena 0000:00:05.0: Elastic Network Adapter (ENA) found at mem c0110000, mac addr 06:81:c7:00:56:11
[  336.616853] ena 0000:00:05.0: ENA device version: 0.10
[  336.616857] ena 0000:00:05.0: ENA controller version: 0.0.1 implementation version 1
[  336.620731] ena 0000:00:05.0: Elastic Network Adapter (ENA) found at mem c0110000, mac addr 06:81:c7:00:56:11
[  339.020763] VFIO - User Level meta-driver version: 0.3
[  339.044469] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  339.044523] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[  339.044583] vfio-pci 0000:00:05.0: Adding to iommu group 1
[  339.044656] vfio-pci 0000:00:05.0: Adding kernel taint for vfio-noiommu group on device
[  340.253900] vfio-pci 0000:00:05.0: vfio-noiommu device opened by user (vpp:4329)

Details

Version
1.5-rolling-20250913-1155
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Related Objects
Search...

Event Timeline

a.kudientsov created this task.Sep 14 2025, 5:25 AM
pasik subscribed.Sep 14 2025, 8:16 AM
a.kudientsov added a parent task: T7070: VPP related bugs the root task.Sep 15 2025, 9:20 AM
Unknown Object (User) assigned this task to Viacheslav.Sep 17 2025, 10:55 AM
Unknown Object (User) triaged this task as High priority.
Viacheslav added a subscriber: zsdc.Oct 20 2025, 11:16 AM

Notes from @zsdc

The problem lies in how the driver override logic operates - it’s applied unconditionally to all interfaces in the VPP config with matching kernel drivers. This means:

  • If you add Interface A in one commit,
  • Then add Interface B in the next,

The config script will reattempt the driver override for Interface A - even though it was already processed.
Relevant code: https://github.com/vyos/vyos-1x/blob/69a9b93f71c818bf134ea06ad73d86a513628f06/src/conf_mode/vpp.py#L624-L660

Viacheslav added a comment.Wed, Nov 19, 4:17 PM

PR https://github.com/vyos/vyos-1x/pull/4857

Viacheslav changed the task status from Open to In progress.Mon, Nov 24, 4:58 PM
Viacheslav mentioned this in rVYOSONEX84c4ceebcb44: T7819: VPP do not override driver if it is already done.Wed, Dec 10, 1:13 PM
Restricted Repository Identity mentioned this in rVYOSONEX7d8767de8d9f: Merge pull request #4857 from sever-sever/T7819.Wed, Dec 10, 1:13 PM
Viacheslav closed this task as Resolved.Thu, Dec 11, 3:58 AM
Viacheslav moved this task from Need Triage to Completed on the VyOS Rolling board.