Page MenuHomeVyOS Platform

[VPP] Cannot set additional CPU worker despite isolation: "Not enough free CPUs in the system"
Closed, ResolvedPublicENHANCEMENT

Description

Summary
I am unable to add an additional VPP worker core on an 8-core CPU system, even after applying CPU isolation parameters. The commit fails with the error: "Cannot set VPP "cpu corelist-workers": Not enough free CPUs in the system." This seems like a validation bug, as there should be enough isolated cores available.

Hardware
CPU: Intel(R) Xeon(R) E-2288G @ 3.70GHz, 8 physical cores (no Hyper-Threading, as per /proc/cpuinfo: cpu cores: 8, siblings: 8)
Memory: 16 GB
NIC: Mellanox MT27800 (ConnectX-5)
Full /proc/cpuinfo output attached:


DMI/hardware details: Supermicro X11SCL-F motherboard

set vpp settings cpu main-core '1'
set vpp settings cpu corelist-workers '2'
set vpp settings cpu corelist-workers '3'
set vpp settings cpu corelist-workers '4'
set vpp settings cpu corelist-workers '5'
set vpp settings cpu corelist-workers '6'
# Isolation:
set system option kernel cpu isolate-cpus '1-7'
set system option kernel cpu nohz-full '1-7'
set system option kernel cpu rcu-no-cbs '1-7'

Steps to Reproduce

  1. Set up VPP with 5 workers (cores 2-6) and main-core 1 — commits successfully.
  2. Apply CPU isolation as above, commit, save, reboot.
  3. Try to add worker 7: set vpp settings cpu corelist-workers '7'
  4. Commit — fails with:
[ vpp ]
Cannot set VPP "cpu corelist-workers": Not enough free CPUs in the
system.
[[vpp]] failed
Commit failed

Alternative attempt: Move main-core to 0 and workers to 1-7 — same error.

cat /proc/cmdline 
BOOT_IMAGE=/boot/2025.08.28-0019-rolling/vmlinuz boot=live rootdelay=5 noautologin net.ifnames=0 biosdevname=0 vyos-union=/boot/2025.08.28-0019-rolling mitigations=off intel_idle.max_cstate=0 processor.max_cstate=1 isolcpus=1-7 nohz_full=1-7 rcu_nocbs=1-7 hugepagesz=1G hugepagesz=2M hugepages=3600 console=tty0 intel_iommu=on modules_load=vfio,vfio_iommu_type1,vfio_pci
cat /sys/devices/system/cpu/isolated

Details

Version
2025.08.28-0019-rolling
Is it a breaking change?
Perfectly compatible
Issue type
Bug (incorrect behavior)

Event Timeline

Unknown Object (User) triaged this task as High priority.Sep 4 2025, 6:33 PM

At least 2 CPUs should be for the system processes (Not VPP)
So the error looks handled correctly if you have 8 cores and are trying to use 7 cores.
So, max-number-cores - 2, you can use for VPP even if you isolate 7.

While technically this is not a VPP config validation error, I think something needs improvement.

  1. We should not allow changing parameters in set vpp settings cpu for all CPU cores at the same time, so at least two were always left untouched.
  2. VPP configuration error message needs to be more explicit, describing why there are not enough cores.
  3. I will update documentation with these options to explain the correct config.
natali-rs1985 changed the task status from Open to In progress.Sep 5 2025, 1:19 PM
natali-rs1985 claimed this task.
natali-rs1985 changed Is it a breaking change? from Unspecified (possibly destroys the router) to Perfectly compatible.
natali-rs1985 changed Issue type from Bug (incorrect behavior) to Unspecified (please specify).
natali-rs1985 changed the subtype of this task from "Bug" to "Enhancement".
natali-rs1985 moved this task from Need Triage to Completed on the VyOS Rolling board.
dmbaturin changed Issue type from Unspecified (please specify) to Bug (incorrect behavior).