On some CPUs with more then x cores offload RPS fails
$ sudo lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 46 bits physical, 57 bits virtual Byte Order: Little Endian CPU(s): 96 On-line CPU(s) list: 0-95 Vendor ID: GenuineIntel ... Thread(s) per core: 2 Core(s) per socket: 24 Socket(s): 2 ... NUMA: NUMA node(s): 2 NUMA node0 CPU(s): 0-23,48-71 NUMA node1 CPU(s): 24-47,72-95
Trying to enable RPS offload:
# set interfaces ethernet eth1 offload rps
[edit]
# commit
[ interfaces ethernet eth1 ]
VyOS had an issue completing a command.
Report time: 2024-11-26 05:04:25
Image version: VyOS 1.4.0
Release train: sagitta
Built by: Sentrium S.L.
Built on: Tue 04 Jun 2024 09:23 UTC
Build UUID: 5e6ae0c4-4d17-4b69-9247-b4ba44a3e3c2
Build commit ID: 35dd8ae6522c78-dirty
Architecture: x86_64
Boot via: installed image
System type: bare metal
Hardware vendor: Supermicro
OSError: [Errno 75] Value too large for defined data type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/libexec/vyos/conf_mode/interfaces_ethernet.py", line 433, in <module>
apply(c)
File "/usr/libexec/vyos/conf_mode/interfaces_ethernet.py", line 410, in apply
e.update(ethernet)
File "/usr/lib/python3/dist-packages/vyos/ifconfig/ethernet.py", line 430, in update
self.set_rps(dict_search('offload.rps', config) != None)
File "/usr/lib/python3/dist-packages/vyos/ifconfig/ethernet.py", line 328, in set_rps
self._write_sysfs(f'/sys/class/net/{self.ifname}/queues/rx-{i}/rps_cpus', f'{rps_cpus:x}')
File "/usr/lib/python3/dist-packages/vyos/ifconfig/control.py", line 130, in _write_sysfs
write_file(filename, str(value))
File "/usr/lib/python3/dist-packages/vyos/utils/file.py", line 69, in write_file
raise e
File "/usr/lib/python3/dist-packages/vyos/utils/file.py", line 61, in write_file
with open(fname, 'w' if not append else 'a') as f:
OSError: [Errno 75] Value too large for defined data type
noteworthy:
cmd 'ethtool --features eth1 lro on'
returned (out):
returned (err):
Could not change any device features
[[interfaces ethernet eth1]] failed
Commit failed
[edit]This calculation formula could be a reason in this case https://github.com/vyos/vyos-1x/blob/2ad3c2860cd19708118d3328106e51605f96c288/python/vyos/ifconfig/ethernet.py#L319
For some number CPUs it could work, for another it fails
Manually adding RPS (not for all cores) seems to be working:
# echo ffff > /sys/class/net/eth0/queues/rx-0/rps_cpus # cat /sys/class/net/eth0/queues/rx-0/rps_cpus 00000000,00000000,0000ffff
But with calculation as in above if fails:
# echo ffffffffffffffffffffffff > /sys/class/net/eth0/queues/rx-0/rps_cpus bash: echo: write error: Value too large for defined data type # cat /sys/class/net/eth0/queues/rx-0/rps_cpus 00000000,00000000,00000000