As part of looking at the current kernel build, I noticed that we're using a 250hz kernel.

This appears to be a holdover from basing the kernel on Debian builds, which still use 250hz (almost every other distro uses 1000hz).

Switching to 1000hz gives us much better latency (50-66%), at the expense of slightly more CPU load in a VM.

The other advantage for VyOS is that it will force waiting threads (eg, waiting for RDMA from a NIC) to relinquish much faster, allowing significantly more PPS, and also let XPD/eBPF scripts have more cpu cycles when they're needed, rather than having to block.