The Mellanox/NVIDIA out-of-tree NIC drivers are built from MLNX_OFED 24.07 (MLNX_OFED_SRC-debian-24.07-0.6.1.0). NVIDIA has declared MLNX_OFED end-of-life and replaced it with DOCA-OFED, which ships as part of DOCA-Host.
NOTE: This driver is not part of the rolling ISO image, and this task does not change that. VyOS only provides the build system for it, so the packages can be built from source. They will not ship in the image until the license allows redistribution.
MLNX_OFED 24.07 does not build against current VyOS kernels:
- Since Linux 6.13, kbuild changes into the external module's directory for M= builds. MLNX_OFED 24.07 passes relative include paths, so its configure step fails with asm/types.h: No such file or directory / Kernel modules cannot be built.
- The build script ignored errors, so this went unnoticed until out-of-tree module builds were made to abort on the first error (T9577).
Other problems with the existing build:
- It required root, because NVIDIA's install.pl installs and removes packages on the build host. It left root-owned files in the source tree and changed the build container's installed packages.
- The modules package shipped OFED build sources (headers, compat sources, Module.symvers) under /usr/src/ofa_kernel, registered as an update-alternatives entry. VyOS does not support DKMS or compiling modules on the target system.
- After the build signed and xz-compressed the modules, the package's md5sums still listed the uncompressed .ko files, so dpkg --verify reported mismatches.
The choice was to upgrade to DOCA-OFED or drop the OFED build from the build system entirely. This task upgrades.
Solution
- Switch to DOCA-OFED 26.07 (MLNX_OFED_SRC-debian-26.07-0.7.7.0) from the DOCA 3.5.0 release on NVIDIA's public repository.
- Patch DOCA-OFED for Linux 6.18.y, whose stable backports changed in-kernel APIs in ways OFED does not expect:
- 6.18.y has the new queue-config hook ndo_default_qcfg but not its validation hook ndo_validate_qcfg. OFED assumes the two arrive together, as in Linux 7.0. A separate feature check now enables the validation hook only when the kernel has it.
- 6.18.y backported upstream a6e4b9a6deb9 ("xsk: honor XDP_TX_METADATA in zero-copy path"), which adds an options argument to xsk_buff_get_metadata(). A new feature check makes mlx5 pass the descriptor options, matching the in-tree driver.
- Drop the DKMS package's dh-exec install script. dh_install evaluates it even though the DKMS package is never built, and dh-exec is not a build dependency for that build profile.