Any problem users encounter with ConnectX hardware brought to the vendors attention immediately results in being told to install the vendor's drivers as upstream Linux is unsupported. OFED drivers+userspace provide a variety of offloads, firmware facilities, and configuration mechanisms not available on upstream Linux' implementation for the hardware.
Propose adding OFED drivers and userspace packages into the build system as it would allow VyOS to handle far more traffic/complexity via HW offload to modern hardware, users to get vendor support for problems with said hardware, and new functionality like functioning as an Infiniband gateway.