Page MenuHomeVyOS Platform

SR-IOV not working on Hyper-V with Mellanox ConnectX-5
Open, NormalPublicBUG

Description

The virtual device does not appear to be detected.

Both VMs have a network adapter with SR-IOV enabled.

VyOS:

vyos@VyOS:~$ sudo su
root@VyOS:~# lspci
root@VyOS:~# lsmod | egrep "(hv|hyper)"
hv_balloon             40960  0
hv_utils               49152  2
hv_storvsc             28672  1
scsi_transport_fc     102400  1 hv_storvsc
hv_netvsc             114688  0
scsi_mod              303104  6 sd_mod,hv_storvsc,scsi_transport_fc,usb_storage,sg,sr_mod
hyperv_keyboard        12288  0
scsi_common            16384  6 scsi_mod,sd_mod,hv_storvsc,usb_storage,sg,sr_mod
hv_vmbus              172032  5 hv_balloon,hv_utils,hv_netvsc,hv_storvsc,hyperv_keyboard

Fresh Debian install:

root@Debian:~# lspci
4caf:00:02.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function] (rev 80)
root@Debian:~# lsmod | egrep "(hv|hyper)"
pci_hyperv             40960  0
pci_hyperv_intf        16384  2 pci_hyperv,mlx5_core
hid_hyperv             16384  0
hv_netvsc             106496  0
hid                   155648  2 hid_hyperv,hid_generic
hv_utils               49152  2
hv_balloon             40960  0
hv_storvsc             24576  3
scsi_transport_fc      90112  1 hv_storvsc
scsi_mod              274432  4 sd_mod,hv_storvsc,scsi_transport_fc,sg
hyperv_keyboard        16384  0
hyperv_fb              24576  1
scsi_common            16384  3 scsi_mod,hv_storvsc,sg
hv_vmbus              151552  8 hv_balloon,hv_utils,hv_netvsc,hid_hyperv,hv_storvsc,hyperv_keyboard,pci_hyperv,hyperv_fb
}

Details

Difficulty level
Unknown (require assessment)
Version
1.5-rolling-202409230006
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Unspecified (please specify)

Event Timeline

syncer triaged this task as Normal priority.Fri, Sep 27, 6:25 PM
syncer added subscribers: c-po, syncer.

@c-po, do we miss Mellanox drivers in 1,5 ?

We should have two variants - Kernel build in and OFED driver

[email protected]:~$ find /lib/modules -name *mlx5_core*
/lib/modules/6.6.52-vyos/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko
/lib/modules/6.6.52-vyos/updates/dkms/mlx5_core.ko

@ctd can you check that you also have the drivers?

c-po added a project: Restricted Project.Sun, Sep 29, 2:32 PM
vyos@VyOS:~$ find /lib/modules -name *mlx5_core*
/lib/modules/6.6.52-vyos/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko
/lib/modules/6.6.52-vyos/updates/dkms/mlx5_core.ko

Drivers seem to be present, but device is not even appearing in lspci. Wouldn't this be more likely due to a hyper-v driver issue than the mellanox driver?