SR-IOV and BSD (PFSense, TrueNAS Core) reset_hw() failure: Reset Failed!

[aarcane]

So I'm trying to evaluate SR-IOV on a test server (Dell R620, dual E5-2670v2, SR-IOV, VT-D, etc all enabled) and everything works on a Linux guest, but I'm getting the following error whenever I attempt to boot up a BSD appliance guest.

The NIC is an Intel X520 based Dell rNDC, and AFAIK is fully functional and supported by SR-IOV and Proxmox.

Is this a known issue in Proxmox? A known issue with BSD? A configuration mistake I've made somewhere? Please advise.

The following is the error I get on both PFSense and TrueNAS Core:

ixv0: <Intel(R) X520 82599 Virtual Function> mem 0x1000004000-0x1000007fff,0x1000000000-0x1000003fff at device 0.0 on pci1

ixv0: ...reset_hw() failure: Reset Failed!

ixv0: IFDI_ATTACH_PRE failed 5

device_attach: ixv0 attach returned 5

ixv0: <Intel(R) X520 82599 Virtual Function> mem 0x1000004000-0x1000007fff,0x1000000000-0x1000003fff at device 0.0 on pci1

ixv0: ...reset_hw() failure: Reset Failed!

ixv0: IFDI_ATTACH_PRE failed 5

device_attach: ixv0 attach returned 5

 

[dcsapak]

probably an issue with the guest driver? does the driver in bsd support the virtual functions at all ?

 

[ITT]

For BSD disable/uncheck "PCI-Express".

 

[aarcane]

probably an issue with the guest driver? does the driver in bsd support the virtual functions at all ?

Yes. ix0 is a pf, ixv0 is a vf.

 

[aarcane]

For BSD disable/uncheck "PCI-Express".

I unchecked PCI-Express, and I even tried changing the machine type back to i440fx, and the error message is the exact same error message. No change, no success.

 

[dcsapak]

mhmm i found: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211062
which seems to indicate that its indeed a driver bug?

 

[aarcane]

mhmm i found: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211062
which seems to indicate that its indeed a driver bug?

I found that from way back in 2016, I'd be very surprised if it isn't already patched in TrueNAS Core, pfSense, and Upstream FreeBSD, especially since the pfSense download I have is from 2022-02-14.

 

[dcsapak]

the bug is from 2016 yes, but i can see comments that it's not working properly still from 2021 and freenas 13
maybe ask the freebsb folks if they know if it's fixed (or should be fixed)

AFAICS this is not related to PVE/virtualization but has more to do with the driver in the guest