I just tried installing the OFED drivers. I can't get it to build. I cleaned up all the errors caused by missing packages etc, but now it just says it can't build modules. This seems to be related to the newer kernel tree, but the newer drivers don't support the CX3 cards. I could try overriding that, but if they won't control the card, building the drivers isn't useful.
I did get SR-IOV to work with the in kernel driver on a Threadripper 2950. Just added the module configuration from the OP, and assigned it. The IOMMU gives each VF a group on that machine. The one I would like it to work on, an Epyc, does not. They are all in the same group, which explains the issues I had using passthrough. I'm not sure there is anything I can do about that. The BIOS/UEFI appear to be the latest version available from Supermicro.
I suppose I could buy a CX4 or newer, but I'm not sure it's worth the hassle or if it will behave better with the Epyc system.
Update: This helped get the IOMMU groups sorted out. https://www.supermicro.com/support/faqs/faq.cfm?faq=31883 After adding a couple of settings I was missing, the groups broke up and I was able to use SR-IOV.
I did get SR-IOV to work with the in kernel driver on a Threadripper 2950. Just added the module configuration from the OP, and assigned it. The IOMMU gives each VF a group on that machine. The one I would like it to work on, an Epyc, does not. They are all in the same group, which explains the issues I had using passthrough. I'm not sure there is anything I can do about that. The BIOS/UEFI appear to be the latest version available from Supermicro.
I suppose I could buy a CX4 or newer, but I'm not sure it's worth the hassle or if it will behave better with the Epyc system.
Update: This helped get the IOMMU groups sorted out. https://www.supermicro.com/support/faqs/faq.cfm?faq=31883 After adding a couple of settings I was missing, the groups broke up and I was able to use SR-IOV.
Last edited: