Only one NIC interface on dual NIC card via PCI passtrough.

[helloha]

Hi all,

I have a couple of supermicro AOC-SG-I2 dual NIC cards with the intel 82575EB chip.

I managed to create IOMMU groups following this guide: https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/

Doing so the IOMMU error disappeared and I could add the NIC to a specific VM.

In the settings of my virtual machine I can add all 4 interfaces separately, they show up like this.

Card1:
NIC1: 01:00.0
NIC2: 01:00:1

Card2:
NIC1: 02:00.0
NIC2: 02:00.1

However the problem is that in pfsense only one NIC per card shows up when adding all 4 interfaces.

Am I missing something? I'm a newbie to proxmox...

Thanks for any tips!
K.

 

[Dunuin]

Some devices dont allow you to passthrough single functions. Did you try to just passthrough the 2 complete nics instead of 4 individual functions, so just two entries 0000:01:00 and 0000:02:00? You can do that by editing the config file or use that "all functions" (or what it is called) checkbox adding the NIC using the GUI.

 

[helloha]

When I check the box for "pass all functions" i can get 2 interfaces working on one card but the second has only one interface showing up.

Is it possible that the fact they shave the same ID could be an issue? They are the same cards.

00:19.0 0200: 8086:1502 (rev 05) --> built in NIC
01:00.0 0200: 8086:10a7 (rev 02) --> pci card
02:00.0 0200: 8086:10a7 (rev 02) --> pci card

 

[helloha]

When I check the box for "pass all functions" i can get 2 interfaces working on one card but the second has only one interface showing up.

Is it possible that the fact they shave the same ID could be an issue? They are the same cards.

00:19.0 0200: 8086:1502 (rev 05) --> built in NIC
01:00.0 0200: 8086:10a7 (rev 02) --> pci card
02:00.0 0200: 8086:10a7 (rev 02) --> pci card

I did a test with running two VM's simultaneously.

VM1 has: 02:00.0 0200 --> boots, to NICS are visible with "pass all functions"
VM2 has: 01:00.0 0200 --> does not boot and errors -->

-device vfio-pci,host=0000:01:00.0,id=hostpci0.0,bus=pci.0,addr=0x10.0,multifunction=on: vfio 0000:01:00.0: failed to open /dev/vfio/1: Device or resource busy
TASK ERROR: start failed: QEMU exited with code 1

There seems to be some entanglement going on?

 

[Dunuin]

Is it possible that the fact they shave the same ID could be an issue? They are the same cards.

00:19.0 0200: 8086:1502 (rev 05) --> built in NIC
01:00.0 0200: 8086:10a7 (rev 02) --> pci card
02:00.0 0200: 8086:10a7 (rev 02) --> pci card

I think that shoudn't be a problem. Did you try another PCIe slot?

 

[helloha]

I think that shoudn't be a problem. Did you try another PCIe slot?

This seemed to have done the trick, thanks!

 

[bkuhns]

Just wanted to chime in to say this sounded exactly like my problem and for some reason using a different PCI slot fixed the issue. I'm not thrilled about that being the fix, but I'm happy that it's working now! Thanks @Dunuin !