[SOLVED] Isolate pci device from host without blacklisting driver

[danioj]

Hi,

TLDR: Could someone please tell me how I isolate an Intel 2 port NIC so the VE host doesn’t recognise it but so I can still pass through to guest - without blacklisting the driver?

Long term lurker first time poster. Been dabbling over the past few days with vt-g and was very happy when I got a W10 VM running BlueIris (my NVR of choice) running with my quick sync capable Intel chip.

I then decided to dabble in virtualising my Pfsense installation. Followed the Netgate tutorial:

https://docs.netgate.com/pfsense/en/latest/recipes/virtualize-proxmox-ve.html

I had it running. Then i noticed that it wasn’t accepting inbound traffic (yes I disabled hardware checksums) - web server, Plex etc.

Same config as my bare metal (except that I had to reassign interfaces for the VM so I know the config works). I could only imagine that there is something in the bridging setup with Proxmox impacting. Appears very similar to this issue:

https://www.reddit.com/r/PFSENSE/comments/i08awd/pfsense_on_proxmox_doesnt_appear_to_work_correctly/

Unfortunately the dude didn’t post the solution as it just “started working”.

So I thought I would just pass the whole 2 port NIC to the VM. Easy right? Wrong. For some reason when I start the VM, at the Proxmox command prompt I can see errors waiting for the device to become available - alas it never does (after many retires) and the VM fails to start. I know I could blacklist the driver but my MB has an Intel port too (that I use for Proxmox admin access) and I don’t want to loose it.

So - I’m down to - how do I stop the NiC getting recognised and used by the Host so I can pass it to the Guest without disabling the driver?

:-(

 

[dcsapak]

you could bind them to the 'vfio-pci' driver via /etc/modprobe.d see for example https://pve.proxmox.com/wiki/PCI(e)_Passthrough#_host_device_passthrough
sometimes it is necessary to define a dependecy (so that it gets called before the real driver loads) with the 'softdep' command (see 'man modprobe.d' for more details on how to do that)

 

[danioj]

you could bind them to the 'vfio-pci' driver via /etc/modprobe.d see for example https://pve.proxmox.com/wiki/PCI(e)_Passthrough#_host_device_passthrough
sometimes it is necessary to define a dependecy (so that it gets called before the real driver loads) with the 'softdep' command (see 'man modprobe.d' for more details on how to do that)

Thank you. Your response was (or should I say - should have been) the answer.

That said, for some reason I could not get vfio-pci to bind the Intel card. I then found the following post and saw that I wasn’t alone:

https://forum.proxmox.com/threads/pcie-passthrough-not-working-at-least-not-automatically.63931/

I managed to change the driver manually (as in the above post) but that still didn’t resolve the problem on VM start. So I reverted back to using bridges and decided to just troubleshoot the recommended method. Had to really as I just couldn’t pass the device through to the VM.

I decided to follow the process identified by a user with a similar issue i identified in the original post.

https://www.reddit.com/r/PFSENSE/comments/i08awd/pfsense_on_proxmox_doesnt_appear_to_work_correctly/

Set the network bridges to Intel first (even though I know performance is bad and not recommended), then once setup (and obviously not working) change to virtio and reboot.

For some VERY unknown reason (similar to the frustrating response in the above post), this worked. I mean, just starting with virtio should just work - but it doesn’t.

I replicated the issue too. It happens each time. Weird.

Anyway - Incoming traffic is coming through and my config is working just as it was bare metal.