PCI Passthrough

[spetrillo]

Ok for those of you that have seen my prior posts I am still trying to work out the right config for PCI passthrough on my Lenovo M720q Tiny. I love this device but getting the right config to work is causing me major stomach acid! I am running ZFS as my filesystem, so I have SystemD handling the config.

As of right now I have the following configured on my device:

CMDLINE:
root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt pci_pt_e820_access=on pci=assign-busses pcie_acs_override=downstream,multifunction i915.enable_gvt=1
MODULES:
vfio
vfio_iommu_type1
vfio_pci
kvmgt

I can see my Intel I350 4 port network card and iGPU have a separate IOMMU group for each interface:

0x020000 │ 0x1521 │ 0000:01:00.0 │ 8 │ 0x8086 │ I350 Gigabit Network Connection │ │ 0x0001 │ Ethernet Server A
├──────────┼────────┼──────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────────────────────────────┼──────┼──────────────────┼──────────────────
│ 0x020000 │ 0x1521 │ 0000:01:00.1 │ 9 │ 0x8086 │ I350 Gigabit Network Connection │ │ 0x0001 │ Ethernet Server A
├──────────┼────────┼──────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────────────────────────────┼──────┼──────────────────┼──────────────────
│ 0x020000 │ 0x1521 │ 0000:01:00.2 │ 10 │ 0x8086 │ I350 Gigabit Network Connection │ │ 0x0001 │ Ethernet Server A
├──────────┼────────┼──────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────────────────────────────┼──────┼──────────────────┼──────────────────
│ 0x020000 │ 0x1521 │ 0000:01:00.3 │ 11 │ 0x8086 │ I350 Gigabit Network Connection │ │ 0x0001 │ Ethernet Server A
├──────────┼────────┼──────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────────────────────────────┼──────┼──────────────────┼──────────────────
│ 0x030000 │ 0x3e92 │ 0000:00:02.0 │ 0 │ 0x8086 │ CoffeeLake-S GT2 [UHD Graphics 630] │ 1 │ 0x312d │


So now some questions I cannot seem to get an answer to:

1) Do I need to blacklist my Intel network driver for passthrough to work? I have seen posts where some have blacklisted and others have not.
2) Do I need to blacklist my Intel iGPU driver for mediated access to work?
3) Do I need a VFIO config that identifies the I350 ports and iGPU for passthrough to work, where the driver is replaced by VFIO-PCI?
4) Should my passthrough PCI devices be showing up in Mapped or Raw Devices section?

Thanks,
Steve

 

[leesteken]

1) Do I need to blacklist my Intel network driver for passthrough to work? I have seen posts where some have blacklisted and others have not.

Not necessarily. If your want to prevent the driver from touching the device, there is a more specific way to do it: early binding to vfio-pci.

2) Do I need to blacklist my Intel iGPU driver for mediated access to work?

Definitely not. For mediated device passthrough you actually need a driver on Proxmox loaded for the device (to do the actual mediation).

3) Do I need a VFIO config that identifies the I350 ports and iGPU for passthrough to work, where the driver is replaced by VFIO-PCI?

Not necessarily (as Proxmox will do this when you start the VM) but you can use this to prevent any driver from touching the device.
For example because the device does not reset properly and won't work in a VM if a driver is loaded on the host first. This is more specific than blacklisting a driver (see question 1) because you only prevent the normal driver(s) from touching this device (based on device ID instead of PCI ID).

4) Should my passthrough PCI devices be showing up in Mapped or Raw Devices section?

They will show up in Raw. You can configure Mapped devices yourself in another part of the GUI but that's more useful when running a cluster where the same device has different (raw) PCI IDs on each machine.

 

[spetrillo]

@leesteken thank you for the responses. This is really helpful to me. I am going to try PCI passthrough as I have it configured presently. If the VM does not work then I will add the VFIO piece and test again.

I will respond here once I have tested everything!

 

[spetrillo]

A quick question about PCI assignments in a VM. With the Intel network card I have I am going to passthrough the entire card. Do I also check ROM-Bar and PCI-Express options?

 

[leesteken]

I will respond here once I have tested everything!

No need to tell me about what you might or might not do in the future. Please just post the (failed) results or new information when it is available.

With the Intel network card I have I am going to passthrough the entire card. Do I also check ROM-Bar and PCI-Express options?

I cannot comment on your specific hardware, sorry.
PCI or PCIe usually does not really matter but the driver inside the VM might assume one of the other. I would use OVMF and PCI-Express for any modern hardware unless there is a specific reason not to.
Similarly, I would enable ROM-Bar unless there is a reason not to because a specific device needs this as a work-around.
Some devices need a specific ROM-file but usually only as a work-around because they don't reset properly otherwise.