Passthrough questions

[Navy2k]

Hi,

I want to pass through some devices to different machines.
First, some info:

08:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901]
        Kernel driver in use: ahci
        Kernel modules: ahci
09:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901]
        Kernel driver in use: ahci
        Kernel modules: ahci

I want to pass through only the 09:00:0 one of these 2 onboard Sata controllers.
I have already passed through a i350, a 10Gig card and a nvme cache to various VMs:

/etc/modules:
vfio
vfio_iommu_type1
vfio_pci ids=8086:1521,1043:874a,c0a9:540a
vfio_virqfd

/etc/modprobe.d/pve-blacklist.conf:
softdep igb pre: vfio-pci
softdep atlantic: vfio-pci
softdep nvme: vfio-pci
options vfio-pci ids=8086:1521
options vfio-pci ids=1043:874a
options vfio-pci ids=c0a9:540a

But how do you specify only the one ID?
I found some explanations but they all were incredibly complex with bash scripts in multiple places? Is this really the way to go?

Thanks in advance.
Ben

 

[Stefan_R]

Yes, you need to unbind and rebind the devices manually. The "complex bash script" is really not that bad:

#!/bin/sh
DEVICE="08:00.0"
echo "$DEVICE" > /sys/bus/pci/drivers/ahci/unbind
echo "$DEVICE" > /sys/bus/pci/drivers/vfio-pci/bind

That should do it, change DEVICE to the PCI address of the SATA controller you want to pass through. Run this after startup, either via a hookscript before your VM starts or simply after boot via a systemd service or my personal favourite, an @reboot entry in your root crontab.

 

[Navy2k]

Hi,

Just to be clear, this is instead of blacklisting and the /etc/modules entry for the hardware ID?

best regards,
Ben

 

[Stefan_R]

Yes. Blacklisting the ahci driver would render *all* your SATA controllers useless for the host (PVE), and specifying the PCI id in vfio-pci.ids would auto-load vfio-pci and you would need to do the unbind/bind dance I posted in reverse to load the ahci driver again.

Also, looking at your posted config, it is wrong. /etc/modules may *only* contain module names, no parameters. So remove the ids=... part from there. However, you got the format correct, which is how it should be specified in /etc/modprobe.d/pve-blacklist.conf, instead of it being on multiple lines (the latter *might* work as well, not sure, but I've only ever seen it in one line).

 

[Navy2k]

Thanks for the heads up, IDs are indeed configured twice...

The syntax in /etc/modprobe.d/pve-blacklist.conf works and I like it because in the experimentation phase I can add or remove for example the 2nd line from both sections (softdep and options) whithoud the need to check the coresponding ID for each driver.