PCI(e) Passthrough for SATA controller

[andcbii]

I'm trying to get Makemkv to work on a VM on my proxmox host. It didn't work by passing the disc drive to the VM from the GUI. I've read the best way to do this was to passthrough a sata controller to the VM and connect the disc drive to that. So now I'm going through this guide to setup up PCI(e) passthrough:

https://pve.proxmox.com/wiki/PCI(e)_Passthrough#qm_pci_passthrough_update_initramfs

My mobo is a ROG STRIX B550-F GAMING and I bough this controller https://www.newegg.com/syba-si-pex40156-pci-express/p/1B3-002S-00003?Item=9SIA3XWC5Z0785

I think I've hit a roadblock with the controller but I'm not sure. This is what I see when I run lspci -nnk (I've removed what I didn't think was relevant)

root@proxmox1:~# lspci -nnk

...

02:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] Device [1022:43eb]

        Subsystem: ASMedia Technology Inc. Device [1b21:1062]

        Kernel driver in use: ahci

        Kernel modules: ahci

...

04:00.0 SATA controller [0106]: ASMedia Technology Inc. Device [1b21:1064] (rev 02)

        Subsystem: ZyDAS Technology Corp. Device [2116:2116]

        Kernel driver in use: ahci

        Kernel modules: ahci

...

My concern is that both the sata controller on the mobo and the new one use the same driver "ahci". If I add

blacklist achi

to /etc/modprobe.d/.conf will that stop the sata controller built in to the mobo from working?

 

[Dunuin]

My concern is that both the sata controller on the mobo and the new one use the same driver "ahci". If I add

blacklist achi

to /etc/modprobe.d/.conf will that stop the sata controller built in to the mobo from working?

Jup. But you also could blacklist individual devices instead of the whole driver.

 

[andcbii]

Hmmm... i think the real issue i'm facing is that no matter which pcie slot i'm using the card doesn't get its own IOMMU group...

IOMMU group 15 02:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ee]
IOMMU group 15 02:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] Device [1022:43eb]
IOMMU group 15 02:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43e9]
IOMMU group 15 03:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU group 15 03:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU group 15 03:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU group 15 04:00.0 SATA controller [0106]: ASMedia Technology Inc. Device [1b21:1064] (rev 02)
IOMMU group 15 06:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I225-V [8086:15f3] (rev 03)

 

[Dunuin]

Check another PCIe slot. On consumer chipsets/mainboards usually only the 1 or 2 PCIe 16x slots got its own IOMMU group. The other PCIe slots are usually connected to the chipset and not directly to the CPU and then share the IOMMU group with all the other onboard stuff attached to the chipset like SATA controllers, USB controllers, NICs, and so on. Google for "pcie_acs_override". If you are lucky you can split your existing IOMMU groups with it.

 

[andcbii]

Well I tried to get the IOMMU group updated by following this:

2) adding "pcie_acs_override=downstream" to kernel boot commandline (grub or systemd-boot) options, which can help on some setup with bad ACS implementation.

I updated /etc/default/grub to

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on pcie_acs_override=downstream"
GRUB_CMDLINE_LINUX=""

and ran update-grub and rebooted, but the group is the same

 

[leesteken]

Hmmm... i think the real issue i'm facing is that no matter which pcie slot i'm using the card doesn't get its own IOMMU group...

On the B550 motherboards only the first x16 PCIe slot and the first M.2 slot are in their own IOMMU group (provided by the CPU). You need an X570 (or X470/X370 with two x8) for this. pcie_acs_override=downstream,multifunction will lie about the groups and also break the security isolation (exposing the memory of the host to the VM). So don't run untrusted software or allow untrusted users on that VM. And there is no guarantee that it will work but you can try.

EDIT: It is also possible that that SATA controller does not reset properly and won't work (or after the first time) in the VM. But maybe you are lucky.

EDIT2: You don't need amd_iommu=on because it is on by default.
Early binding the SATA controller to vfio-pci with a softdep ahci pre: vfio-pcii might help if it does not reset properly.
You can also try the bus with kernel 5.15 reset method if it does not properly reset with FLR.

 

[andcbii]

On the B550 motherboards only the first x16 PCIe slot and the first M.2 slot are in their own IOMMU group (provided by the CPU). You need an X570 (or X470/X370 with two x8) for this. pcie_acs_override=downstream,multifunction will lie about the groups and also break the security isolation (exposing the memory of the host to the VM). So don't run untrusted software or allow untrusted users on that VM. And there is no guarantee that it will work but you can try.

EDIT: It is also possible that that SATA controller does not reset properly and won't work (or after the first time) in the VM. But maybe you are lucky.

EDIT2: You don't need amd_iommu=on because it is on by default.
Early binding the SATA controller to vfio-pci with a softdep ahci pre: vfio-pcii might help if it does not reset properly.
You can also try the bus with kernel 5.15 reset method if it does not properly reset with FLR.

Thanks for the reply. I moved the controller to the first slot and now it's working as expected. I had to move my gpu to the second slot for now...