[SOLVED] SATA passthrough on B550

O

onbartik

New Member
Mar 4, 2025
3
0
1
Hello community, here is one "never-ending" problem. I've been through the forum, but I got stuck. I aim to make a PCIE passthrough of the SATA controller on the GIGABYTE MC12-le0 board with a B550 chipset and 5650G Ryzen5 CPU. I know this board is kinda cursed, and B550 is pretty problematic as its IOMMU groups are really unfriendly. Here are some details:

here are loaded modules:
Bash: 
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

I use GRUB, so I applied the acs patch. I know, I know, but it is the last resort here.

Bash: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt pcie_acs_override=downstream,multifunction"

With this patch, the IOMMU groups look like this:

Bash: 
IOMMU Group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU Group 1 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU Group 2 00:02.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge [1022:1634]
IOMMU Group 3 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU Group 4 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus [1022:1635]
IOMMU Group 5 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 51)
IOMMU Group 5 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU Group 6 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 0 [1022:166a]
IOMMU Group 6 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 1 [1022:166b]
IOMMU Group 6 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 2 [1022:166c]
IOMMU Group 6 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 3 [1022:166d]
IOMMU Group 6 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 4 [1022:166e]
IOMMU Group 6 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 5 [1022:166f]
IOMMU Group 6 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 6 [1022:1670]
IOMMU Group 6 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 7 [1022:1671]
IOMMU Group 7 01:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] 500 Series Chipset USB 3.1 XHCI Controller [1022:43ee]
IOMMU Group 8 01:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] 500 Series Chipset SATA Controller [1022:43eb]
IOMMU Group 9 01:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 500 Series Chipset Switch Upstream Port [1022:43e9]
IOMMU Group 10 02:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 11 02:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 12 02:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 13 02:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 14 03:00.0 PCI bridge [0604]: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge [1a03:1150] (rev 04)
IOMMU Group 14 04:00.0 VGA compatible controller [0300]: ASPEED Technology, Inc. ASPEED Graphics Family [1a03:2000] (rev 41)
IOMMU Group 15 05:00.0 Non-Volatile memory controller [0108]: Hosin Global Electronics Patriot P300 NVMe SSD (DRAM-less) [1ed0:2283]
IOMMU Group 16 07:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)
IOMMU Group 17 08:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)
IOMMU Group 18 09:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [Radeon Vega Series / Radeon Vega Mobile Series] [1002:1638] (rev d9)
IOMMU Group 19 09:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor [1022:15df]
IOMMU Group 20 09:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1 [1022:1639]
IOMMU Group 21 09:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1 [1022:1639]
IOMMU Group 22 09:00.5 Multimedia controller [0480]: Advanced Micro Devices, Inc. [AMD] ACP/ACP3X/ACP6x Audio Coprocessor [1022:15e2] (rev 01)
IOMMU Group 23 09:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h/19h HD Audio Controller [1022:15e3]

After that, I created
Code: 
pve-blacklist.conf
in
Code: 
/etc/modprobe.d/
and put there the following lines:

Bash: 
blacklist ahci
options vfio_iommu_type1 allow_unsafe_interrupts=1
options vfio-pci ids=1022:43eb
softdep ahci pre: vfio-pci

But when I run
Code: 
lspci -nnk -s 01:00.1
I get:

Bash: 
01:00.1 SATA controller [0106]:
                                      Advanced Micro Devices, Inc. [AMD] 500 Series Chipset SATA Controller [1022:43eb]
                                      Subsystem: Gigabyte Technology Co., Ltd 500 Series Chipset SATA Controller [1458:1000]
                                      Kernel driver in use: vfio-pci
                                      Kernel modules: ahci

And when I run the VM (TrueNAS Scale) the starting crushes with the following error:
Bash: 
kvm: -device vfio-pci,host=0000:01:00.1,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: vfio 0000:01:00.1: Failed to set up TRIGGER eventfd signaling for interrupt INTX-0: VFIO_DEVICE_SET_IRQS failure: Transport endpoint is not connected
TASK ERROR: start failed: QEMU exited with code 1

Is there anything else I can try? Many thanks for any feedback, guys.

Ondrej
 
onbartik said:
Hello community, here is one "never-ending" problem. I've been through the forum, but I got stuck. I aim to make a PCIE passthrough of the SATA controller on the GIGABYTE MC12-le0 board with a B550 chipset and 5650G Ryzen5 CPU. I know this board is kinda cursed, and B550 is pretty problematic as its IOMMU groups are really unfriendly. Here are some details:

here are loaded modules:
Bash: 
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

I use GRUB, so I applied the acs patch. I know, I know, but it is the last resort here.

Bash: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt pcie_acs_override=downstream,multifunction"

With this patch, the IOMMU groups look like this:

Bash: 
IOMMU Group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU Group 1 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU Group 2 00:02.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge [1022:1634]
IOMMU Group 3 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU Group 4 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus [1022:1635]
IOMMU Group 5 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 51)
IOMMU Group 5 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU Group 6 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 0 [1022:166a]
IOMMU Group 6 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 1 [1022:166b]
IOMMU Group 6 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 2 [1022:166c]
IOMMU Group 6 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 3 [1022:166d]
IOMMU Group 6 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 4 [1022:166e]
IOMMU Group 6 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 5 [1022:166f]
IOMMU Group 6 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 6 [1022:1670]
IOMMU Group 6 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 7 [1022:1671]
IOMMU Group 7 01:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] 500 Series Chipset USB 3.1 XHCI Controller [1022:43ee]
IOMMU Group 8 01:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] 500 Series Chipset SATA Controller [1022:43eb]
IOMMU Group 9 01:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 500 Series Chipset Switch Upstream Port [1022:43e9]
IOMMU Group 10 02:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 11 02:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 12 02:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 13 02:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 14 03:00.0 PCI bridge [0604]: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge [1a03:1150] (rev 04)
IOMMU Group 14 04:00.0 VGA compatible controller [0300]: ASPEED Technology, Inc. ASPEED Graphics Family [1a03:2000] (rev 41)
IOMMU Group 15 05:00.0 Non-Volatile memory controller [0108]: Hosin Global Electronics Patriot P300 NVMe SSD (DRAM-less) [1ed0:2283]
IOMMU Group 16 07:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)
IOMMU Group 17 08:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)
IOMMU Group 18 09:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [Radeon Vega Series / Radeon Vega Mobile Series] [1002:1638] (rev d9)
IOMMU Group 19 09:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor [1022:15df]
IOMMU Group 20 09:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1 [1022:1639]
IOMMU Group 21 09:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1 [1022:1639]
IOMMU Group 22 09:00.5 Multimedia controller [0480]: Advanced Micro Devices, Inc. [AMD] ACP/ACP3X/ACP6x Audio Coprocessor [1022:15e2] (rev 01)
IOMMU Group 23 09:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h/19h HD Audio Controller [1022:15e3]

After that, I created
Code: 
pve-blacklist.conf
in
Code: 
/etc/modprobe.d/
and put there the following lines:

Bash: 
blacklist ahci
options vfio_iommu_type1 allow_unsafe_interrupts=1
options vfio-pci ids=1022:43eb
softdep ahci pre: vfio-pci

But when I run
Code: 
lspci -nnk -s 01:00.1
I get:

Bash: 
01:00.1 SATA controller [0106]:
                                      Advanced Micro Devices, Inc. [AMD] 500 Series Chipset SATA Controller [1022:43eb]
                                      Subsystem: Gigabyte Technology Co., Ltd 500 Series Chipset SATA Controller [1458:1000]
                                      Kernel driver in use: vfio-pci
                                      Kernel modules: ahci

And when I run the VM (TrueNAS Scale) the starting crushes with the following error:
Bash: 
kvm: -device vfio-pci,host=0000:01:00.1,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: vfio 0000:01:00.1: Failed to set up TRIGGER eventfd signaling for interrupt INTX-0: VFIO_DEVICE_SET_IRQS failure: Transport endpoint is not connected
TASK ERROR: start failed: QEMU exited with code 1

Is there anything else I can try? Many thanks for any feedback, guys.

Ondrej
Click to expand...
The error suggests an interrupt handling issue with the SATA controller, likely due to how the AMD B550 chipset handles IOMMU. Try these steps:

1️ Forcefully Bind VFIO to SATA Controller
Add this to /etc/modprobe.d/vfio.conf:
Code: 
options vfio-pci ids=1022:43eb disable_vga=1

Then regenerate the initramfs and reboot:
Code: 
update-initramfs -u
reboot

2️ Use MSI (Message Signaled Interrupts) Instead of INTx

Check if the SATA controller supports MSI interrupts:
Code: 
cat /proc/interrupts | grep vfio

If not enabled, add this to your Proxmox VM config (/etc/pve/qemu-server/<vmid>.conf):
Code: 
args: -set device.hostpci0.msix=on

3️ Try Different PCI Bus Assignments
Modify the VM config to use pcie=1:
Code: 
hostpci0: 0000:01:00.1,pcie=1
If it still fails, try x-vga=on:
Code: 
hostpci0: 0000:01:00.1,pcie=1,x-vga=on

4️ Check for BIOS/UEFI Settings
  • Ensure IOMMU is enabled in BIOS
  • Disable CSM (Legacy Mode)
  • Try "Above 4G Decoding = Enabled"
If the issue persists, try passing the entire PCI bridge (01:00.x) instead of just the SATA controller.
 
shbaek said:
The error suggests an interrupt handling issue with the SATA controller, likely due to how the AMD B550 chipset handles IOMMU. Try these steps:

1️ Forcefully Bind VFIO to SATA Controller
Add this to /etc/modprobe.d/vfio.conf:
Code: 
options vfio-pci ids=1022:43eb disable_vga=1

Then regenerate the initramfs and reboot:
Code: 
update-initramfs -u
reboot

2️ Use MSI (Message Signaled Interrupts) Instead of INTx

Check if the SATA controller supports MSI interrupts:
Code: 
cat /proc/interrupts | grep vfio

If not enabled, add this to your Proxmox VM config (/etc/pve/qemu-server/<vmid>.conf):
Code: 
args: -set device.hostpci0.msix=on

3️ Try Different PCI Bus Assignments
Modify the VM config to use pcie=1:
Code: 
hostpci0: 0000:01:00.1,pcie=1
If it still fails, try x-vga=on:
Code: 
hostpci0: 0000:01:00.1,pcie=1,x-vga=on

4️ Check for BIOS/UEFI Settings
  • Ensure IOMMU is enabled in BIOS
  • Disable CSM (Legacy Mode)
  • Try "Above 4G Decoding = Enabled"
If the issue persists, try passing the entire PCI bridge (01:00.x) instead of just the SATA controller.
Click to expand...
Thanks for the reply @shbaek . I tried allt he options apart from the BIOS options (but I will!) but this step:

Code: 
args: -set device.hostpci0.msix=on

gives me this error:

Code: 
kvm: -device vfio-pci,host=0000:01:00.1,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: Property 'vfio-pci.msix' not found
TASK ERROR: start failed: QEMU exited with code 1

Did I miss something?
 
onbartik said:
Thanks for the reply @shbaek . I tried allt he options apart from the BIOS options (but I will!) but this step:

Code: 
args: -set device.hostpci0.msix=on

gives me this error:

Code: 
kvm: -device vfio-pci,host=0000:01:00.1,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: Property 'vfio-pci.msix' not found
TASK ERROR: start failed: QEMU exited with code 1

Did I miss something?
Click to expand...

The error "Property 'vfio-pci.msix' not found" suggests that QEMU does not support enabling MSI-X (Message Signaled Interrupts) in this way. Try the following solutions:

1️ Manually Enable MSI for the Device​

Check if MSI is supported on your SATA controller:
Bash: 
cat /proc/interrupts | grep vfio

If it's using INTx instead of MSI, force MSI by running:
Bash: 
echo 1 > /sys/bus/pci/devices/0000:01:00.1/msi_bus
Then restart the VM and check if the issue is resolved.

2️ Remove msix=on and Try Standard Passthrough​

Instead of using msix=on, modify the VM config (/etc/pve/qemu-server/<vmid>.conf) as follows:
Bash: 
hostpci0: 0000:01:00.1,pcie=1

If it still fails, try:
Bash: 
hostpci0: 0000:01:00.1,pcie=1,x-vga=on

3️ Check QEMU Version and Update If Needed​

Run:
Bash: 
pveversion -v | grep qemu

If you are on an older version, consider updating QEMU:
Bash: 
apt update && apt install qemu-system-x86

4️ Try Passing the Entire PCI Bridge​

If your SATA controller is part of a larger IOMMU group, pass the entire PCI bridge instead of just the SATA controller. Find the PCI bridge by running:
Bash: 
lspci -nnk | grep -A2 "01:00"

If 01:00.0 is a PCI bridge, try passing 01:00.x instead of just 01:00.1.
Example VM config:
Bash: 
hostpci0: 0000:01:00.0,pcie=1

Conclusion:​

  • Check if MSI is supported and manually enable it.
  • Remove msix=on from the VM config and use standard PCIe passthrough.
  • Ensure QEMU is updated to avoid missing property errors.
  • Try passing the entire PCI bridge instead of just the SATA controller.
 
After many long days of trying I gave it up and got dedicated HBA. It works like charm with no patches and coMpromises. Pitty I don't have any use for the 6 sata ports on MB...
 
You must log in or register to reply here.
Top Bottom