PCI GPU Passtrough (RX580) crashes host when starting VM

[dotdelete]

Hi Everyone,

So i have been struggling with this for the last couple of days. I have been trying to pass my GPU Radeon RX580 to my ubuntu server VM. I will list my settings below.

Whenever I press start on the VM in proxmox the host machine (where proxmox is on) crashes and restarts. It only gives me this error code "TASK ERROR: start failed: QEMU exited with code 1"

Hardware:
intel i5 6600k
GPU RX580
B150 - Motherboard

BIOS:
VT-D enabled

Proxmox:
Via GRUB

Any help and suggestions are greatly appreciated! If more info is needed please let me know. I am happy to provide anything that is needed.

Spoiler: Contents of /etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="root=/dev/mapper/pve-root ro quiet intel_iommu=on initcall_blacklist=sysfb_init"

Spoiler: Output of "dmesg | grep -e DMAR -e IOMMU

[ 0.008138] ACPI: DMAR 0x00000000A39B9DD8 0000B0 (v01 INTEL SKL 00000001 INTL 00000001)
[ 0.008191] ACPI: Reserving DMAR table memory at [mem 0xa39b9dd8-0xa39b9e87]
[ 0.030722] DMAR: IOMMU enabled
[ 0.084284] DMAR: Host address width 39
[ 0.084285] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[ 0.084295] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 7e3ff0505e
[ 0.084297] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[ 0.084300] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[ 0.084301] DMAR: RMRR base: 0x000000b11ba000 end: 0x000000b11d9fff
[ 0.084304] DMAR: RMRR base: 0x000000b3800000 end: 0x000000b7ffffff
[ 0.084305] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1
[ 0.084307] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[ 0.084308] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.085839] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 0.300817] DMAR: [Firmware Bug]: RMRR entry for device 06:00.0 is broken - applying workaround
[ 0.300819] DMAR: No ATSR found
[ 0.300820] DMAR: No SATC found
[ 0.300821] DMAR: IOMMU feature fl1gp_support inconsistent
[ 0.300822] DMAR: IOMMU feature pgsel_inv inconsistent
[ 0.300823] DMAR: IOMMU feature nwfs inconsistent
[ 0.300823] DMAR: IOMMU feature eafs inconsistent
[ 0.300824] DMAR: IOMMU feature prs inconsistent
[ 0.300825] DMAR: IOMMU feature nest inconsistent
[ 0.300825] DMAR: IOMMU feature mts inconsistent
[ 0.300826] DMAR: IOMMU feature sc_support inconsistent
[ 0.300827] DMAR: IOMMU feature dev_iotlb_support inconsistent
[ 0.300828] DMAR: dmar0: Using Queued invalidation
[ 0.300830] DMAR: dmar1: Using Queued invalidation
[ 0.302501] DMAR: Intel(R) Virtualization Technology for Directed I/O

Spoiler: IOMMU group check

for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done
IOMMU group 0 00:02.0 VGA compatible controller [0300]: Intel Corporation HD Graphics 530 [8086:1912] (rev 06)
IOMMU group 10 00:1f.0 ISA bridge [0601]: Intel Corporation B150 Chipset LPC/eSPI Controller [8086:a148] (rev 31)
IOMMU group 10 00:1f.2 Memory controller [0580]: Intel Corporation 100 Series/C230 Series Chipset Family Power Management Controller [8086:a121] (rev 31)
IOMMU group 10 00:1f.3 Audio device [0403]: Intel Corporation 100 Series/C230 Series Chipset Family HD Audio Controller [8086:a170] (rev 31)
IOMMU group 10 00:1f.4 SMBus [0c05]: Intel Corporation 100 Series/C230 Series Chipset Family SMBus [8086:a123] (rev 31)
IOMMU group 11 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I219-V [8086:15b8] (rev 31)
IOMMU group 12 04:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge [1b21:1080] (rev 04)
IOMMU group 13 06:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller [1b21:1242]
IOMMU group 1 00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers [8086:191f] (rev 07)
IOMMU group 2 00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
[B]IOMMU group 2 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e7)
IOMMU group 2 01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0][/B]
IOMMU group 3 00:14.0 USB controller [0c03]: Intel Corporation 100 Series/C230 Series Chipset Family USB 3.0 xHCI Controller [8086:a12f] (rev 31)
IOMMU group 4 00:16.0 Communication controller [0780]: Intel Corporation 100 Series/C230 Series Chipset Family MEI Controller #1 [8086:a13a] (rev 31)
IOMMU group 5 00:17.0 SATA controller [0106]: Intel Corporation Q170/Q150/B150/H170/H110/Z170/CM236 Chipset SATA Controller [AHCI Mode] [8086:a102] (rev 31)
IOMMU group 6 00:1c.0 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #5 [8086:a114] (rev f1)
IOMMU group 7 00:1d.0 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #9 [8086:a118] (rev f1)
IOMMU group 8 00:1d.2 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #11 [8086:a11a] (rev f1)
IOMMU group 9 00:1d.3 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #12 [8086:a11b] (rev f1)

Spoiler: Blacklisting drivers

cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=1002:67df,1002:aaf0 disable_vga=1