GPU Passthrough Help

[DottedLine]

I'm trying to get GPU passthrough to work, but every time I install the video driver, the screen either goes black or flashes splotchy colors.

Relevant Hardware:

• i5-3570
• GA-H77M-D3H
• XFX Radeon R9 380
• USB keyboard and mouse

Followed the steps from https://pve.proxmox.com/wiki/Pci_passthrough

Steps taken:

• changed /etc/default/grub
• added vfio modules to /etc/modules
• added vendor id to /etc/modprobe.d/vfio.conf
• blacklisted video drivers
• Created vm
  • Windows 8.1
  • New disk using Virtio, iothread, write back cache (originally tried scsi, but the install would hang at around 50%)
  • network as virtio
  • added Win 8.1 dvd as sata0
  • added latest virtio drivers iso as sata1
  • 2 cores, 1 socket
  • checked qemu agent
  • changed bios to ovmf

Relevant Files / Output

pveversion -v
proxmox-ve: 4.1-45 (running kernel: 4.4.6-1-pve)
pve-manager: 4.1-30 (running version: 4.1-30/9e199213)
pve-kernel-4.4.6-1-pve: 4.4.6-45
pve-kernel-4.2.6-1-pve: 4.2.6-36
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 1.0-1
pve-cluster: 4.0-39
qemu-server: 4.0-69
pve-firmware: 1.1-8
libpve-common-perl: 4.0-59
libpve-access-control: 4.0-16
libpve-storage-perl: 4.0-50
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.5-13
pve-container: 1.0-59
pve-firewall: 2.0-24
pve-ha-manager: 1.0-27
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.5-7
lxcfs: 2.0.0-pve2
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5-pve9~jessie

/etc/default/grub:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=downstream"

/etc/modules
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

/etc/modprobe.d/vfio.conf
options vfio-pci ids=1002:6939,1002:aad8 disable_vga=1

/etc/modprobe.d/video.conf
blacklist radeon
blacklist flgrx
blacklist nvidia
blacklist nouveau

find /sys/kernel/iommu_groups/ -type l
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/2/devices/0000:00:02.0
/sys/kernel/iommu_groups/3/devices/0000:00:14.0
/sys/kernel/iommu_groups/4/devices/0000:00:16.0
/sys/kernel/iommu_groups/5/devices/0000:00:1a.0
/sys/kernel/iommu_groups/6/devices/0000:00:1b.0
/sys/kernel/iommu_groups/7/devices/0000:00:1c.0
/sys/kernel/iommu_groups/8/devices/0000:00:1c.4
/sys/kernel/iommu_groups/9/devices/0000:00:1c.6
/sys/kernel/iommu_groups/10/devices/0000:00:1d.0
/sys/kernel/iommu_groups/11/devices/0000:00:1f.0
/sys/kernel/iommu_groups/11/devices/0000:00:1f.2
/sys/kernel/iommu_groups/11/devices/0000:00:1f.3
/sys/kernel/iommu_groups/12/devices/0000:01:00.0
/sys/kernel/iommu_groups/12/devices/0000:01:00.1
/sys/kernel/iommu_groups/13/devices/0000:02:00.0
/sys/kernel/iommu_groups/14/devices/0000:03:00.0
/sys/kernel/iommu_groups/15/devices/0000:04:00.0

dmesg | grep -e DMAR -e IOMMU
[ 0.000000] ACPI: DMAR 0x00000000CB67A8B0 0000B8 (v01 INTEL SNB 00000001 INTL 00000001)
[ 0.000000] DMAR: IOMMU enabled
[ 0.027480] DMAR: Host address width 36
[ 0.027482] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[ 0.027489] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c0000020e60262 ecap f0101a
[ 0.027490] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[ 0.027493] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap c9008020660262 ecap f0105a
[ 0.027494] DMAR: RMRR base: 0x000000cc106000 end: 0x000000cc114fff
[ 0.027494] DMAR: RMRR base: 0x000000cd800000 end: 0x000000cf9fffff
[ 0.027496] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1
[ 0.027497] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[ 0.027498] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.027723] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 0.442079] DMAR: No ATSR found
[ 0.442143] DMAR: dmar0: Using Queued invalidation
[ 0.442491] DMAR: dmar1: Using Queued invalidation
[ 0.442497] DMAR: Setting RMRR:
[ 0.442510] DMAR: Setting identity map for device 0000:00:02.0 [0xcd800000 - 0xcf9fffff]
[ 0.442655] DMAR: Setting identity map for device 0000:00:14.0 [0xcc106000 - 0xcc114fff]
[ 0.442666] DMAR: Setting identity map for device 0000:00:1a.0 [0xcc106000 - 0xcc114fff]
[ 0.442678] DMAR: Setting identity map for device 0000:00:1d.0 [0xcc106000 - 0xcc114fff]
[ 0.442685] DMAR: Prepare 0-16MiB unity mapping for LPC
[ 0.442690] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[ 0.442865] DMAR: Intel(R) Virtualization Technology for Directed I/O
[ 12.433462] [drm] DMAR active, disabling use of stolen memory
[ 12.620651] AMD IOMMUv2 driver by Joerg Roedel <jroedel@suse.de>
[ 12.620653] AMD IOMMUv2 functionality not available on this system

100.conf
agent: 1
bios: ovmf
bootdisk: virtio0
cores: 2
hostpci0: 01:00,x-vga=on
memory: 10240
name: Win
net0: bridge=vmbr0,virtio=36:64:30:31:33:64
numa: 0
ostype: win8
scsihw: virtio-scsi-pci
smbios1: uuid=9412a341-eb3c-46df-bdfb-eaaeeb442a6d
snaptime: 1461050347
sockets: 1
tablet: 0
usb0: host=1532:010d
usb1: host=04d9:fc02
vga: off
virtio0: local:100/vm-100-disk-1.qcow2,cache=writeback,size=100G

Using the above config, when I install the AMD drivers I get a black screen and the moniter suspends. I have to reset the guest and reboot the host. Trying to start the guest again results in the Windows boot screen, then a black screen with the monitor suspending.

I tried changing the following in the config file:

• changed hostpci0: 01:00,x-vga=on to hostpci0: 01:00,pcie=1,x-vga=on
• added machine:q35
• added cpu: host
• changed ostype: win8 to ostype: other

After making those changes I tried installing the AMD driver. The results were slightly different.
The monitor would cycle between supsending and turning on to splotchyness

Rebooting the guest would only result in a BSOD

I'm sorry for the lengthy post, but I tried to include as much information as possible. Any help, suggestions, or ideas on how I can get this to work would be appreciated.