Issues with GPU passthrough Threadripper and AMD GPU

iPenguin

New Member
May 11, 2023
11
0
1
I'm new to Proxmox and still learning. I followed some guides to do GPU passthrough, but I'm still having issues. My 6700 XT is detected within my Windows 10 VM and shows no issues in Device Manager. But when I try to install the AMD Driver, the VM stops responding. When it responds again, I'm at the login screen. Thanks in advance!

My CPU is the Threadripper 3955WX which I'm running on the Gigabyte WRX80-SU8-IPMI and 6700 XT. I have a ZFS pool for my Proxmox drive and a separate pool for my VMs. I have Re-sizeable Bar turned off in BIOS.

Here is the VM setup, I did not touch the .conf file
2023-05-25_12-49-03.png

pveversion -v
Code:
proxmox-ve: 7.4-1 (running kernel: 5.15.107-2-pve)
pve-manager: 7.4-3 (running version: 7.4-3/9002ab8a)
pve-kernel-5.15: 7.4-3
pve-kernel-5.15.107-2-pve: 5.15.107-2
pve-kernel-5.15.107-1-pve: 5.15.107-1
pve-kernel-5.15.102-1-pve: 5.15.102-1
ceph-fuse: 15.2.17-pve1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx4
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4-3
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.4-1
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-3
libpve-rs-perl: 0.7.6
libpve-storage-perl: 7.4-2
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
proxmox-backup-client: 2.4.2-1
proxmox-backup-file-restore: 2.4.2-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.7.0
pve-cluster: 7.3-3
pve-container: 4.4-3
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-2
pve-firewall: 4.3-2
pve-firmware: 3.6-5
pve-ha-manager: 3.6.1
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-1
qemu-server: 7.4-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.11-pve1

qm config 120
Code:
balloon: 0
bios: ovmf
boot: order=scsi0;net0;ide0
cores: 8
cpu: host
efidisk0: VMs:120/vm-120-disk-0.qcow2,efitype=4m,pre-enrolled-keys=1,size=528K
hostpci0: 0000:87:00,pcie=1,x-vga=1
ide0: local:iso/virtio-win-0.1.225.iso,media=cdrom,size=519590K
machine: pc-q35-7.2
memory: 16384
meta: creation-qemu=7.2.0,ctime=1684875426
name: GamingVM
net0: virtio=E6:7C:03:E9:F9:3B,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsi0: VMs:120/vm-120-disk-1.qcow2,size=120G,ssd=1
scsihw: virtio-scsi-pci
smbios1: uuid=1c371330-d730-4cd2-aa23-261370874297
sockets: 1
tpmstate0: VMs:120/vm-120-disk-2.raw,size=4M,version=v2.0
usb0: host=8-1.1
usb1: host=8-1.2
usb2: host=8-1.3
usb3: host=8-1.4
vga: none

I read that because I'm using a ZFS pool I will be on systemmd
/etc/kernel/cmdline
Code:
root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet amd_iommu=on

/etc/modules
Code:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

/etc/modprobe.d/iommu_unsafe_interrupts.conf
Code:
options vfio_iommu_type1 allow_unsafe_interrupts=1

lspci
Code:
87:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 22 (rev c1)
87:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Device ab28

lspci -n -s 87:00
Code:
87:00.0 0300: 1002:73df (rev c1)
87:00.1 0403: 1002:ab28

/etc/modprobe.d/vfio.conf
Code:
options vfio-pci ids=1002:73df,1002:ab28 disable_vga=1

/etc/modprobe.d/blacklist.conf
Code:
blacklist amdgpu
blacklist radeon
blacklist nouveau
blacklist nvidia
 
Last edited:
I'm new to Proxmox and still learning. I followed some guides to do GPU passthrough, but I'm still having issues. My 6700 XT is detected within my Windows 10 VM and shows no issues in Device Manager. But when I try to install the AMD Driver, the VM stops responding. When it responds again, I'm at the login screen. Thanks in advance!
any logs on the host in the journal/syslog or in the guest (event viewer? ) ?
 
any logs on the host in the journal/syslog or in the guest (event viewer? ) ?

Sorry, I attached the Syslog from the time I started the VM to when it crashes when I attempt to install the AMD Driver. This time it didn't come back up. I had to stop/start it.

Here are the only errors I see in Event Viewer
2023-05-26_07-50-38.png
 

Attachments

  • Proxmox_Syslog.txt
    22.4 KB · Views: 2
can you also post your pveversion -v and the vm config ? (qm config ID)
 
can you try without secure boot in the vm? you have to press esc during the uefi loading screen, and then go to 'device manager' -> 'secure boot configuration' -> disable it
 
can you try without secure boot in the vm? you have to press esc during the uefi loading screen, and then go to 'device manager' -> 'secure boot configuration' -> disable it

Same issue, freezes when I attempt to install the driver.

I verified Secure Boot is disabled by going back into the UEFI. No errors in device Manager

Here is the syslog if needed.
 

Attachments

  • May 30 021512 iP-Server postfixsmtp.txt
    119.4 KB · Views: 1
ok, nothing special in the syslog either.


could you try with a linux vm? just to see if the problem is with the driver or the passthrough itself?
if it works in a linux vm, my guess would be a problem with the windows driver...

edit: is your host booting in uefi or bios mode?
 
Last edited:
ok, nothing special in the syslog either.


could you try with a linux vm? just to see if the problem is with the driver or the passthrough itself?
if it works in a linux vm, my guess would be a problem with the windows driver...

edit: is your host booting in uefi or bios mode?
I can try Linux. Are there any guides / distros you recommend? I'm not super familiar with Linux, but i'm open to learning/trying.

UEFI with Secure Boot. I tried booting UEFI without Secure boot.

Not sure if this matters
Above 4G Decoding = Enabled
Re-size BAR = Disabled
SR-IOV Support = Enabled
BME DMA Mitigation = Disabled
Hot-Plug Support = Enabled
 
Last edited:
I can try Linux. Are there any guides / distros you recommend? I'm not super familiar with Linux, but i'm open to learning/trying.
i'd probably just use ubuntu desktop, should be newcomer friendly enough
 
i'd probably just use ubuntu desktop, should be newcomer friendly enough

Thank you for your help so far. I enabled the default display and noticed the error is a BSOD VIDEO DXGKRNL FATAL ERROR. I'm googling to see how others have resolved this error.

I was able to install the AMD graphics driver using the full version rather than the auto detect one, but that didn't resolve the error.

Using i440fx v7.2 works without crashing. I read that q35 is more desired. I'm trying different versions to see if they crash.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!