GPU Passthrough - AMD WRX80

V

vigorianer

Member
Sep 16, 2024
39
3
8
Does anyone have experience with GPU passthrough on the Asus WRX80 board and an AMD 7900 XTX GPU?

I've worked through many tutorials, but I can't get the graphics card to pass through.

As soon as I add the PCI, the VM reboots or crashes.
I'd also like to add a Focusrite audio interface for audio recording via a USB-C connection.

What information would you need to help me?

CPU: AMD Ryzen Threadrippter Pro 3975W
Motherboard: Asus WRX80 SE. Firmware 1602
1. GPU: AMD Radeon 7900 XTX
1.GPU: AMD Radeon 6600 XT
Win11 Pro
Proxmox VE 8.3.3
 
Hi,

vigorianer said:
As soon as I add the PCI, the VM reboots or crashes.
Click to expand...
just the VM or the whole host?

Please provide the (full) output of
Code: 
pvesh get /nodes/localhost/hardware/pci --pci-class-blacklist ""
in code tags please.
That will list all PCIe devices with their IOMMU group.
 
At the beginning the whole host. After several repair attempts I have no more crashes, but the VM refuses to work with the passed through GPU. I always have to switch to standard display so that I can see anything.

sorry, I couldn't find the command for an output on tags. That's why I'm using a PDF.

thanks for the help
 

Attachments

Does anyone have instructions on how to create vbios? I'm not really getting anywhere.
How do I create a vbios.c file and then load it
 
VM 100 qmp command 'set_password' failed - Could not set password
TASK ERROR: Failed to run vncproxy.

Every VM don t start.

proxmox-ve: 8.3.0 (running kernel: 6.8.12-8-pve)
pve-manager: 8.3.3 (running version: 8.3.3/f157a38b211595d6)
proxmox-kernel-helper: 8.1.0
proxmox-kernel-6.8: 6.8.12-8
proxmox-kernel-6.8.12-8-pve-signed: 6.8.12-8
proxmox-kernel-6.8.12-7-pve-signed: 6.8.12-7
proxmox-kernel-6.8.12-4-pve-signed: 6.8.12-4
ceph-fuse: 17.2.7-pve3
corosync: 3.1.7-pve3
criu: 3.17.1-2+deb12u1
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx11
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.1
libproxmox-backup-qemu0: 1.5.1
libproxmox-rs-perl: 0.3.4
libpve-access-control: 8.2.0
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.0.10
libpve-cluster-perl: 8.0.10
libpve-common-perl: 8.2.9
libpve-guest-common-perl: 5.1.6
libpve-http-server-perl: 5.2.0
libpve-network-perl: 0.10.0
libpve-rs-perl: 0.9.1
libpve-storage-perl: 8.3.3
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.5.0-1
proxmox-backup-client: 3.3.2-1
proxmox-backup-file-restore: 3.3.2-2
proxmox-firewall: 0.6.0
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.3.1
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.7
proxmox-widget-toolkit: 4.3.4
pve-cluster: 8.0.10
pve-container: 5.2.3
pve-docs: 8.3.1
pve-edk2-firmware: 4.2023.08-4
pve-esxi-import-tools: 0.7.2
pve-firewall: 5.1.0
pve-firmware: 3.14-3
pve-ha-manager: 4.0.6
pve-i18n: 3.3.3
pve-qemu-kvm: 9.0.2-5
pve-xtermjs: 5.3.0-3
qemu-server: 8.3.7
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.7-pve1
 
No VM starts

Error:

VM 100 qmp command 'set_password' failed - Could not set password
TASK ERROR: Failed to run vncproxy.



My VM

boot: order=virtio0;ide0;ide2;net0
cores: 4
cpu: host,hidden=1
hostpci0: 0000:03:00.0,romfile=Navi31.rom
ide0: local:iso/virtio-win-0.1.266.iso,media=cdrom,size=707456K
ide2: local:iso/Win10_22H2_German_x64.iso,media=cdrom,size=5992742K
machine: pc-q35-9.0
memory: 4096
meta: creation-qemu=9.0.2,ctime=1739815942
name: Win10VM
net0: virtio=BC:24:11:0D:7A:98,bridge=vmbr0,firewall=1,link_down=1
numa: 0
ostype: win10
scsihw: virtio-scsi-single
smbios1: uuid=27ce2477-bd72-4081-9823-65059c32c9ba
sockets: 1
spice_enhancements: foldersharing=1,videostreaming=all
vga: none
virtio0: local-zfs:vm-100-disk-0,iothread=1,size=50G
vmgenid: 5664d9bd-419e-4f32-81d9-0e3463562129
 
Do you have experience with GPU passthrough with other machines or is this your first time?

The order is wrong, with passthrough GPU as primary you don't have the virtual GPU needed for VNC.

You first install the VM with "default" GPU adapter, so you can use VNC! Don't touch passthrough yet! Then install virtio-software, let windows finish its updates.
Shutdown the VM, after that configure your passthrough (hostpci0: 0000:03:00.0,romfile=Navi31.rom looks good), fire up the VM. VGA output at the physical GPU should work now. If monitor goes on, but screen is black, be patient. Windows will boot blind and this can take 10-15 mins until it has configured the drivers. Then you get the output on the screen, but it takes time.
 
After 25 different tutorials and 6 weeks of tinkering, the GPU passthrough is finally running on my Asus Wrx80 board with RX7900 GK.Thank you for your support, I've learned a lot.
 
Nice!
Can you post your now working vm-config and if you changed something in BIOS? It sure would help others!
 
Here is my Tutorial for: It Works!

Proxmox GPU Passthrough with RX 7900 XTX and Asus WRX80 Motherboard
Proxmox VE 8.3.4

Asus Pro WS WRX80E-SAGE SE WIFI motherboard - Bios

Other settings which are all still set to their default:

Launch CSM -> Disabled
AMD fTPM -> Firmware TPM
CPU Config SVM Mode -> Enabled
PCI Above 4G Decoding -> Enabled
Hot Plug Support -> Enabled

1 - disable TPM
2 - disable the secure boot - removing all keys (yes, you need to delete all of them)
3- uefi enabled
4 - nvme formatted and initialized with gpt


---------------------------------------------------------------------------------------------------------------


Verify IOMMU is enabled


dmesg | grep -e DMAR -e IOMMU

There should be a line that looks like "DMAR: IOMMU enabled". If there is no output, something is wrong.


OPTIONAL: PVE GRUB Config
Works perfectly without it, but it may get you more performance or resolve some issues in some usecases:
1. In the PVE Shell type: nano /etc/default/grub
2. change the GRUB_CMDLINE_LINUX_DEFAULT="quiet" line to GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt"
3. Close with CTRL + X, and save with y
4. execute: update-grub



nano /etc/modules-load.d/modules.conf

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd



Save and execute following command:

update-initramfs -u -k all


Download GPU ROM
zB. Navi31.rom = Bios from RX 7900 TXT (GPU-Z Works fine)


scp C:\Windows\Temp\Navi31.rom root@172.14.3.58:/usr/share/kvm/


Then, we need to tell vfio which devices to virtualize (the GPU 1002:744c + Audio 1002:ab30)

echo "options vfio-pci ids=1002:744c,1002:ab30" >> /etc/modprobe.d/vfio.conf




VM Configuration (VM102)


nano /etc/pve/qemu-server/102.conf



agent: 1
balloon: 0
bios: ovmf
boot: order=scsi0;net0;ide2
cores: 8
cpu: host,hidden=1
efidisk0: local-zfs:vm-102-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:03:00.0,pcie=1,romfile=Navi31.rom
ide2: local:iso/virtio-win-0.1.266.iso,media=cdrom,size=707456K
machine: pc-q35-9.0
memory: 16000
meta: creation-qemu=9.0.2,ctime=1737980494
name: Win11VM-Klon
net0: virtio=BC:24:11:9F:76:BF,bridge=vmbr0,firewall=1
numa: 0
ostype: win11
scsi0: local-zfs:vm-102-disk-1,cache=writeback,discard=on,iothread=1,size=200G
scsihw: virtio-scsi-single
smbios1: uuid=277169be-7f88-4665-8307-4e565d85349c
sockets: 1
tpmstate0: local-zfs:vm-102-disk-2,size=4M,version=v2.0
usb0: host=6-4,usb3=1
usb1: host=3-5.1,usb3=1
usb2: host=3-5.1,usb3=1
usb4: host=5-6,usb3=1
vga: none
vmgenid: 123d8b5f-1c6a-470d-8806-f2872c6d9ff8
 
  • Like
Reactions: mr44er
Dear Community,

Currently, I am experiencing an issue with my Asus WRX80 motherboard and a passed-through GPU. My system was set up and ran smoothly for several months. Unfortunately, after some tests, my passed-through Asus RX 7900 XTX is no longer recognized, and I am getting the error code 43. Extensive searching and reinstallation on two VMs have not resolved the issue. Can anyone help me with this? Has anyone experienced this before?
 
I reinstalled the AMD driver several times under Windows 11. Allways error 43. The GPU is pass-through and can work. However, the AMD driver is not recognized, error 43. Bmd Resolve Must have the amd driver to work. Any idear?
 
vigorianer said:
machine: pc-q35-9.0
Click to expand...
I had AMD Windows driver issues in the past (when using AMD GPUs that reset fine or with vendor-reset) with Windows 10 (while Linux VMa worked fine) and using an older Machine Version fixed it. Instead of using the latest (at the time)pc-q35-8.x, I used pc-q35-7.2 (or selecting 7.2 for Machine Version in the Proxmox web GUI). I don't know how well Windows responds to such changes but maybe you could try?
 
You must log in or register to reply here.
Top Bottom