5900HX with Radeon Graphics iGPU Passthrough to VM

[MisterDeeds]

Hello all

I am trying to passthrough an iGPU from an AMD Ryzen 9 5900HX to a VM. To do this, I proceeded as follows:

vi /etc/kernel/cmdline
    root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet amd_iommu=on iommu=pt textonly video=astdrmfb pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:eek:ff video=efifb:eek:ff video=simplefb:eek:ff initcall_blacklist=sysfb_init disable_vga=1 vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1 modprobe.blacklist=radeon,nouveau,nvidia,nvidiafb,nvidia-gpu,snd_hda_intel,snd_hda_codec_hdmi,i915,amdgpu,ccp

proxmox-boot-tool refresh

reboot

dmesg | grep -e DMAR -e IOMMU
    [    0.000000] Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA
    [    0.402727] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
    [    0.403245] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
    [    0.716364] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).

vi /etc/modules
    vfio
    vfio_iommu_type1
    vfio_pci
    vfio_virqfd

lspci -nnv | grep VGA
    04:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [1002:1638] (rev ff) (prog-if ff)

echo "options vfio-pci ids=1002:1638 disable_vga=1" > /etc/modprobe.d/vfio.conf

echo "options kvm ignore_msrs=1 report_ignored_msrs=0" > /etc/modprobe.d/kvm.conf

update-initramfs -u -k all

reboot

Finally, I added the PCI device.



The device also has its own IOMMU Group ID:


The VM boots, but it is very slow. Operation is not possible. I send you the dmesg output in the attachment.

Does anyone have any ideas?

Thank you and best regards

 

[leesteken]

You are already throwing everything at the kernel parameters (including lots that you don't need or are invalid), so I don't know what would help (if they are all active, check with cat /proc/cmdline).

According to the log: vfio-pci 0000:04:00.7: can't change power state from D0 to D3hot (config space inaccessible). Don't use passthrough All Functions, because the VGA function04:00.0 and the audio function 04:00.1 are all you need. The other functions 04:00.2 and up are giving you problems.
Try passing through 04:00.0 only first, and maybe add 04:00.1 as another device.

EDIT: I only know generic things about passthrough and no nothing about your specific integrated graphics or Windows.

 

[MisterDeeds]

Hello and thank you for the answer. You are right, it was displayed that the PCI address 0000:04:00.1 only was passed through, but effectively all PCI devices of address 0000:04:00 were passed through.



I have now added the GPU and the audio card via conf file. This now also works and the VM starts as it should.

Spoiler: conf

agent: 1
bios: ovmf
boot: order=scsi0;net0;ide0
cores: 4
cpu: host,hidden=1
efidisk0: SSD:vm-200-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:04:00.0,pcie=1
hostpci1: 0000:04:00.1,pcie=1
ide0: none,media=cdrom
machine: pc-q35-7.0
memory: 8192
meta: creation-qemu=7.0.0,ctime=1667918978
name: PcWin01
net0: virtio=36:94:94:C4:76:12,bridge=vmbr0,firewall=1
numa: 1
onboot: 1
ostype: win10
scsi0: SSD:vm-200-disk-1,cache=writeback,discard=on,size=50G,ssd=1
scsihw: virtio-scsi-pci
smbios1: uuid=0db1591c-b956-467e-82cf-7fb49f40d20c
sockets: 1
vga: none
vmgenid: 7d095607-7342-44c4-be09-99d617d9fa68

Unfortunately, the error message 43 is displayed in the device manager.



According to this post https://forum.proxmox.com/threads/h...-r9-5900hx-igpu-in-pve-7-2.109572/post-477856 it is due to the drivers. However, I am already using the latest drivers from the AMD website.

Is there a solution for this?

 

[Pandaaaa906]

Hello and thank you for the answer. You are right, it was displayed that the PCI address 0000:04:00.1 only was passed through, but effectively all PCI devices of address 0000:04:00 were passed through.

View attachment 43065

I have now added the GPU and the audio card via conf file. This now also works and the VM starts as it should.

Spoiler: conf

agent: 1
bios: ovmf
boot: order=scsi0;net0;ide0
cores: 4
cpu: host,hidden=1
efidisk0: SSD:vm-200-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:04:00.0,pcie=1
hostpci1: 0000:04:00.1,pcie=1
ide0: none,media=cdrom
machine: pc-q35-7.0
memory: 8192
meta: creation-qemu=7.0.0,ctime=1667918978
name: PcWin01
net0: virtio=36:94:94:C4:76:12,bridge=vmbr0,firewall=1
numa: 1
onboot: 1
ostype: win10
scsi0: SSD:vm-200-disk-1,cache=writeback,discard=on,size=50G,ssd=1
scsihw: virtio-scsi-pci
smbios1: uuid=0db1591c-b956-467e-82cf-7fb49f40d20c
sockets: 1
vga: none
vmgenid: 7d095607-7342-44c4-be09-99d617d9fa68

Unfortunately, the error message 43 is displayed in the device manager.

View attachment 43063

According to this post https://forum.proxmox.com/threads/h...-r9-5900hx-igpu-in-pve-7-2.109572/post-477856 it is due to the drivers. However, I am already using the latest drivers from the AMD website.

Is there a solution for this?

i was temporary successed
the key is to acquire vbios file from your motherboard bios
copy to the host
modify the vm settings

it worked, it has reset problem(u cant individually reboot that vm, which will cause whole host break down), but it worked

unfortunately it started to fail couple months ago, might be caused by kernel updates, still figuring why...

 

[MisterDeeds]

i was temporary successed
the key is to acquire vbios file from your motherboard bios
copy to the host
modify the vm settings

it worked, it has reset problem(u cant individually reboot that vm, which will cause whole host break down), but it worked

unfortunately it started to fail couple months ago, might be caused by kernel updates, still figuring why...

Hi and thanks for the reply. Do you have the vBIOS file right now that you could share with us?

Thanks and best regards

 

[viumden]

Hi and thanks for the reply. Do you have the vBIOS file right now that you could share with us?

Thanks and best regards

Try the link: https://1drv.ms/u/s!AirGtsBE4ge4iJJu3MrKG1g6bS7MTg

 

[Pandaaaa906]

Hi and thanks for the reply. Do you have the vBIOS file right now that you could share with us?

Thanks and best regards

***WARNING IM NOT SURE THAT MY VBIOS WOULD WORK FOR U OR NOT***
i extract it from my device bios, my device is Morefine s500+

decompress and copy to /usr/share/kvm/vbios_1638.dat
setup win10 vm, config rdp stuffs, close vm
add pci device to vm, disable default pve display at pve manage page
edit vm settings
... cpu: host, hidden=1 hostpci0: .....,romfile=vbios_1638.dat ...
boot vm and wish you good luck ;D

 

[MisterDeeds]

Dear viumden, dear Pandaaaa906

Thank you for your effort and feedback. Unfortunately it did not work for me with both vBIOS files (still code 43)... I use the latest driver adrenalin-edition-22.11.2

Maybe someone else had already success with it and can share their experience

Thanks anyway and best regards

 

[Pandaaaa906]

Dear viumden, dear Pandaaaa906

Thank you for your effort and feedback. Unfortunately it did not work for me with both vBIOS files (still code 43)... I use the latest driver adrenalin-edition-22.11.2

Maybe someone else had already success with it and can share their experience

Thanks anyway and best regards

u might have the pci settings wrong

【PVE7 AMD 5700G 核显直通 (iGPU Passthrough)-哔哩哔哩】 https://b23.tv/xcrFq1F

u could check this video
it's chinese but might help

 

[MisterDeeds]

Hello together

Thank you very much for the inputs. I have a Minis forum Mini PC HX90 (Ryzen 9 5900HX). Unfortunately I can not get it to run in any way. Also I do not manage to export the correct ROM file... the only ROM file I can find is from another PCI device




unfortunately it does not work with the uploaded ROMs.

 

[0ngogoblo]

I've been having a hell of a time trying to get GPU passthrough on my Minisforum HX90 to my linux VM. I can't even get the VM working right with passthrough on.
Doesn't even seem worth it at this point unless someone smarter than me can come up with a complete working guide lol

 

[awaitz]

I've been super happy with the Minisforum HX90 --- Ryzen 9 5900HX + 6600M GPU -- and we've been playing games (via emulation) from all sorts of consoles we've owned over the years from early Nintendo days through PS3 and Switch. Now have my heart set on a virtualized setup with GPU pass-through. We want to share the GPU over the network to other machines in the house and at other times we just want to use this machine to play games or watch movies on a large screen TV. The main reason to switch to a virtualized setup is various complex and often contradictory software limitations. If I get anywhere, I'm happy to share. In the end, I might buy a second machine and just do what we want with twice the hardware.

 

[patrickgraham]

I've been super happy with the Minisforum HX90 --- Ryzen 9 5900HX + 6600M GPU -- and we've been playing games (via emulation) from all sorts of consoles we've owned over the years from early Nintendo days through PS3 and Switch to play NDS ROMS. Now have my heart set on a virtualized setup with GPU pass-through. We want to share the GPU over the network to other machines in the house and at other times we just want to use this machine to play games or watch movies on a large screen TV. The main reason to switch to a virtualized setup is various complex and often contradictory software limitations. If I get anywhere, I'm happy to share. In the end, I might buy a second machine and just do what we want with twice the hardware.

I appreciate your ideas