VM Freezes on loading when passing through GPU (unless display set to none)

[triple07]

Hello all, newbie looking for some advice after running out of ideas through Googling and searching the forums. Hopefully someone is willing/able to assist. I've been trying to convert my old gaming rig over to a Proxmox server for homelabbing and mostly learning virtualization. Specs are:

• Ryzen 7800X3D
• Gigabyte X670 Aorus Elite AX
• 64 GB RAM
• ASRock Intel A380 Challenger ITX

Basically bought a cheap Intel A380 to add to an existing build with the thought of using it for transcoding in a PLEX/Jellyfin server or just adding some processing power to a VM.
Anyway on to the situation: I have read various official and unofficial guides to getting PCI passthrough up and running and can passthrough individual NVMe drives with no issue. When I try to pass through the GPU I get issues. Whenever I load the VM with the GPU it never gets any further than the initial logo loading screen. Appears to freeze as shutdown signal doesn't work either. If I remove the GPU, the VM boots fine no issues. Tried using multiple Linux distros including Ubuntu, Fedora, Linux Mint all with the same outcome.

After troubleshooting for hours I came to realize that the VM would start just fine if the GPU was passed AND display was set to none. This would indeed pass the GPU through as I was able to confirm via connecting to the GPU directly. Only problem is I have no way to actually control or connect to the VM remotely. Tried every display option and none is the only option that doesn't freeze the VM.

Been following the guide here: https://forum.proxmox.com/threads/p...x-ve-8-installation-and-configuration.130218/ and here: https://pve.proxmox.com/wiki/PCI_Passthrough

Output from dmesg commands listed in guides:

dmesg | grep -e DMAR -e IOMMU

[    1.541831] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    1.611434] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).

dmesg | grep 'remapping'

[    1.543134] AMD-Vi: Interrupt remapping enabled

dmesg | grep -i vfio

[    6.911210] VFIO - User Level meta-driver version: 0.3
[    6.915320] vfio-pci 0000:03:00.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=io+mem:owns=none
[    6.915422] vfio_pci: add [8086:56a5[ffffffff:ffffffff]] class 0x000000/00000000
[    6.915517] vfio_pci: add [8086:4f92[ffffffff:ffffffff]] class 0x000000/00000000

I have also verified that the GPU is on its own IOMMU group, 2 actually, Arc A380 is IOMMU group 15 and the audio controller is 16. No other devices share these groups.

Fedora VM Settings:


Anyone have any suggestions as to where to go from here? My only other thought is that I could install a remote desktop application on the VM then set the display to none and remote in that way, however, I was hoping to use something built in to Proxmox.

Thanks for any assistance and let me know if any more details are needed.

 

[zenowl77]

maybe if the A380 is your only GPU do not use VirtIO gpu as the display option seeing as that should require use of the GPU you are passing through?

i have this exact same gpu and i never have an issue with display options except virtio and virtgl specifically, have you tried just using default?

 

[triple07]

maybe if the A380 is your only GPU do not use VirtIO gpu as the display option seeing as that should require use of the GPU you are passing through?

i have this exact same gpu and i never have an issue with display options except virtio and virtgl specifically, have you tried just using default?

Unfortunately it still freezes with default selected as display. Good to know the GPU can work for PCI passthrough. Are you using an Intel or AMD CPU?

 

[zenowl77]

well, that sucks. wonder why it isn't passing well...

mine is an intel, i7-7820x to be exact, no igpu either i am actually using the arc as a primary gpu with tesla p4 secondary and it still passes fine although with mine i had to remove it from the bus and rescan it so the vm could take it.
what i had to run to pass it to a vm as the primary gpu: (but this was to pass it at all)
echo 1 > /sys/bus/pci/devices/0000\:67\:00.0/remove
echo 1 > /sys/bus/pci/devices/0000\:67\:00.0/rescan