Nvidia 5000 series passthrough , failing miserably.

[adolfotregosa]

I'm trying..... My 4060ti 8GB works perfectly but I wanted to have 16GB of vram so I got the 5060ti 16GB.
I have never had an nvidia that I could not passthough from 10XX, 20xx, 30xx and 40xx but I cannot with this 5060ti. With original vbios, modded vbios. uefi...csm... The monitor turns on but I get no image. SSHing into the VM the nvidia driver is loaded but without output. nvidia-smi confirms no monitor detected. I've yet to see the ovmf bios screen !
Tried proxmox kernel 6.8, 611 and 6.14, 4G decoding and rebar on/off in bios, no dice.

Surely I'm not alone... am I ?

My hookscript still brings back proxmox cli when I stop the VM. I even try enabling the iGPU and leaving the 5060TI alone, no dice. Monitor "turns on" but I get no image.

Any ideas??

 

[adolfotregosa]

Well, progress but it's a compromise. I'm using linux guest btw.
I'm not able to see tianocore bios screen so you have to prepare the vm using some virtual gpu first and install the nvidia driver.

You have to untick rombar when adding the gpu!!! As soon the nvidia driver loads inside the VM, image will show up.
I'm able to recover proxmox terminal after vm shutdown using hookscript to unbind from vfio-pci and rebind nvidia. (And also to unload the nvidia driver in proxmox before the vm starts).

The most important option is to load nvidia_drm with modeset=1 on both proxmox and the linux vm!

so put in some conf file in e.g. /etc/modprobe.d/nvidia.conf on both proxmox and your linux guest:

options nvidia_drm modeset=1

and update your initramfs, e.g. update-initramfs -u -k all

Unfortunately this is not 100% reliable. Sometimes the machine freezes when powering off the VM. Some reset problem OR the nvidia driver is still very problematic for the 5000 series making all sort of problems.