GPU Passthrough issues after upgrade from PVE8-PVE9

[syphondex]

I appreciate that this is a common issue, but every fix i've tried from both reddit and the proxmox support forums doesn't appear to be working.

Issue: GPU passthrough of a P2000 Quadro was working fine prior to an in place upgrade from PVE 8-9, VM boots. If i assign a GPU and boot the VM it immediately crashes the Host which all searches appear at first blush to indicate an Iommu issue, but those fixes don't appear to be work. Tearing my hair out here, even though i'm sure it's probably something simple. I'm not super new to proxmox but certainly not used to getting this deep into the guts. Any help would be greatly appreciated.

Iommu shows no conflicts

/sys/kernel/iommu_groups/60/devices/0000:ff:1f.0
/sys/kernel/iommu_groups/60/devices/0000:ff:1f.2
/sys/kernel/iommu_groups/6/devices/0000:82:00.0
/sys/kernel/iommu_groups/6/devices/0000:82:00.1
/sys/kernel/iommu_groups/7/devices/0000:83:00.0
/sys/kernel/iommu_groups/7/devices/0000:83:00.1

relevant lspci entries

82:00.0 VGA compatible controller: NVIDIA Corporation GP106GL [Quadro P2000] (rev a1)82:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio Controller (rev a1)

CMDline

root@zeus:~# cat /proc/cmdlineBOOT_IMAGE=/boot/vmlinuz-6.14.11-4-pve root=/dev/mapper/pve-root ro quiet mitigations=off intel_iommu=on initcall_blacklist=sysfb_initroot@zeus:~#

PVEVersion

root@zeus:~# pveversion pve-manager/9.0.11/3bf5476b8a4699e2 (running kernel: 6.14.11-4-pve)root@zeus:~#

 

[syphondex]

DMesg also doesn't show the RPool errors that most report as part of this issue.
VM config below - note that the gpu currently isn't being passed so i could get it running.
I will also add that I've created a new VM, assigned the GPU and immediately crashed the host on VM boot, so clearly not the VM configuration, but here it is anyway.

root@zeus:~# cat /etc/pve/qemu-server/101.conf
agent: 1
balloon: 0
boot: order=scsi0;ide2;net0
cores: 4
ide2: none,media=cdrom
machine: q35
memory: 65536
name: PlexServer
net0: virtio=CE:92:58:35:40:BD,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: PStorage:101/vm-101-disk-0.raw,size=240G
scsi1: PStorage:101/vm-101-disk-1.qcow2,size=16G
scsi2: PStorage:101/vm-101-disk-2.qcow2,size=32G
scsihw: virtio-scsi-pci
smbios1: uuid=7aec2596-3e16-494a-96b6-2b001b727f9b
sockets: 2
startup: order=3,up=60
tags: linux
vmgenid: 1619db19-da90-4829-95ee-7e434bf5b248
root@zeus:~#