GPU PCIe Passthrough stopped working after update

[TorqueWrench]

It's a problem many of us (maybe all of us?) ran into with Windows 10 guests and Nvidia GPUs. You need to boot into safe mode and enable MSI to fix it:

Fixing GPU Passthrough On Windows 10 VM In Proxmox

 

[wpowiertowski]

That prevents Windows guest from using the GPU as it's primary GPU

Plus like I mentioned above - this has b

Try changing the line

machine: q35

to

machine: pc-q35-3.1

and leave x-vga=1 and vga: none as you had them (you technically didn't have vga: none, you just ommitted it which is the same thing).

If that doesn't do it, add the machine: pc-q35-3.1 line and add the following line at the end of the XXX.conf file:

args: -machine type=q35,kernel_irqchip=on

Thanks for more suggestions, but nope still the same result

> cat /etc/pve/qemu-server/107.conf
agent: 1
bootdisk: sata0
cores: 10
cpu: host
hostpci0: 65:00,pcie=1,romfile=vbios.bin,x-vga=1
ide2: none,media=cdrom
machine: pc-q35-3.1
memory: 24576
name: Windows10
net0: e1000=CE:B6:D8:EC:F1:E5,bridge=vmbr0
net1: virtio=A6:FA:FB:8E:7F:43,bridge=vmbr1
numa: 0
onboot: 1
ostype: win10
sata0: tank:vm-107-disk-0,size=150G,ssd=1
sata1: tank:vm-107-disk-1,size=200G,ssd=1
scsihw: virtio-scsi-pci
smbios1: uuid=17efdf00-377c-4ac4-b337-3ff7c4fe1d79
sockets: 1
usb0: host=1-3.1.2.4,usb3=1
vmgenid: 74cbd8bf-841a-4b27-b2ca-07cecfaffd53
args: -machine type=q35,kernel_irqchip=on

Same error message:

kvm: -device vfio-pci,host=0000:65:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,x-vga=on,multifunction=on,romfile=/usr/share/kvm/vbios.bin: vfio 0000:65:00.0: failed getting region info for VGA region index 8: Invalid argument
device does not support requested feature x-vga

 

[Joris L.]

That prevents Windows guest from using the GPU as it's primary GPU

Which is in-line with
vfio 0000:65:00.0: failed getting region info for VGA region index 8: Invalid argument
device does not support requested feature x-vga

 

[wpowiertowski]

It's a problem many of us (maybe all of us?) ran into with Windows 10 guests and Nvidia GPUs. You need to boot into safe mode and enable MSI to fix it:

Fixing GPU Passthrough On Windows 10 VM In Proxmox

That looked promising - but no-go for me - didn't change anything on the Host or Guest side.

Without x-vga=1 my Windows guest boots just fine (as it did before) but doesn't use the GPU for display output.
With x-vga=1 i get KVM failure, it doesn't even attempt Guest boot.

Something in the last Proxmox update just broke KVM here.

 

[Joris L.]

That looked promising - but no-go for me - didn't change anything on the Host or Guest side.

Without x-vga=1 my Windows guest boots just fine (as it did before) but doesn't use the GPU for display output.
With x-vga=1 i get KVM failure, it doesn't even attempt Guest boot.

Something in the last Proxmox update just broke KVM here.

It is demanding to get it working but i can say nothing is broken, at least not for my system. This machine is running on Proxmox VE 6.2 since about one month. GPU passthrough is quite demanding in my experience but if it works, it works very close to native speed.

Staring a VM from the shell using qm start <vmidhere> does often return valuable information as to why a VM does not start. It cost me a lot of extra work spending time on chasing problems in configuration while it was the vmid.conf file being altered (this happens when using multiple PCI devices)

Eventually i started using a Linux VM to debug passthrough and managed stop-start-shutdown for the VM from the shell/console. It clearly did show the device showing up when configured well.

 

[ekkzorzizten]

Hi! any progress? same thing happend to me. Tried reinstalling 6.1 again WITHOUT doing any updates and it works fine, but as soon as i update proxmox it stops working no matter what i do and when my windows VM is supposted to boot i instead get the error "vfio-pci 000:06:00.0: BAR 1 can´t reserve [mem 0x0000000-0xdfffffff 64bit pref]. Everything was working with only the steps mentioned in the guide: https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/

 

[Joris L.]

Hi! any progress? same thing happend to me. Tried reinstalling 6.1 again WITHOUT doing any updates and it works fine, but as soon as i update proxmox it stops working no matter what i do and when my windows VM is supposted to boot i instead get the error "vfio-pci 000:06:00.0: BAR 1 can´t reserve [mem 0x0000000-0xdfffffff 64bit pref]. Everything was working with only the steps mentioned in the guide: https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/

Not sure what the big difference between 6.1 and 6.2 could be but kernel version and related modules. What is important is you understand your system. The many reported configuration parameters do not work or are required for all systems. I doubt gpu passtrhough would have worked out of the box on any system. Are you sure it worked before ?

If you ask me this is the minimum required for it to work well

iommu=pt <yourcpu>l_iommu=on video=efifb:off,vesafb:off pci=nobar where <yourcpu> is either amd or intel

with the required configuration file for vfio-pci under /etc/modprobe.d/ listing the ids=.... for pass-through

additionally, when using an Intel CPU only

/etc/modprobe.d/kvm.conf
options kvm ignore_msrs=1 report_ignored_msrs=0

Also, set the CPU to host, the chipset to Q35 and experiment with the GPU settings.

What i found is for my system i require a GPU reset script which causes the machine to suspend and manually resume to have the GPU availble for a new VM again.

 

[wpowiertowski]

I did eventually made it work but with a different configuration, for some reason the x-vga option seems broken in 6.2.
I had to reinstall my Win10 from scratch and provide it various drivers during installation to recognize the virtualized devices but it does work now with this config:

args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
bios: ovmf
bootdisk: scsi0
cores: 8
cpu: host,hidden=1,flags=+pcid
efidisk0: tank:vm-107-disk-1,size=1M
hostpci0: 65:00,pcie=1,romfile=gtx1650.rom
ide2: none,media=cdrom
machine: pc-q35-3.1
memory: 32768
name: Windows10
net0: virtio=8A:BF:B3:80:C9:E9,bridge=vmbr0
net1: virtio=62:16:E7:9A:81:6F,bridge=vmbr1
numa: 0
onboot: 1
ostype: win10
scsi0: tank:vm-107-disk-0,size=500G,ssd=1
scsihw: virtio-scsi-pci
smbios1: uuid=af1a0c73-dd95-4514-89d7-700cd246c0f7
sockets: 1
usb0: host=045e:07a5,usb3=1
vga: none
vmgenid: 375c08c9-d3ee-48b5-826b-c97b44064f9b

 

[scrapiron]

Changing vga: none is the last thing that made my GPU passthrough work on an external monitor. Thank you!