[SOLVED] GPU Passthrough Issues After Upgrade to 7.2

I let it boot without any cron scripts running.
lspci -v shows that it bound correctly to amdgpu.

Following dmesg, the card got initialized without errors by amdgpu.
iomem shows nothing about BOOTFB, so this looks good I assume.

When i start the VM, both gpu and its audio device show kernel driver vfio.
The screen just turns black after the boot screen.

Using journalctl -b 0 I found these errors during VM startup:

amdgpu: cp queue pipe 4 queue 0 preemption failed
kernel: amdgpu 0000:09:00.0: amdgpu: Fail to disable thermal alert!
 
*maybe* this is windows related.
As i said, I am (sometimes) able to get a screen and enter Windows using the GPU. It just doesnt load with "code 43".

For fun i added another VM and put in the Ubuntu 22.04 desktop installer.
Out of the box, even in the installer, the gpu is picked up and 4K resolution is applied.

I'm currently testing if gaming and benchmarks work - so the GPU is *really* fully available.
If that does, maybe spinning up a fresh windows 11 (i use win10 usually) gaming vm helps.
 
Windows 11 also throws Code 43 error.
On Ubuntu 22.04 I am able to use 2 monitors at 4K, watch videos and play games via Steam & Proton.
 
Oh my god. I got it working.
The Proxmox Wiki already contained everything i needed to know.

https://pve.proxmox.com/wiki/Pci_passthrough - it specifically mentions code 43 when Resizable BAR / Smart Access Memory is enabled on the Host.
VMs do not support it yet.

Disabled it, and now it is working kinda out of the box.
Have to verify with benchmarks, but dual screen, 4K resolution, no code43 in devicemgr.... it looks pretty darn good!

EDIT: tested various games and ran timespy benchmarks. it works excellent.
I'm curious why it did not fail on Ubuntu with SAM enabled, though.
 
Last edited:
  • Like
Reactions: nick.kopas and leesteken
leesteken said:
/
Click to expand...

hardwareadictos said:
Hey guys, just to point my situation here.

I have two proxmox servers both with GPU passthrough enabled. A GT-710 which didn't need any tricky procedure to work on Proxmox 7.2 and v5.15 kernel and another GTX-970 on another system which didn't work with any method posted here:

Just made a script:
#!/bin/bash
echo 1 > /sys/bus/pci/devices/0000\:08\:00.0/remove
echo 1 > /sys/bus/pci/rescan

Which is executed by cron at boot:

@reboot /scripts/gpu/gpu_reset.sh

That's my GRUB CMDLINE:

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt vfio_iommu_type1.allow_unsafe_interrupts=1 video=efifb:eek:ff video=vesafb:eek:ff mpt3sas.max_queue_depth=10000"

The fact is that if i execute it with system booted it reserves the GPU properly, if i enable cron execution the system just freezes some seconds rear booting. Despite off, my Windows 11 machine which has that GPU on passthrough doesn't recognise the GPU properly on any case.

Those issues are solved (like other users) pinning 5.13 kernel...

Cheers.
Click to expand...
And thats what happens when i run the rescan script manually and i boot the vm:

1654964190244.png

This ends on a Proxmox host freeze.

For me the only way of making my GTX 970 working is to rollback to 5.13 kernel.

@WesC did you find any improvements on the mainline kernel?
 
Last edited:
I wanted to chime in as a newcomer to GPU passthrough, after 2 days of messing around I got a GTX 960 passed through to an EFI Windows box. Need to vent this overwhelming joy to somewhere : )

Using 5.15.35-1-pve kernel, the host is booted in full UEFI (no CSM). The PCI rebind does the trick.

I'm doing this on an AMD box with no other GFX cards, and I needed to:
- Patch the VBIOS, downloaded from TechBIOS, following these steps How to easily passthough a Nvidia GPU as primary without dumping your own vbios! in KVM unRAID - YouTube. Be sure to get one that has matching PCI IDs, the subsystem ID does not seem to matter.
- upload the patched rom to /usr/share/kvm on the Proxmox host
- add the patched rom to the card by editing /etc/pve/qemu-server/<vmid>.conf:
hostpci0: 0000:01:00,pcie=1,x-vga=1,romfile=filename.rom
- Kernel command line: amd_iommu=on iommu=pt video=efifb:off video=vesafb:off video=simplefb:off (Not sure that all of them are required, but everything works at the moment so not changing anything for a while)
- Latest NVIDIA drivers in the guest
- Do the PCI rebinding manually, for some reason the hook script from @nick.kopas did not work for me. I guess adding a sleep 1 to the script might help, but for me the manual rebind is not a big deal.
- After this the HDMI port works. DisplayPorts do not work, haven't tested the DVI. If I had realized to try all the ports, this might have saved me quite a bit of time.
 
  • Like
Reactions: hardwareadictos
oranki said:
I wanted to chime in as a newcomer to GPU passthrough, after 2 days of messing around I got a GTX 960 passed through to an EFI Windows box. Need to vent this overwhelming joy to somewhere : )

Using 5.15.35-1-pve kernel, the host is booted in full UEFI (no CSM). The PCI rebind does the trick.

I'm doing this on an AMD box with no other GFX cards, and I needed to:
- Patch the VBIOS, downloaded from TechBIOS, following these steps How to easily passthough a Nvidia GPU as primary without dumping your own vbios! in KVM unRAID - YouTube. Be sure to get one that has matching PCI IDs, the subsystem ID does not seem to matter.
- upload the patched rom to /usr/share/kvm on the Proxmox host
- add the patched rom to the card by editing /etc/pve/qemu-server/<vmid>.conf:
hostpci0: 0000:01:00,pcie=1,x-vga=1,romfile=filename.rom
- Kernel command line: amd_iommu=on iommu=pt video=efifb:off video=vesafb:off video=simplefb:off (Not sure that all of them are required, but everything works at the moment so not changing anything for a while)
- Latest NVIDIA drivers in the guest
- Do the PCI rebinding manually, for some reason the hook script from @nick.kopas did not work for me. I guess adding a sleep 1 to the script might help, but for me the manual rebind is not a big deal.
- After this the HDMI port works. DisplayPorts do not work, haven't tested the DVI. If I had realized to try all the ports, this might have saved me quite a bit of time.
Click to expand...
THANK YOU

Just managed to get my GTx 970 working with step 4 and the hookscript for rebinding, for some reason putting it on cron didn't work. @nick.kopas hookscript attatched to my vm worked fine :)
 
nick.kopas said:
Nope, I'm guessing that's not on the enterprise repository? Are you running single GPU passthrough?

I'm also testing a fix for this issue. So, I'm probably going to just leave my current kernel in place for now.
Click to expand...
Yes I’m running a single 3080FE. I did initially have this error. But I also changed my processor from a 3900X to a 5700X - which is the only change.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom