[SOLVED] GPU Passthrough Issues After Upgrade to 7.2
- Thread starter nick.kopas
- Start date
If you have multiple GPUs, you might not need this work-around. It is needed because the passthrough GPU is used during POST/boot, typically because it is the only GPU. Can't you make the system POST/boot with one of the other GPUs (either via the BIOS or by switching the PCIe slots)?
Last edited:
I do have a boot GPU specified in BIOS, but this issue affected my system anyways. I have the hookscript setup working well for me, but I have been keeping an eye out for a more "persistent" change. Not sure I want to kill my access to boot logging though.
That is unexpected. Can you tell me more about which GPUs you have and which one is used for POST/boot/console? Maybe we can find a better/more specific work-around.
Host OS GPU (console, post, etc.): ASPEED Graphics Controller (rev41)
Passthrough GPU: NVIDIA GeForce GP106 (GTX1060 6GB)
Passthrough GPU: NVIDIA GeForce GP106 (GTX1060 6GB)
Did you early bind the GTX1060 to vfio-pci (using ids=... and maybe a soffdep), to prevent it from being used by Proxmox?
Under Host Configuration that wiki page shows how to add
options vfio-pci ids=... to a file in /etc/modprobe.d/ (vfio.conf for example) using the ids of your GTX1060 (use lspci -nn to look them up). If the nouveau driver is (still) loaded for the GPU (check with lspci -nnk), you need to blacklist it or use a softdep.This should prevent Proxmox (i.e., the Linux kernel) from touching the GPU before starting the VM, which should prevent the need for a work-around for that GPU.
Last edited:
Ah, I did end up doing that. Went through my configs and found them as expected. However, with lscpi -nnk are we only concerned with the kernel driver in use, or also the kernel modules? Because as of now I do have the nouveau and nvidia drivers blacklsited and the kernel driver in use is vfio-pci.
Yes, the in-use one. If that's just after a reboot of the Proxmox host and before starting the VM, then that's fine. And I don't understand why you need additional work-arounds before starting the VM. (I also hope you did not install the proprietary NVidia drivers on the Proxmox host.)
I sure did not install any drivers on the Proxmox host. Only changes I've made to the install are the steps to enable pcie passthrough.
I will verify again next time I can reboot the host, but for what its worth all of this was set up and working before the 5.15 kernel. I only had to start using the hookscript to get passthrough working again under 5.15.
I will verify again next time I can reboot the host, but for what its worth all of this was set up and working before the 5.15 kernel. I only had to start using the hookscript to get passthrough working again under 5.15.
Then there should be no driver named
nvidia that you need to blacklist.The most recent 5.15 kernels are better in some respects. With my AMD GPU use for POST?boot messages, I don't even need to blacklist, early bind or work-around the boot display. The amdgpu driver handles it for me. but I did have issues with the early 5.15 kernels. Check
cat /proc/iomem for BOOTFB. That's the typical cause of the BAR can't reserve memory errors and usually only for GPU's used for POST/boot/console. I just don't get what is causing the problem that you need the work-around for.EDIT: Did you run
update-initramfs -u and update-grub or proxmox-boot-tool refresh to activate changes to /etc/modprobe.d/. Note that the GTX 1060 should not display anything before starting the VM, otherwise something is wrong.
Last edited:
No BOOTFB in /proc/iomem. And I did run the update-initramfs -u and proxmox-boot-tool refresh. As stated, it worked fine before the 5.15 branch. I am still on 5.15.35-6, so maybe I need to update again and try removing the hookscript and see what happens.
As an update:
Updated to the 5.15.39-1 kernel this morning and disabled the hookscript. Upon reboot the VM started without issue, and I verified passthrough was behaving as expected. So it looks like at some point a kernel update resolved the issue causing the framebuffer to improperly reserve the secondary GPU.
Updated to the 5.15.39-1 kernel this morning and disabled the hookscript. Upon reboot the VM started without issue, and I verified passthrough was behaving as expected. So it looks like at some point a kernel update resolved the issue causing the framebuffer to improperly reserve the secondary GPU.
I use below sources.list and command 'apt-get upgarde'
deb http://ftp.hk.debian.org/debian bullseye main contrib
deb http://ftp.hk.debian.org/debian bullseye-updates main contrib
deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription
# security updates
deb http://security.debian.org bullseye-security main contrib
I cannot upgrade to 5.15.39-1 kernel.
Please tell how to upgarde the kernel.
Thanks
Last edited:
Never run
apt-get upgrade. Run apt-get update and then apt-get dist-upgrade. Click the manual for more information.Thanks
After I upgrade to 5.15.39-1 kernel, I still cannot solve the passthrough GPU issue. The config worked in 7.1
kvm: -device vfio-pci,host=00:02.0,addr=0x02,x-igd-gms=1,romfile=/root/vbios.rom: Failed to mmap 0000:00:02.0 BAR 2. Performance may be slow
kvm: -device vfio-pci,host=00:02.0,addr=0x02,x-igd-gms=1,romfile=/root/vbios.rom: IGD device 0000:00:02.0 cannot support legacy mode due to existing devices at address 1f.0
TASK OK
Last edited: