If you have multiple GPUs, you might not need this work-around. It is needed because the passthrough GPU is used during POST/boot, typically because it is the only GPU. Can't you make the system POST/boot with one of the other GPUs (either via the BIOS or by switching the PCIe slots)?And there's no way to disable per GPU?
That is unexpected. Can you tell me more about which GPUs you have and which one is used for POST/boot/console? Maybe we can find a better/more specific work-around.I do have a boot GPU specified in BIOS, but this issue affected my system anyways. I have the hookscript setup working well for me, but I have been keeping an eye out for a more "persistent" change. Not sure I want to kill my access to boot logging though.
Under Host Configuration that wiki page shows how to add
options vfio-pci ids=...to a file in /etc/modprobe.d/ (vfio.conf for example) using the ids of your GTX1060 (use
lspci -nnto look them up). If the nouveau driver is (still) loaded for the GPU (check with
lspci -nnk), you need to blacklist it or use a softdep.
Yes, the in-use one. If that's just after a reboot of the Proxmox host and before starting the VM, then that's fine. And I don't understand why you need additional work-arounds before starting the VM. (I also hope you did not install the proprietary NVidia drivers on the Proxmox host.)Ah, I did end up doing that. Went through my configs and found them as expected. However, with lscpi -nnk are we only concerned with the kernel driver in use, or also the kernel modules? Because as of now I do have the nouveau and nvidia drivers blacklsited and the kernel driver in use is vfio-pci.
Then there should be no driver namedI sure did not install any drivers on the Proxmox host. Only changes I've made to the install are the steps to enable pcie passthrough.
nvidiathat you need to blacklist.
The most recent 5.15 kernels are better in some respects. With my AMD GPU use for POST?boot messages, I don't even need to blacklist, early bind or work-around the boot display. The amdgpu driver handles it for me. but I did have issues with the early 5.15 kernels. CheckI will verify again next time I can reboot the host, but for what its worth all of this was set up and working before the 5.15 kernel. I only had to start using the hookscript to get passthrough working again under 5.15.
BOOTFB. That's the typical cause of the
BAR can't reserve memoryerrors and usually only for GPU's used for POST/boot/console. I just don't get what is causing the problem that you need the work-around for.
proxmox-boot-tool refreshto activate changes to /etc/modprobe.d/. Note that the GTX 1060 should not display anything before starting the VM, otherwise something is wrong.
I found this too - and I'm really itching to understand why.As an update:
Updated to the 5.15.39-1 kernel this morning and disabled the hookscript. Upon reboot the VM started without issue, and I verified passthrough was behaving as expected. So it looks like at some point a kernel update resolved the issue causing the framebuffer to improperly reserve the secondary GPU.
I use below sources.list and command 'apt-get upgarde'I found this too - and I'm really itching to understand why.
Never runI use below sources.list and command 'apt-get upgarde'
deb http://ftp.hk.debian.org/debian bullseye main contrib
deb http://ftp.hk.debian.org/debian bullseye-updates main contrib
deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription
# security updates
deb http://security.debian.org bullseye-security main contrib
I cannot upgrade to 5.15.39-1 kernel.
Please tell how to upgarde the kernel.
apt-get upgrade. Run
apt-get updateand then
apt-get dist-upgrade. Click the manual for more information.