I have a workstation setup running Proxmox 8.1, with two GPUs (GTX 1630 and GTX 1080 ti). They are both physically connected to my monitor under two input ports (HDMI and DP), and I can switch between them seamlessly, for instance, allowing me to have a Windows 11 VM and Ubuntu VM running side-by-side.
In the last couple of months, I noticed that every time I tried to shutdown the Windows 11 VM attached to the GTX 1630, it froze. When I stopped the VM, I got these messages (09:00.0 => GTX 1630):
Each time I got a "not ready" message (as I was monitoring syslog), the host would temporarily freeze, but after a few seconds, it would completely hang and the only solution was to reboot the host. Not good.
I tried a few things that I found on forums, but with no avail. And then I realised that this was working before, so I pulled an older backup, and alas, it worked without any issues. So, after updating windows, I discovered that the Windows (guest) nvidia driver was the culprit. The old version: 531.41 (from March 2023) works without issues, but the latest ones do not.
I consider this to be solved because I am able to operate my VMs and host without any issues, but it is not ideal that I cannot update the nvidia driver on my windows machine, which may indicate an unsolved issue. Also, I am currently getting this message when I boot the Windows VM, but does not seem to have a negative effect:
I will investigate this further, but I just wanted to share my experience for those that are facing a similar issue, or for those that can shed some light on this.
---
My GRUB CMDLINE:
---
Okay, after reading the Proxmox PCI passthrough guide, there was the suggestion to disable CSM for GPU passthrough, which I did, and I haven't had the "no more image in the PCI ROM" error.
In the last couple of months, I noticed that every time I tried to shutdown the Windows 11 VM attached to the GTX 1630, it froze. When I stopped the VM, I got these messages (09:00.0 => GTX 1630):
Code:
vfio-pci 0000:09:00.0: not ready 1023ms after FLR; waiting
vfio-pci 0000:09:00.0: not ready 2047ms after FLR; waiting
vfio-pci 0000:09:00.0: not ready 4095ms after FLR; waiting
vfio-pci 0000:09:00.0: not ready 8191ms after FLR; waitingEach time I got a "not ready" message (as I was monitoring syslog), the host would temporarily freeze, but after a few seconds, it would completely hang and the only solution was to reboot the host. Not good.
I tried a few things that I found on forums, but with no avail. And then I realised that this was working before, so I pulled an older backup, and alas, it worked without any issues. So, after updating windows, I discovered that the Windows (guest) nvidia driver was the culprit. The old version: 531.41 (from March 2023) works without issues, but the latest ones do not.
I consider this to be solved because I am able to operate my VMs and host without any issues, but it is not ideal that I cannot update the nvidia driver on my windows machine, which may indicate an unsolved issue. Also, I am currently getting this message when I boot the Windows VM, but does not seem to have a negative effect:
Code:
Dec 22 11:16:45 work kernel: vfio-pci 0000:09:00.0: No more image in the PCI ROM
Dec 22 11:16:45 work kernel: vfio-pci 0000:09:00.0: No more image in the PCI ROMI will investigate this further, but I just wanted to share my experience for those that are facing a similar issue, or for those that can shed some light on this.
---
My GRUB CMDLINE:
Code:
GRUB_CMDLINE_LINUX="quiet nomodeset initcall_blacklist=sysfb_init video=vesafb:off video=efifb:off video=simplefb:off pcie_acs_override=downstream,multifunction amd_iommu=on vfio-pci.ids=1022:1487,1022:149c,10de:1b06,10de:10ef,10de:1f83,10de:10fa,1b21:1242"---
Okay, after reading the Proxmox PCI passthrough guide, there was the suggestion to disable CSM for GPU passthrough, which I did, and I haven't had the "no more image in the PCI ROM" error.
Last edited: