Kernel 6.8.12-12-pve Update results in VFIO PCI passthrough issues

T

THE_BIG_ONE

New Member
Jul 15, 2025
3
5
3
I'm running Proxmox on 6 home PCs all with PCIE pass-through. This morning I just updated to the latest kernel and other proxmox updates that were released today. All systems were fine except for one older Pre-Ryzen based PC.

The pre-ryzen PC had several devices passed through such as SATA controllers and USB controllers. There were no previous issues. The devices are bound early and never touched by the OS.

After performing the update, VMs were unable to launch resulting in the following error message:

Code: 
kvm: -device vfio-pci,host=0000:00:12.0,id=hostpci1.0,bus=ich9-pcie-port-2,addr=0x0.0,multifunction=on: vfio 0000:00:12.0: error getting device from group 5: Permission denied
Verify all devices in group 5 are bound to vfio-<bus> or pci-stub and not already in use
TASK ERROR: start failed: QEMU exited with code 1

The updates that were performed today are the following:
  • libnvpair3linux
  • libuutil3linux
  • libzfs4linux
  • libzpool5linux
  • proxmox-headers-6.8
  • proxmox-headers-6.8.12-12-pve
  • proxmox-kernel-6.8
  • proxmox-kernel-6.8.12-12-pve-signed
  • pve-firmware
  • spl
  • zfs-initramfs
  • zfs-zed
  • zfsutils-linux
I decided to first see if the issue was a result of other updates or the kernel itself. No additional dmesg / journalctl errors or hints existed between one kernel to the next.
I rolled back to the previous kernel [ 6.8.12-11-pve ] using "proxmox-boot-tool kernel pin 6.8.12-11-pve" and the issue disappeared. As I stated, all other newer systems still performed as expected with their passed through devices except for this one. I realize that with a lack of logging details, this is probably not something that can be resolved here quickly. Just posting for reference that the new kernel causes some type of vfio breakage with some special cases, in this case, an old FX based system.
 
  • Like
Reactions: hpaluch-pil, Lefuneste, nlsschmdt and 1 other person
I've got the same problem using an Intel Xeon E-2356G on a Supermicro X12STL-F. Working with 6.8.12-11-pve but not with 6.8.12-12-pve. I have used the proxmox-boot-tool to pin the kernel to 6.8.12-11 for now. In my case it's passing through the VFs of a Mellanox Connect-X 5.
 
Same issue here.

Pinned the kernel to 6.8.12-11, but although the guest VM started it was not able to use the iGPU properly.

Guest is running kernel:
Linux xyz 6.12.35+deb13-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.12.35-1 (2025-07-03) x86_64 GNU/Linux

no errors during boot
Code: 
$ sudo dmesg |grep i915
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-6.12.35+deb13-amd64 root=UUID=2ca7bf6a-aec0-4338-a719-29d94c4ef6a8 ro quiet net.ifnames=0 biosdevname=0 transparent_hugepage=never intel_iommu=on i915.enable_guc=3 module_blacklist=xe
[    0.011332] Kernel command line: BOOT_IMAGE=/vmlinuz-6.12.35+deb13-amd64 root=UUID=2ca7bf6a-aec0-4338-a719-29d94c4ef6a8 ro quiet net.ifnames=0 biosdevname=0 transparent_hugepage=never intel_iommu=on i915.enable_guc=3 module_blacklist=xe
[    4.961982] i915: loading out-of-tree module taints kernel.
[    4.962013] i915: module verification failed: signature and/or required key missing - tainting kernel
[    5.183444] i915: You are using the i915-sriov-dkms module, a ported version of the i915 module with SR-IOV support.
[    5.183446] i915: Please file any bug report at https://github.com/strongtz/i915-sriov-dkms/issues/new.
[    5.183446] i915: Module Homepage: https://github.com/strongtz/i915-sriov-dkms
[    5.183599] i915 0000:01:00.0: [drm] Found ALDERLAKE_S (device ID 4690) display version 12.00 stepping C0
[    5.183626] i915 0000:01:00.0: Running in SR-IOV VF mode
[    5.184566] i915 0000:01:00.0: [drm] GT0: GUC: interface version 0.1.20.1
[    5.184868] i915 0000:01:00.0: [drm] VT-d active for gfx access
[    5.184896] i915 0000:01:00.0: [drm] Using Transparent Hugepages
[    5.192279] i915 0000:01:00.0: [drm] GT0: GUC: interface version 0.1.20.1
[    5.192680] i915 0000:01:00.0: [drm] GT0: GUC: interface version 0.1.20.1
[    5.193212] i915 0000:01:00.0: GuC firmware PRELOADED version 0.0 submission:SR-IOV VF
[    5.193215] i915 0000:01:00.0: HuC firmware PRELOADED
[    5.195566] i915 0000:01:00.0: [drm] Protected Xe Path (PXP) protected content support initialized
[    5.195573] i915 0000:01:00.0: [drm] PMU not supported for this GPU.
[    5.195727] [drm] Initialized i915 1.6.0 for 0000:01:00.0 on minor 1

But as soon I try to use the card, it hangs and I get this erros in dmesg:
Code: 
[  180.411649] i915 0000:01:00.0: [drm] *ERROR* GT0: GUC: TLB invalidation response timed out for seqno 25
[  182.427676] i915 0000:01:00.0: [drm] *ERROR* GT0: GUC: TLB invalidation response timed out for seqno 26
[  184.443530] i915 0000:01:00.0: [drm] *ERROR* GT0: GUC: TLB invalidation response timed out for seqno 27
[  186.491544] i915 0000:01:00.0: [drm] *ERROR* GT0: GUC: TLB invalidation response timed out for seqno 28
[  188.507515] i915 0000:01:00.0: [drm] *ERROR* GT0: GUC: TLB invalidation response timed out for seqno 29
[  190.523640] i915 0000:01:00.0: [drm] *ERROR* GT0: GUC: TLB invalidation response timed out for seqno 30
[  192.571507] i915 0000:01:00.0: [drm] *ERROR* GT0: GUC: TLB invalidation response timed out for seqno 31
[  194.587491] i915 0000:01:00.0: [drm] *ERROR* GT0: GUC: TLB invalidation response timed out for seqno 32
[  196.603416] i915 0000:01:00.0: [drm] *ERROR* GT0: GUC: TLB invalidation response timed out for seqno 33
[  198.619335] i915 0000:01:00.0: [drm] *ERROR* GT0: GUC: TLB invalidation response timed out for seqno 34
[  200.635408] i915 0000:01:00.0: [drm] *ERROR* GT0: GUC: TLB invalidation response timed out for seqno 35
[  202.651346] i915 0000:01:00.0: [drm] *ERROR* GT0: GUC: TLB invalidation response timed out for seqno 36


Tried rebuilding the modules both in the host and the guest, with the host kernel pinned to 6.8.12-11 but no luck



Issue was also reported in the github https://github.com/strongtz/i915-sriov-dkms/issues/321
 
Last edited:
Yes exact same issue here once I updated to kernel 6.8.12-12. There definitely is some regression with this kernel version.

Several VM on multiple servers give error message when mounting vfio devices at VM start

kvm: -device vfio-pci,host=0000:04:00.7,id=hostpci1,bus=ich9-pcie-port-2,addr=0x0: vfio 0000:04:00.7: error getting device from group 66: Permission denied
Verify all devices in group 66 are bound to vfio-<bus> or pci-stub and not already in use
stopping swtpm instance (pid 11818) due to QEMU startup error
TASK ERROR: start failed: QEMU exited with code 1

I had to pin kernel version to restore functionalities provided by impacted VM

proxmox-boot-tool kernel list

proxmox-boot-tool kernel pin 6.8.12-11-pve
 
  • Like
Reactions: Shermo and nyx1197
Same issue on a PowerEdge R350 with Intel(R) Xeon(R) E-2378 CPU @ 2.60GHz.

rebooted back to 6.8.12-11-pve resolved the issue.
 
  • Like
Reactions: kasumi
Lefuneste said:
Yes exact same issue here once I updated to kernel 6.8.12-12. There definitely is some regression with this kernel version.

Several VM on multiple servers give error message when mounting vfio devices at VM start

kvm: -device vfio-pci,host=0000:04:00.7,id=hostpci1,bus=ich9-pcie-port-2,addr=0x0: vfio 0000:04:00.7: error getting device from group 66: Permission denied
Verify all devices in group 66 are bound to vfio-<bus> or pci-stub and not already in use
stopping swtpm instance (pid 11818) due to QEMU startup error
TASK ERROR: start failed: QEMU exited with code 1

I had to pin kernel version to restore functionalities provided by impacted VM

proxmox-boot-tool kernel list

proxmox-boot-tool kernel pin 6.8.12-11-pve
Click to expand...
Thanks, bro. This is really helpful for me.
 
Same kernel version, but another problem for me.
N

Thread 'Kernel 6.8.12-12-pve Update borken Ollama on AMDGPU issues'

Machine Model: Inspur 5212H5
CPU: Intel Xeon Gold 6138
GPU: 2x AMD Radeon Instinct MI50 32GB
GPU Driver: https://repo.radeon.com/amdgpu/6.3.2/ubuntu

I deployed Ollama in an LXC container on PVE using Docker Compose, and it works well. After updating to the latest version of the kernel, 6.8.12-12-pve, PVE receives an NMI interrupt error during the startup phase, and Ollama encounters an error when attempting to load the model.
When i use older kernel, like 6.8.12-11.pve, problem is gone.

Here is kernel log.

Code: 
kernel: [  167.335499] BUG: Bad page state in process ollama...
 
Thanks to Google for finding this, it explains my problem!

I updated yesterday and one of my LXC's running Frigate wouldn't start, this wasn't a huge surprise as I need a custom firmware module for my Hailo-8 device. So I proceeded to run the script I've used previously to rebuild that:
https://github.com/blakeblackshear/frigate/blob/dev/docker/hailo8l/user_installation.sh

Unfortunately that was failing:

Code: 
unable to load hailo_pci module, common reasons for this are:
- key was rejected by service: secure boot is enabling disallowing install.
- permissions are not setup correctly.

Thanks to @Lefuneste above I've run those commands to get me back up and running for now!
 
dcsapak said:
just fyi: we're currently looking into this, and believe we have identified a commit (+possible fix) that's responsible, i'll send a patch later if it turns out that it fixes the issue
Click to expand...
Looking forward to a fix for this. It seems there are others having similar issues with other SR-IOV devices too (e.g. Network cards). for me, it is for iGPU and affecting VMs only (LXCs with shared GPU working fine)
 
just fyi, 6.8.12-13 (with the fix) should be up soon on the pvetest repository
 
  • Like
Reactions: rjblake
THE_BIG_ONE said:
Can you be more specific as how to apply this patch? Attempted to use command patch pcipatch,patch only to be met with a blinking cursor. I have never patched a file before. I created a file called pcipatch.patch with the following:
Click to expand...
it's already applied in our git, so you could clone that repository, change to the appropriate branch and build it

otherwise the patch could be applied with something like 'git am <path-to-file>'

but since it's already applied, bumped and is currently being synced to pvetest, this should not be necessary anymore
 
  • Like
Reactions: rjblake
Thank you. I was in process of cloning the repository and about to build the kernel when I saw that. Deleted my post and updated to 6.8.12-13-pve and the VFIO issue has been resolved.
 
  • Like
Reactions: t.lamprecht
You must log in or register to reply here.
Top Bottom