Warning: UPDATING these packages broke my pci passthrough.

[v95klima]

Just broke when updated yesterday. All VMs depending on AMD 6900 XT stopped working. Was working for last 4 years, and now with update broke.
Please help?

error writing '1' to '/sys/bus/pci/devices/0000:04:00.0/reset': Inappropriate ioctl for device
failed to reset PCI device '0000:04:00.0', but trying to continue as not all devices need a reset
kvm: -device vfio-pci,host=0000:01:00.2,id=hostpci2,bus=ich9-pcie-port-3,addr=0x0: vfio 0000:01:00.2: error getting device info: Input/output error
TASK ERROR: start failed: QEMU exited with code 1

dmesg | grep vfio-pci
[ 5.355747] vfio-pci 0000:11:00.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=nonewns=none
[ 236.763142] vfio-pci 0000:04:00.0: vgaarb: deactivate vga console
[ 236.763152] vfio-pci 0000:04:00.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=nonewns=none
[ 236.763448] vfio-pci 0000:04:00.0: vgaarb: VGA decodes changed: olddecodes=none,decodes=io+memwns=none
[ 236.763552] vfio-pci 0000:04:00.0: vgaarb: deactivate vga console
[ 236.763556] vfio-pci 0000:04:00.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=nonewns=none
[ 239.572122] vfio-pci 0000:04:00.0: enabling device (0002 -> 0003)
[ 239.573632] vfio-pci 0000:04:00.1: enabling device (0000 -> 0002)
[ 239.577215] vfio-pci 0000:01:00.2: enabling device (0000 -> 0002)
[ 239.683537] vfio-pci 0000:01:00.2: vfio: runtime resume failed -13

cat /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt initcall_blacklist=sysfb_init video=efifbff amd_pstate=passive"
GRUB_CMDLINE_LINUX=""


root@pve:~# pveversion

pve-manager/8.4.5/57892e8e686cb35b (running kernel: 6.8.12-12-pve)

root@pve:~# pveversion -v

proxmox-ve: 8.4.0 (running kernel: 6.8.12-12-pve)

pve-manager: 8.4.5 (running version: 8.4.5/57892e8e686cb35b)

proxmox-kernel-helper: 8.1.1

proxmox-kernel-6.8.12-12-pve-signed: 6.8.12-12

proxmox-kernel-6.8: 6.8.12-12

proxmox-kernel-6.8.12-11-pve-signed: 6.8.12-11

proxmox-kernel-6.8.12-10-pve-signed: 6.8.12-10

proxmox-kernel-6.5.13-6-pve-signed: 6.5.13-6

proxmox-kernel-6.5: 6.5.13-6

proxmox-kernel-6.5.11-4-pve-signed: 6.5.11-4

amd64-microcode: 3.20240820.1~deb12u1

ceph-fuse: 17.2.7-pve1

corosync: 3.1.9-pve1

criu: 3.17.1-2+deb12u1

glusterfs-client: 10.3-5

ifupdown2: 3.2.0-1+pmx11

ksm-control-daemon: 1.5-1

libjs-extjs: 7.0.0-5

libknet1: 1.30-pve2

libproxmox-acme-perl: 1.6.0

libproxmox-backup-qemu0: 1.5.2

libproxmox-rs-perl: 0.3.5

libpve-access-control: 8.2.2

libpve-apiclient-perl: 3.3.2

libpve-cluster-api-perl: 8.1.2

libpve-cluster-perl: 8.1.2

libpve-common-perl: 8.3.2

libpve-guest-common-perl: 5.2.2

libpve-http-server-perl: 5.2.2

libpve-network-perl: 0.11.2

libpve-rs-perl: 0.9.4

libpve-storage-perl: 8.3.6

libspice-server1: 0.15.1-1

lvm2: 2.03.16-2

lxc-pve: 6.0.0-1

lxcfs: 6.0.0-pve2

novnc-pve: 1.6.0-2

proxmox-backup-client: 3.4.3-1

proxmox-backup-file-restore: 3.4.3-1

proxmox-backup-restore-image: 0.7.0

proxmox-firewall: 0.7.1

proxmox-kernel-helper: 8.1.1

proxmox-mail-forward: 0.3.3

proxmox-mini-journalreader: 1.5

proxmox-offline-mirror-helper: 0.6.7

proxmox-widget-toolkit: 4.3.12

pve-cluster: 8.1.2

pve-container: 5.2.7

pve-docs: 8.4.0

pve-edk2-firmware: 4.2025.02-4~bpo12+1

pve-esxi-import-tools: 0.7.4

pve-firewall: 5.1.2

pve-firmware: 3.16-3

pve-ha-manager: 4.0.7

pve-i18n: 3.4.5

pve-qemu-kvm: 9.2.0-7

pve-xtermjs: 5.5.0-2

qemu-server: 8.4.1

smartmontools: 7.3-pve1

spiceterm: 3.3.0

swtpm: 0.8.0+pve1

vncterm: 1.8.0

zfsutils-linux: 2.2.8-pve1

 

[Uuugh]

Unfortunately, it appears that resolving this issue is not on Proxmox's agenda (note the length of time for this thread).
Like you, one day it worked, and then it didn't. I suspect this is a problem for many others using PCI passthrough for their GPUs; they just haven't realized yet that it is the cause of their system crashing.
My next step is to reformat and reinstall. That "magic seems to occasionally work.

 

[v95klima]

Successfull Workaround by rolling back kernel to 6.8.12-11-pve:

proxmox-boot-tool kernel list

promox-boot-tool kernel pin 6.8.12-11-pve

 

[kimus]

after almost... 1 year.. this guide fix (for me) the famous problem

error writing '1' to '/sys/bus/pci/devices/0000:c6:00.0/reset': Inappropriate ioctl for device

its a windows side, but for me it works, I don't have to restart the host anymore, win11 vm stop and restart like a charm
I hope it's useful for someone

 

[Uuugh]

Successfull Workaround by rolling back kernel to 6.8.12-11-pve:

proxmox-boot-tool kernel list

promox-boot-tool kernel pin 6.8.12-11-pve

apt update && apt install pve-kernel-6.8.12-11-pve
promox-boot-tool kernel pin 6.8.12-11-pve

and then you'll need to

update-initramfs -u -k all

Thanks again @v95klima !