Proxmox backup fails for VM with GPU passthrough in stop state

D

dodoj

Active Member
Aug 17, 2020
14
0
41
64
Graphic card is passed to VM machine, using standard procedure (0000:82:00,pcie=1,x-vga=1). It is AMD RX 500 RX card. Machine VM works normaly, can be started and stopped without any problem, card is well recognized and it is used as main GPU.

1) Problem #1

when making backup in stop mode, there is a problem:

error writing '1' to '/sys/bus/pci/devices/0000:82:00.0/reset': Inappropriate ioctl for device
failed to reset PCI device '0000:82:00.0', but trying to continue as not all devices need a reset
...
ERROR: start failed: command '/usr/bin/kvm ... -device vfio-pci,host=0000:82:00.0 ...

2) Problem #2

After backup failed, it was not possible to start VM:

error writing '1' to '/sys/bus/pci/devices/0000:82:00.0/reset': Inappropriate ioctl for device
failed to reset PCI device '0000:82:00.0', but trying to continue as not all devices need a reset
swtpm_setup: Not overwriting existing state file.
TASK ERROR: start failed: command '/usr/bin/kvm -id 102 -name 'XXXXXXX,debug-threads=on' ... -machine 'hpet=off,type=pc-q35-5.1+pve0'' failed: got timeout

what is interesting, second try to start VM was succesfull - but it took long time.

Third start of VM was completely normal.

Please help.
 
dodoj said:
error writing '1' to '/sys/bus/pci/devices/0000:82:00.0/reset': Inappropriate ioctl for device
failed to reset PCI device '0000:82:00.0', but trying to continue as not all devices need a reset
Click to expand...
This is not a problem and just an informational message. Since you probably installed and enabled vendor-reset, there should be no problems with resetting the RX5?0.
dodoj said:
...
ERROR: start failed: command '/usr/bin/kvm ... -device vfio-pci,host=0000:82:00.0 ...
Click to expand...
There must be something else causing the VM to fail to start. Please show the VM configuration (qm config 102), the whole Task log and the System log (journalctl) from around the time of starting the VM.

EDIT: If there are no actual errors and only a timeout then try starting the VM with less memory (like half or a quarter) and without hugepages. Maybe Proxmox cannot free enough memory quickly enough.
 
Last edited:
I dont see any other reason causing VM to start. In normal us, I can start / stop machine. I can make backub of running machine. But when machine is stopped, and I try to make backup. backup job tries to start machine and this fails.

root@pve2:~# qm config 102
agent: 1
balloon: 0
bios: ovmf
boot: order=virtio0;ide0
cores: 8
cpu: Haswell-noTSX
efidisk0: vmdisk:102/vm-102-disk-0.qcow2,efitype=4m,pre-enrolled-keys=1,size=528K
hostpci0: 0000:82:00,pcie=1,x-vga=1
ide0: none,media=cdrom
machine: pc-q35-5.1
memory: 65536
name: TdvWin10Calc
net0: virtio=26:9F:D3:DE:DB:07,bridge=vmbr5214,firewall=1
net1: virtio=36:3F:FF:5B:14:14,bridge=vmbr51,firewall=1
numa: 0
ostype: win10
scsihw: virtio-scsi-single
smbios1: uuid=8cbcd3c3-5757-4fd8-a81d-e40f9769c660
sockets: 1
spice_enhancements: videostreaming=all
tpmstate0: vmdisk:102/vm-102-disk-1.raw,size=4M,version=v2.0
vga: none
virtio0: vmdisk:102/vm-102-disk-2.raw,aio=threads,cache=writeback,discard=on,iothread=1,size=104G
virtio1: vmdisk:102/vm-102-disk-3.raw,aio=threads,cache=writeback,discard=on,iothread=1,size=300G
virtio2: vmdisk:102/vm-102-disk-4.raw,aio=threads,cache=writeback,discard=on,iothread=1,size=200G
vmgenid: 762d1d7c-8a2d-4ee9-8391-0b71af7fd1ec
 
This is from backup log:

NFO: Starting Backup of VM 102 (qemu)
INFO: Backup started at 2025-10-27 03:27:26
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: VM Name: ************
INFO: include disk 'virtio0' 'vmdisk:102/vm-102-disk-2.raw' 104G
INFO: include disk 'virtio1' 'vmdisk:102/vm-102-disk-3.raw' 300G
INFO: include disk 'virtio2' 'vmdisk:102/vm-102-disk-4.raw' 200G
INFO: include disk 'efidisk0' 'vmdisk:102/vm-102-disk-0.qcow2' 528K
INFO: include disk 'tpmstate0' 'vmdisk:102/vm-102-disk-1.raw' 4M
INFO: creating Proxmox Backup Server archive 'vm/102/2025-10-27T02:27:26Z'
INFO: starting kvm to execute backup task
error writing '1' to '/sys/bus/pci/devices/0000:82:00.0/reset': Inappropriate ioctl for device
failed to reset PCI device '0000:82:00.0', but trying to continue as not all devices need a reset
swtpm_setup: Not overwriting existing state file.
ERROR: start failed: command '/usr/bin/kvm -id 102 -name 'TdvWin10Calc,debug-threads=on' -no-shutdown -chardev 'socket,id=qmp,path=/var/run/qemu-server/102.qmp,server=on,wait=off' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect-ms=5000' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/102.pid -daemonize -smbios 'type=1,uuid=8cbcd3c3-5757-4fd8-a81d-e40f9769c660' -drive 'if=pflash,unit=0,format=raw,readonly=on,file=/usr/share/pve-edk2-firmware//OVMF_CODE_4M.secboot.fd' -drive 'if=pflash,unit=1,id=drive-efidisk0,format=qcow2,file=/mnt/vmdisk/images/102/vm-102-disk-0.qcow2' -smp '8,sockets=1,cores=8,maxcpus=8' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga none -nographic -global 'kvm-pit.lost_tick_policy=discard' -cpu 'Haswell-noTSX,enforce,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vendor_id=proxmox,hv_vpindex,kvm=off,+kvm_pv_eoi,+kvm_pv_unhalt,vendor=GenuineIntel' -m 65536 -object 'iothread,id=iothread-virtio0' -object 'iothread,id=iothread-virtio1' -object 'iothread,id=iothread-virtio2' -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg -device 'vmgenid,guid=762d1d7c-8a2d-4ee9-8391-0b71af7fd1ec' -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=0000:82:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on' -device 'vfio-pci,host=0000:82:00.1,id=hostpci0.1,bus=ich9-pcie-port-1,addr=0x0.1' -chardev 'socket,id=tpmchar,path=/var/run/qemu-server/102.swtpm' -tpmdev 'emulator,id=tpmdev,chardev=tpmchar' -device 'tpm-tis,tpmdev=tpmdev' -chardev 'socket,path=/var/run/qemu-server/102.qga,server=on,wait=off,id=qga0' -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:fd374abd4bb' -drive 'if=none,id=drive-ide0,media=cdrom,aio=io_uring' -device 'ide-cd,bus=ide.0,unit=0,drive=drive-ide0,id=ide0,bootindex=101' -drive 'file=/mnt/vmdisk/images/102/vm-102-disk-2.raw,if=none,id=drive-virtio0,cache=writeback,discard=on,format=raw,aio=threads,detect-zeroes=unmap' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,iothread=iothread-virtio0,bootindex=100' -drive 'file=/mnt/vmdisk/images/102/vm-102-disk-3.raw,if=none,id=drive-virtio1,cache=writeback,discard=on,format=raw,aio=threads,detect-zeroes=unmap' -device 'virtio-blk-pci,drive=drive-virtio1,id=virtio1,bus=pci.0,addr=0xb,iothread=iothread-virtio1' -drive 'file=/mnt/vmdisk/images/102/vm-102-disk-4.raw,if=none,id=drive-virtio2,cache=writeback,discard=on,format=raw,aio=threads,detect-zeroes=unmap' -device 'virtio-blk-pci,drive=drive-virtio2,id=virtio2,bus=pci.0,addr=0xc,iothread=iothread-virtio2' -netdev 'type=tap,id=net0,ifname=tap102i0,script=/usr/libexec/qemu-server/pve-bridge,downscript=/usr/libexec/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=26:9F:D3:DE:DB:07,netdev=net0,bus=pci.0,addr=0x12,id=net0' -netdev 'type=tap,id=net1,ifname=tap102i1,script=/usr/libexec/qemu-server/pve-bridge,downscript=/usr/libexec/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=36:3F:FF:5B:14:14,netdev=net1,bus=pci.0,addr=0x13,id=net1' -rtc 'driftfix=slew,base=localtime' -machine 'hpet=off,type=pc-q35-5.1+pve0' -S' failed: got timeout
INFO: aborting backup job
ERROR: VM 102 qmp command 'backup-cancel' failed - unable to connect to VM 102 qmp socket - Connection refused
VM 102 not running
ERROR: Backup of VM 102 failed - start failed: command '/usr/bin/kvm -id 102 -name 'TdvWin10Calc,debug-threads=on' -no-shutdown -chardev 'socket,id=qmp,path=/var/run/qemu-server/102.qmp,server=on,wait=off' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect-ms=5000' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/102.pid -daemonize -smbios 'type=1,uuid=8cbcd3c3-5757-4fd8-a81d-e40f9769c660' -drive 'if=pflash,unit=0,format=raw,readonly=on,file=/usr/share/pve-edk2-firmware//OVMF_CODE_4M.secboot.fd' -drive 'if=pflash,unit=1,id=drive-efidisk0,format=qcow2,file=/mnt/vmdisk/images/102/vm-102-disk-0.qcow2' -smp '8,sockets=1,cores=8,maxcpus=8' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga none -nographic -global 'kvm-pit.lost_tick_policy=discard' -cpu 'Haswell-noTSX,enforce,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vendor_id=proxmox,hv_vpindex,kvm=off,+kvm_pv_eoi,+kvm_pv_unhalt,vendor=GenuineIntel' -m 65536 -object 'iothread,id=iothread-virtio0' -object 'iothread,id=iothread-virtio1' -object 'iothread,id=iothread-virtio2' -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg -device 'vmgenid,guid=762d1d7c-8a2d-4ee9-8391-0b71af7fd1ec' -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=0000:82:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on' -device 'vfio-pci,host=0000:82:00.1,id=hostpci0.1,bus=ich9-pcie-port-1,addr=0x0.1' -chardev 'socket,id=tpmchar,path=/var/run/qemu-server/102.swtpm' -tpmdev 'emulator,id=tpmdev,chardev=tpmchar' -device 'tpm-tis,tpmdev=tpmdev' -chardev 'socket,path=/var/run/qemu-server/102.qga,server=on,wait=off,id=qga0' -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:fd374abd4bb' -drive 'if=none,id=drive-ide0,media=cdrom,aio=io_uring' -device 'ide-cd,bus=ide.0,unit=0,drive=drive-ide0,id=ide0,bootindex=101' -drive 'file=/mnt/vmdisk/images/102/vm-102-disk-2.raw,if=none,id=drive-virtio0,cache=writeback,discard=on,format=raw,aio=threads,detect-zeroes=unmap' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,iothread=iothread-virtio0,bootindex=100' -drive 'file=/mnt/vmdisk/images/102/vm-102-disk-3.raw,if=none,id=drive-virtio1,cache=writeback,discard=on,format=raw,aio=threads,detect-zeroes=unmap' -device 'virtio-blk-pci,drive=drive-virtio1,id=virtio1,bus=pci.0,addr=0xb,iothread=iothread-virtio1' -drive 'file=/mnt/vmdisk/images/102/vm-102-disk-4.raw,if=none,id=drive-virtio2,cache=writeback,discard=on,format=raw,aio=threads,detect-zeroes=unmap' -device 'virtio-blk-pci,drive=drive-virtio2,id=virtio2,bus=pci.0,addr=0xc,iothread=iothread-virtio2' -netdev 'type=tap,id=net0,ifname=tap102i0,script=/usr/libexec/qemu-server/pve-bridge,downscript=/usr/libexec/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=26:9F:D3:DE:DB:07,netdev=net0,bus=pci.0,addr=0x12,id=net0' -netdev 'type=tap,id=net1,ifname=tap102i1,script=/usr/libexec/qemu-server/pve-bridge,downscript=/usr/libexec/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=36:3F:FF:5B:14:14,netdev=net1,bus=pci.0,addr=0x13,id=net1' -rtc 'driftfix=slew,base=localtime' -machine 'hpet=off,type=pc-q35-5.1+pve0' -S' failed: got timeout
INFO: Failed at 2025-10-27 03:34:41
 
This is not my case. Only this VM is using this pass-thru device (this is GPU amd radeon 500 RX). No other VM is using this pass-thru device:

Start VM works, stop VM works, backup with running machine works. Backup with stop machine is not working!!! - backup job starts VM (? why) and this fails.
 
You must log in or register to reply here.
Top Bottom