PCIe passthrough (IOMMU) randomly not working after run backup job

[sinex]

Hi all, I use for backup job "Mode: stop" scenario. All three VMs have pcie passthrough devices - network cards, one of the vm have network card and raid controller. After VM reboot and backup job star, passthrough devices can randomly not work(more often) or work normally. I can’t connect this problem with anything. There are two different motherboards and the same problem is encountered on both. If I restart the virtual machines manually, even repeatedly, I do not encounter such a problem.

For resolving, i shutdown VM, send
echo "1" > /sys/bus/pci/devices/0000\:06\:00.0/remove
echo "1" > /sys/bus/pci/rescan
then starting VM.

My config:
Proxmox 6.1 (latest updates)
CPU - E2136
M/B - Supermicro X11SCM-LN8F
Memory - 32Gb ECC

 

[Alwin]

If I restart the virtual machines manually, even repeatedly, I do not encounter such a problem.

When a restart is triggered inside the VM, it will only reboot the OS, but the KVM process stays the same. With the stop mode for the backup, the KVM process will be a new one every time.

For resolving, i shutdown VM, send
echo "1" > /sys/bus/pci/devices/0000\:06\:00.0/remove
echo "1" > /sys/bus/pci/rescan
then starting VM.

Best use a hookscript to automate this. As it runs on every VM start.
https://pve.proxmox.com/pve-docs/chapter-qm.html#_hookscripts