services hung because of backup storage problem

[RolandK]

hello,

our backup storage (synology nas mounted via cifs) crashed last night during backup run.
some VMs have been stopped with io-error because of this (as they had been backed up when the crash happened), but unfortunately even with backup storage unmounted (umount -f...) i could not restart these, because some proxmox service seem to get stuck because of the crashed storage.

i killed the PID of the approrpiate VMs and restarted them via GUI, but i could not connect to the console, nor did the VMs guest OS come up.

apparently "systemctl restart pveproxy" or "systemctrl restart pvesr" got stuck and i need to reboot the host to fix it, so i needed to take down VMs which where still up and running.
my question is if it should be considered a BUG if external backup storage availability has such big influence and needs rebooting the host because some services got stuck instead of properly timing out or failing.

on one of the servers i can still show the problem, there are no important VMs on it.

regards
roland

 

[itNGO]

Are you using Proxmox Backup Server or direct vz-Backup?

 

[RolandK]

both. the crash was on the vz-backup storage

 

[RolandK]

when changing VM hardware properties for example, i'm getting "Connection error 596: Connection timed out"

when starting a VM, i see kvm process and the VM is shown running in gui, but in tasks "start failed: command .... failed: got timeout" is shown

oh, btw:
pve-manager/6.4-13/9f411e79 (running kernel: 5.4.166-1-pve)