services hung because of backup storage problem

RolandK

Member
Mar 5, 2019
162
19
23
49
hello,

our backup storage (synology nas mounted via cifs) crashed last night during backup run.
some VMs have been stopped with io-error because of this (as they had been backed up when the crash happened), but unfortunately even with backup storage unmounted (umount -f...) i could not restart these, because some proxmox service seem to get stuck because of the crashed storage.

i killed the PID of the approrpiate VMs and restarted them via GUI, but i could not connect to the console, nor did the VMs guest OS come up.

apparently "systemctl restart pveproxy" or "systemctrl restart pvesr" got stuck and i need to reboot the host to fix it, so i needed to take down VMs which where still up and running.
my question is if it should be considered a BUG if external backup storage availability has such big influence and needs rebooting the host because some services got stuck instead of properly timing out or failing.

on one of the servers i can still show the problem, there are no important VMs on it.

regards
roland
 

itNGO

Well-Known Member
Jun 12, 2020
524
114
48
44
Germany
it-ngo.com
Are you using Proxmox Backup Server or direct vz-Backup?
 

RolandK

Member
Mar 5, 2019
162
19
23
49
when changing VM hardware properties for example, i'm getting "Connection error 596: Connection timed out"

when starting a VM, i see kvm process and the VM is shown running in gui, but in tasks "start failed: command .... failed: got timeout" is shown

oh, btw:
pve-manager/6.4-13/9f411e79 (running kernel: 5.4.166-1-pve)
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!