Backup hangs - ends with the "question mark' sign (please help)

[kisiel]

Hi
I run a cluster of 8.0.4 nodes
one of the HA containers seems to be special.
When I start the backup, it would hang making the whole node unresponsive (grey question mark).
although the other containers on that node seems to keep working.
It does not matter if I run it in snapshot or suspend
As HA it does not want to work with shutdown.

The only option to come back to normal is to restart the node.

I am not sure how to start analysing it.
Here is the backup log (or what is available through web-interface)

INFO: starting new backup job: vzdump --storage pbs_home --all 0 --mailto [redacted] --pool daily.backup.suspend --mode suspend --mailnotification always --notes-template '{{guestname}}' --node hpve3
INFO: Starting Backup of VM 102 (lxc)
INFO: Backup started at 2023-11-20 15:19:15
INFO: status = running
INFO: backup mode: suspend
INFO: bandwidth limit: 20000 KB/s
INFO: ionice priority: 7
INFO: CT Name: mfs.service.new
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/data') in backup
INFO: starting first sync /proc/8496/root/ to /tmp/vzdumptmp1249837_102/
INFO: first sync finished - transferred 1.46G bytes in 72s
INFO: suspending guest

 

[sb-jw]

Have you looked at the logs etc. to see if there are any anomalies? Can you secure the container when it is turned off? Can you back it up if it's running on another node?

 

[kisiel]

i checked logs etc.
it feels like under some circumstances containers don't like to be snapshotted (especially with some fuse mounts)
works perfectly with shutdown and sometimes with suspend
problem is - those won't work with HA setup

 

[Rolland]

If HA is configured, only snapshot mode can be used