Proxmox VM crash when the backup runs

[csl]

Hi all,

It’s taken some time to work out when a VM (Rocky 9 with cPanel installed) does crash when a backup runs.

It seems to be when SFTP has run, then when a timed backup runs this happens:

2024-10-29 13:54:34 INFO: Starting Backup of VM 105 (qemu)
2024-10-29 13:54:34 INFO: status = running
2024-10-29 13:54:34 INFO: VM Name: WHM
2024-10-29 13:54:34 INFO: include disk 'scsi0' 'Storage:105/vm-105-disk-0.qcow2' 500G
2024-10-29 13:54:34 INFO: backup mode: snapshot
2024-10-29 13:54:34 INFO: ionice priority: 7
2024-10-29 13:54:34 INFO: creating vzdump archive '/mnt/pve/QNAP_Backup/dump/vzdump-qemu-105-2024_10_29-13_54_34.vma.zst'
2024-10-29 13:54:34 INFO: issuing guest-agent 'fs-freeze' command
2024-10-29 13:59:03 ERROR: VM 105 qmp command 'guest-fsfreeze-freeze' failed - client closed connection
2024-10-29 13:59:03 INFO: issuing guest-agent 'fs-thaw' command
2024-10-29 13:59:03 ERROR: VM 105 not running
2024-10-29 13:59:03 ERROR: unable to connect to VM 105 qmp socket - No such file or directory
2024-10-29 13:59:03 INFO: aborting backup job
2024-10-29 13:59:03 ERROR: VM 105 not running
2024-10-29 13:59:03 INFO: resuming VM again
2024-10-29 13:59:03 ERROR: Backup of VM 105 failed - VM 105 not running

The VM then seems like it’s frozen but if I do ‘qm unlock 105’ it has no effect. The only thing that seems to work is to do a hard ‘Stop’ and then ‘Start’ to get the VM to work again.

I have tried the scripts/securetmp fix to no avail.

The only way to wipe SFTP history is to reboot the VM. If I do that the backup works fine every day. If I don’t run SFTP on the VM the backup works fine. If I run SFTP and leave it, it waits until the backup runs and the VM crashes/freezes.

Anyone got any ideas?

Cheers,

Alex

 

[csl]

No one had similar?

I need to find out if there is anything different in whether things are still logged in (via PAM) or not. For example just closing the window in WinSCP might be leaving it logged in? Try and zoom in a bit?

 

[leesteken]

Maybe the storage becomes full and the VM gets an I/O-error because of that (and cannot continue or freezes and cannot log this)? This is not uncommon when the backup drive is not mounted and you are writing to the same drive (many threads about that but I don't know how to search for them).