TASK ERROR: VM is locked (backup)

[jmjosebest]

Hi, after a full host restart I get this error starting a virtual machine KVM.

TASK ERROR: VM is locked (backup)

/var/lib/vz/lock is empty...
How can I unlock?
Thanks

 

[jmjosebest]

Ok

qm unlock 101

 

[PC Rescue]

Hey thanks jmjosebest, you saved my day!

 

[Riesling.Dry]

o.k., unlock works. But how can it be prevented that the VE locks itself up in the first place?I have one that keeps falling into this state, cannot be (re-)started via the web-interface and needs above manual unlock to be (re-)started again. What could be the cause?
Cheers

 

[fabian]

find out why the backup job crashed.. is it backing up to unreliable (network) storage?

 

[Riesling.Dry]

Thank you, Fabian.
So it is definitely a crashed backup-job that prevents the VE from (re-)starting? How could I find out what is the cause of the crash during backup?
Backups are performed on local storage (ZFS pool: rpool)
None of the other 5 VEs on this box has this problem; they are all backed up together once a week.

 

[fabian]

check the system logs around the time of the failed backup, try manually backing up the VM a couple of times and check for failure.

it might have been a one time fluke.

 

[Riesling.Dry]

I can provoke a system crash by manually starting a backup of the VE that I use to find in a locked state.
After 1 or 2 mins., process z_wr_iss takes 97% CPU and subsequently the server fails. It then automatically reboots, but the VE is still locked from the pending/failed backup process and does not automatically recover.
I attach a screenshot of the last state seen in "top".
Could this be prevented by adding some RAM to the box?

 

[Riesling.Dry]

just a brief follow up: I upgraded my box (hp micro server gen8). RAM from 2GB to 4GB, replaced the Celeron w. an i3. No more problems since then. proxmox' official system requirements state 8GB RAM for a good reason

 

[bilalasif89]

Ok

qm unlock 101

makes my day

 

[CTF]

Thank you, this helped me too!

 

[kwinz]

I have a backup job with Mode: stop (not snapshot or suspend).

After I manually aborted this backup job via GUI's "Stop" button, the VMs remained locked.
And I could not backup them any more.

ERROR: Backup of VM 108 failed - VM is locked (backup)
INFO: Failed at 2023-03-11 11:09:33
INFO: Backup job finished with errors
TASK ERROR: job errors

Manually unlocking them with qm unlock 108 as per this thread did fix the problem.

However I still think this is a bug. When aborting a backup job via GUI, the GUI should make sure that the VMs get properly unlocked.

 

[fabian]

yes, see https://bugzilla.proxmox.com/show_bug.cgi?id=2723

 

[jmjosebest]

Bug reported in:

https://cloudlinux.zendesk.com/hc/e...calls-fsfreeze-CloudLinux-VM-hangs-on-ProxMox

https://gitlab.com/qemu-project/qemu/-/issues/520

Also, in the new Proxmox VE 7.4 I see:

Running fsfreeze/fsthaw before starting a backup can now optionally be disabled in the QEMU guest agent options.
Note: Disabling this option can potentially lead to backups with inconsistent filesystems and should therefore only be disabled if you know what you are doing.

 

[Bastler11]

Hello, during I made a backup from my VM my backup storage (Sinology) shut down. After starting it new and trying to make a new backup I got the information:

VM is locked (backup)


How could I fix it?

Thanks!

 

[jmjosebest]

Hello, during I made a backup from my VM my backup storage (Sinology) shut down. After starting it new and trying to make a new backup I got the information:

VM is locked (backup)

View attachment 56012
How could I fix it?

Thanks!

Run:

qm unlock 100

 

[OD3UNICRON]

I ran this command qm unlock 102 then I got this "unable to open file '/etc/pve/nodes/UniMatrix001/qemu-server/102.conf.tmp.37412' - Input/output error" please help

 

[fabian]

please post the output of "pveversion -v", "pvecm status" and "systemctl status pve-cluster corosync".