[SOLVED] Unable to boot vms

[kafisatz]

Hi all

I am unable to boot most of my VMs.
Two are running (i.e. I was able to boot them).
Ceph seems to work fine.

When I try to start a VM I don't get any error message. Any ideas why?

Also, what is the expected output of the command `pmxcfs`
I am getting '[main] crit: unable to acquire pmxcfs lock: Resource temporarily unavailable'. But as two VMs are working, I am puzzled...

I recently restarted most services and hosts, because of the pmxcfs status:
https://forum.proxmox.com/threads/m...e-temporarily-unavailable.133452/#post-588256

 

[kafisatz]

root@pve1:~# qm start 101
Requesting HA start for VM 101
root@pve1:~# tail -10 /var/log/syslog
2023-09-11T09:16:16.902992+02:00 pve1 systemd[1]: session-13.scope: Deactivated successfully.
2023-09-11T09:16:32.871840+02:00 pve1 qm[16007]: <root@pam> starting task UPID:pve1:00003E88:0002916C:64FEBED0:hastart:101:root@pam:
2023-09-11T09:16:33.189934+02:00 pve1 qm[16007]: <root@pam> end task UPID:pve1:00003E88:0002916C:64FEBED0:hastart:101:root@pam: OK
2023-09-11T09:16:54.459533+02:00 pve1 qm[16176]: <root@pam> starting task UPID:pve1:00003F31:000299DB:64FEBEE6:hastart:101:root@pam:
2023-09-11T09:16:54.775271+02:00 pve1 qm[16176]: <root@pam> end task UPID:pve1:00003F31:000299DB:64FEBEE6:hastart:101:root@pam: OK
2023-09-11T09:17:01.207469+02:00 pve1 CRON[16255]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
2023-09-11T09:17:02.503136+02:00 pve1 qm[16257]: <root@pam> starting task UPID:pve1:00003F92:00029CFF:64FEBEEE:hastart:101:root@pam:
2023-09-11T09:17:02.819102+02:00 pve1 qm[16257]: <root@pam> end task UPID:pve1:00003F92:00029CFF:64FEBEEE:hastart:101:root@pam: OK
2023-09-11T09:17:07.166944+02:00 pve1 qm[16276]: <root@pam> starting task UPID:pve1:00003F95:00029ED1:64FEBEF3:hastart:101:root@pam:
2023-09-11T09:17:07.479066+02:00 pve1 qm[16276]: <root@pam> end task UPID:pve1:00003F95:00029ED1:64FEBEF3:hastart:101:root@pam: OK

 

[kafisatz]

SOLVED:
turns out restoring from backup does the trick.
Maybe the data on ceph got corrupted when I rebooted/updated the nodes yesterday. Strange that (a) two VMs were working all along and (b) I did not see any error message when trying to start the other VMs