cluster broke

[sahostking]

I think I broke cluster somehow. Don't think I set it up right.

I need to backup all vms as I don't want to use cluster setup anymore.

But when I try to backup to NFS server I'm getting this:

()
INFO: starting new backup job: vzdump 100 --compress lzo --node vz-jhb-2 --mode snapshot --storage nfs-storagevzjhb1 --remove 0
INFO: Starting Backup of VM 100 (qemu)
INFO: status = running
INFO: unable to open file '/etc/pve/nodes/vz-jhb-2/qemu-server/100.conf.tmp.1442' - Permission denied
INFO: update VM 100: -lock backup
ERROR: Backup of VM 100 failed - command 'qm set 100 --lock backup' failed: exit code 2
INFO: Backup job finished with errors
TASK ERROR: job errors


Any ideas how to solve this. The first box I took down and reinstalled which is where I want to move the VMs to by restoring backups. I'm trying to rerun backups so I have new version of vms to restore. But keeps failing. Note I had a 2 node cluster.

Is it because I rebuilt first one and no cluster screwed up How to backup to nfs successfully so I can restore vms to new clean node?

 

[wosp]

What does "pvecm status" show?
I think you lost quorum (multicast working?) and therefor activity is blocked, so you can't write a temp. file to /etc/pve. If this is the case you can try to regain quorum or try to run "pvecm expected 1" and try again.

 

[sahostking]

pvecm status
Cannot initialize CMAP service

pvecm expected 1
Cannot initialize CMAP service

Can't stat corosync it seems too.

root@vz-jhb-2:~# tail -f /var/log/daemon.log
May 14 09:58:17 vz-jhb-2 pmxcfs[30191]: [dcdb] crit: cpg_initialize failed: 2
May 14 09:58:17 vz-jhb-2 pmxcfs[30191]: [status] crit: cpg_initialize failed: 2
May 14 09:58:23 vz-jhb-2 pmxcfs[30191]: [quorum] crit: quorum_initialize failed: 2

 

[sahostking]

did this which allows me to back it up now:

service pve-cluster stop
pmxcfs -l