I have a cluster currently running Proxmox 4.0. It was working fine until I attempted to add 3 additional nodes. After adding the nodes, the cluster stopped running and quorum activity is now blocked. I did receive errors on the new nodes about being unable to write to /etc/pve/nodes/HOSTNAME when connecting them. Also, /etc/pve/ is read only now.
I think I might have caused this when I used Saltstack to deploy my signed SSL certificate to the new nodes prior to joining the cluster. The new nodes are have been taken down now.
What I am wondering now is if there is a way I can get the original cluster nodes to start talking again and remove any trace of the new nodes. I am including the the output of "pvecm status" and pvecm nodes."
I think I might have caused this when I used Saltstack to deploy my signed SSL certificate to the new nodes prior to joining the cluster. The new nodes are have been taken down now.
What I am wondering now is if there is a way I can get the original cluster nodes to start talking again and remove any trace of the new nodes. I am including the the output of "pvecm status" and pvecm nodes."
root@LV-HV-04:/etc/pve/nodes# pvecm status
Quorum information
------------------
Date: Wed Dec 16 15:28:03 2015
Quorum provider: corosync_votequorum
Nodes: 3
Node ID: 0x00000001
Ring ID: 900
Quorate: No
Votequorum information
----------------------
Expected votes: 6
Highest expected: 6
Total votes: 3
Quorum: 4 Activity blocked
Flags:
Membership information
----------------------
Nodeid Votes Name
0x00000001 1 10.3.8.10 (local)
0x00000002 1 10.3.8.11
0x00000003 1 10.3.8.12
root@LV-HV-04:/etc/pve/nodes# pvecm nodes
Membership information
----------------------
Nodeid Votes Name
1 1 LV-HV-04 (local)
2 1 LV-HV-05
3 1 LV-HV-06
root@LV-HV-04:/etc/pve# ll
total 9
drwxr-xr-x 2 root www-data 0 Jan 1 1970 .
drwxr-xr-x 107 root root 4096 Dec 15 20:47 ..
-r--r----- 1 root www-data 451 Nov 25 12:28 authkey.pub
-r--r----- 1 root www-data 9242 Jan 1 1970 .clusterlog
-r--r----- 1 root www-data 706 Dec 15 19:54 corosync.conf
-r--r----- 1 root www-data 16 Nov 25 12:23 datacenter.cfg
-rw-r----- 1 root www-data 2 Jan 1 1970 .debug
-r--r----- 1 root www-data 226 Nov 25 22:03 domains.cfg
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 local -> nodes/LV-HV-04
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 lxc -> nodes/LV-HV-04/lxc
-r--r----- 1 root www-data 426 Jan 1 1970 .members
dr-xr-xr-x 2 root www-data 0 Nov 25 12:28 nodes
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 openvz -> nodes/LV-HV-04/openvz
dr-x------ 2 root www-data 0 Nov 25 12:28 priv
-r--r----- 1 root www-data 2664 Nov 25 21:58 pve-root-ca.pem
-r--r----- 1 root www-data 1675 Nov 25 12:28 pve-www.key
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 qemu-server -> nodes/LV-HV-04/qemu-server
-r--r----- 1 root www-data 3737 Jan 1 1970 .rrd
-r--r----- 1 root www-data 201 Dec 1 19:58 storage.cfg
-r--r----- 1 root www-data 408 Nov 25 22:06 user.cfg
-r--r----- 1 root www-data 429 Jan 1 1970 .version
-r--r----- 1 root www-data 1011 Jan 1 1970 .vmlist
-r--r----- 1 root www-data 119 Nov 25 12:28 vzdump.cron