I added the 2-nd node to a cluster, and it is missing /etc/pve/node directory.
both use :
pve-manager: 2.0-12 (pve-manager/2.0/784729f4)
running kernel: 2.6.32-6-pve
proxmox-ve-2.6.32: 2.0-53
pve-kernel-2.6.32-6-pve: 2.6.32-53
lvm2: 2.02.86-1pve2
clvm: 2.02.86-1pve2
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-1
libqb: 0.6.0-1
redhat-cluster-pve: 3.1.7-1
pve-cluster: 1.0-12
qemu-server: 2.0-9
pve-firmware: 1.0-13
libpve-common-perl: 1.0-8
libpve-access-control: 1.0-2
libpve-storage-perl: 2.0-8
vncterm: 1.0-2
vzctl: 3.0.29-3pve3
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 0.15.0-1
ksm-control-daemon: 1.1-1
(qemu-server is one upgrade behind on one of the nodes , but probably does not have anything to do with the issue ).
this is what I did to create the cluster:
then to add a node: ( note this line: Waiting for quorum... Timed-out waiting for cluster' )
this did not look correct on either node:
so i rebooted fbc186 then this part looks ok:
root@fbc1 ~ # pvecm nodes
Node Sts Inc Joined Name
1 M 4 2011-12-01 21:01:24 fbc1
2 M 12 2011-12-01 21:14:27 fbc186
[/code]
here is ls -la /etc/pve both nodes:
note on fbc186 , nodes/fbc186/openvz is missing .
both use :
pve-manager: 2.0-12 (pve-manager/2.0/784729f4)
running kernel: 2.6.32-6-pve
proxmox-ve-2.6.32: 2.0-53
pve-kernel-2.6.32-6-pve: 2.6.32-53
lvm2: 2.02.86-1pve2
clvm: 2.02.86-1pve2
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-1
libqb: 0.6.0-1
redhat-cluster-pve: 3.1.7-1
pve-cluster: 1.0-12
qemu-server: 2.0-9
pve-firmware: 1.0-13
libpve-common-perl: 1.0-8
libpve-access-control: 1.0-2
libpve-storage-perl: 2.0-8
vncterm: 1.0-2
vzctl: 3.0.29-3pve3
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 0.15.0-1
ksm-control-daemon: 1.1-1
(qemu-server is one upgrade behind on one of the nodes , but probably does not have anything to do with the issue ).
this is what I did to create the cluster:
Code:
root@fbc1 /usr/bin # pvecm create fbc
Restarting pve cluster filesystem: pve-cluster.
Starting cluster:
Checking if cluster has been disabled at boot... [ OK ]
Checking Network Manager... [ OK ]
Global setup... [ OK ]
Loading kernel modules... [ OK ]
Mounting configfs... [ OK ]
Starting cman... [ OK ]
Waiting for quorum... [ OK ]
Starting fenced... [ OK ]
Starting dlm_controld... [ OK ]
Unfencing self... [ OK ]
Joining fence domain... [ OK ]
root@fbc1 /usr/bin # pvecm status
Version: 6.2.0
Config Version: 1
Cluster Name: fbc
Cluster Id: 703
Cluster Member: Yes
Cluster Generation: 4
Membership state: Cluster-Member
Nodes: 1
Expected votes: 1
Total votes: 1
Node votes: 1
Quorum: 1
Active subsystems: 5
Flags:
Ports Bound: 0
Node name: fbc1
Node ID: 1
Multicast addresses: 239.192.2.193
Node addresses: 10.100.100.1
then to add a node: ( note this line: Waiting for quorum... Timed-out waiting for cluster' )
Code:
root@fbc186 ~ # pvecm add 10.100.100.1
root@10.100.100.1's password:
copy corosync auth key
stopping pve-cluster service
Stopping pve cluster filesystem: pve-cluster.
backup old database
Starting pve cluster filesystem : pve-cluster.
Starting cluster:
Checking if cluster has been disabled at boot... [ OK ]
Checking Network Manager... [ OK ]
Global setup... [ OK ]
Loading kernel modules... [ OK ]
Mounting configfs... [ OK ]
Starting cman... [ OK ]
Waiting for quorum... Timed-out waiting for cluster
[FAILED]
cluster not ready - no quorum?
root@fbc186 ~ # pvecm status
Version: 6.2.0
Config Version: 2
Cluster Name: fbc
Cluster Id: 703
Cluster Member: Yes
Cluster Generation: 4
Membership state: Cluster-Member
Nodes: 1
Expected votes: 2
Total votes: 1
Node votes: 1
Quorum: 2 Activity blocked
Active subsystems: 1
Flags:
Ports Bound: 0
Node name: fbc186
Node ID: 2
Multicast addresses: 239.192.2.193
Node addresses: 10.100.100.186
this did not look correct on either node:
Code:
root@fbc186 ~ # pvecm nodes
Node Sts Inc Joined Name
1 X 0 fbc1
2 M 4 2011-12-01 21:03:48 fbc186
root@fbc1 ~ # pvecm nodes
Node Sts Inc Joined Name
1 M 4 2011-12-01 21:01:24 fbc1
2 X 0 fbc186
so i rebooted fbc186 then this part looks ok:
root@fbc1 ~ # pvecm nodes
Node Sts Inc Joined Name
1 M 4 2011-12-01 21:01:24 fbc1
2 M 12 2011-12-01 21:14:27 fbc186
[/code]
here is ls -la /etc/pve both nodes:
Code:
root@fbc186 /etc/pve # ls -la /etc/pve
total 5
drwxr-x--- 2 root www-data 0 Dec 31 1969 .
drwxr-xr-x 84 root root 4096 Dec 1 21:13 ..
-rw-r----- 1 root www-data 277 Dec 1 21:03 cluster.conf
-r--r----- 1 root www-data 153 Dec 31 1969 .clusterlog
-rw-r----- 1 root www-data 2 Dec 31 1969 .debug
lrwxr-x--- 1 root www-data 0 Dec 31 1969 local -> nodes/fbc186
-r--r----- 1 root www-data 223 Dec 31 1969 .members
lrwxr-x--- 1 root www-data 0 Dec 31 1969 openvz -> nodes/fbc186/openvz
lrwxr-x--- 1 root www-data 0 Dec 31 1969 qemu-server -> nodes/fbc186/qemu-server
-r--r----- 1 root www-data 200 Dec 31 1969 .rrd
-r--r----- 1 root www-data 230 Dec 31 1969 .version
-r--r----- 1 root www-data 18 Dec 31 1969 .vmlist
root@fbc1 /etc/pve # ls -la /etc/pve
total 16
drwxr-x--- 2 root www-data 0 Dec 31 1969 .
drwxr-xr-x 125 root root 12288 Dec 1 21:01 ..
-r--r----- 1 root www-data 451 Oct 31 12:53 authkey.pub
-r--r----- 1 root www-data 277 Dec 1 21:03 cluster.conf
-r--r----- 1 root www-data 228 Dec 1 21:03 cluster.conf.old
-r--r----- 1 root www-data 938 Dec 31 1969 .clusterlog
-rw-r----- 1 root www-data 2 Dec 31 1969 .debug
lr-xr-x--- 1 root www-data 0 Dec 31 1969 local -> nodes/fbc1
-r--r----- 1 root www-data 219 Dec 31 1969 .members
dr-xr-x--- 2 root www-data 0 Oct 31 12:53 nodes
lr-xr-x--- 1 root www-data 0 Dec 31 1969 openvz -> nodes/fbc1/openvz
dr-x------ 2 root www-data 0 Oct 31 12:53 priv
-r--r----- 1 root www-data 1533 Oct 31 12:53 pve-root-ca.pem
-r--r----- 1 root www-data 1675 Oct 31 12:53 pve-www.key
lr-xr-x--- 1 root www-data 0 Dec 31 1969 qemu-server -> nodes/fbc1/qemu-server
-r--r----- 1 root www-data 1243 Dec 31 1969 .rrd
-r--r----- 1 root www-data 216 Dec 1 21:09 storage.cfg
-r--r----- 1 root www-data 228 Dec 31 1969 .version
-r--r----- 1 root www-data 393 Dec 31 1969 .vmlist
-r--r----- 1 root www-data 281 Nov 26 19:38 vzdump.cron
note on fbc186 , nodes/fbc186/openvz is missing .