Hello,
I have been reading this and other forums for potential solutions but have hit a wall. I have a two node proxmox cluster I tried to set up. I have verified multicast is working yet the nodes are unable to quorum with one another. Quorum times out. These two servers are in different geographic locations and connecting via VPN but like I said, multicast is indeed working. Trying to start the cluster manager on the master gives me a quorum timeout?
Master node (hostname: proxmox) is at IP 10.118.60.1
2nd node (hostname: thor) is at IP 10.118.60.159 (This node was deleted when I figured I would just try fixing the problem by reinstalling. However the node still appears on the config for the 3rd node)
3rd node (hostname: loki) is at IP 10.118.60.5
Master:
Node:
I try to start the cluster manager on the master:
I try to start the cluster manager on the node:
Ok, lets make sure multicast is working. I run ssmpingd on the node loki:
and asmping on the master:
from loki node:
Just to be exhaustive lets do the reverse and run ssmpingd on the master and asmping on the node:
Edit: I also tested multicast with omping:
I have been reading this and other forums for potential solutions but have hit a wall. I have a two node proxmox cluster I tried to set up. I have verified multicast is working yet the nodes are unable to quorum with one another. Quorum times out. These two servers are in different geographic locations and connecting via VPN but like I said, multicast is indeed working. Trying to start the cluster manager on the master gives me a quorum timeout?
Master node (hostname: proxmox) is at IP 10.118.60.1
2nd node (hostname: thor) is at IP 10.118.60.159 (This node was deleted when I figured I would just try fixing the problem by reinstalling. However the node still appears on the config for the 3rd node)
3rd node (hostname: loki) is at IP 10.118.60.5
Master:
Code:
root@proxmox:/# pvecm status
Version: 6.2.0
Config Version: 4
Cluster Name: joecluster
Cluster Id: 42548
Cluster Member: Yes
Cluster Generation: 24
Membership state: Cluster-Member
Nodes: 1
Expected votes: 2
Total votes: 1
Node votes: 1
Quorum: 2 Activity blocked
Active subsystems: 1
Flags:
Ports Bound: 0
Node name: proxmox
Node ID: 1
Multicast addresses: 239.192.166.218
Node addresses: 10.1.0.3
root@loki:~# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
10.2.0.149 loki.pxnodes loki
10.118.60.5 pvelocalhost
10.118.60.1 proxmox
root@loki:~# pvecm nodes
Node Sts Inc Joined Name
1 X 0 proxmox
2 X 0 thor
3 M 12 2013-07-18 22:56:51 loki
root@loki:~# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.118.60.0 * 255.255.255.0 U 0 0 0 openvpnbr0
10.118.60.0 * 255.255.255.0 U 0 0 0 tap0
10.2.0.0 * 255.255.255.0 U 0 0 0 vmbr0
224.0.0.0 * 240.0.0.0 U 0 0 0 openvpnbr0
default Wart2 0.0.0.0 UG 0 0 0 vmbr0Node:
Code:
root@loki:~# pvecm status
Version: 6.2.0
Config Version: 3
Cluster Name: joecluster
Cluster Id: 42548
Cluster Member: Yes
Cluster Generation: 12
Membership state: Cluster-Member
Nodes: 1
Expected votes: 3
Total votes: 1
Node votes: 1
Quorum: 2 Activity blocked
Active subsystems: 1
Flags:
Ports Bound: 0
Node name: loki
Node ID: 3
Multicast addresses: 239.192.166.218
Node addresses: 10.2.0.149
root@proxmox:/# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
10.1.0.3 proxmox.lan proxmox
10.118.60.1 pvelocalhost
root@proxmox:/# pvecm nodes
Node Sts Inc Joined Name
1 M 24 2013-07-19 00:04:54 proxmox
3 X 0 loki
root@proxmox:/# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.1.0.7 * 255.255.255.255 UH 0 0 0 venet0
10.118.60.0 * 255.255.255.0 U 0 0 0 openvpnbr0
10.1.0.0 * 255.255.255.0 U 0 0 0 vmbr0
224.0.0.0 * 240.0.0.0 U 0 0 0 openvpnbr0
default Wart 0.0.0.0 UG 0 0 0 vmbr0I try to start the cluster manager on the master:
Code:
root@proxmox:/# /etc/init.d/cman status
fenced is stopped
root@proxmox:/# /etc/init.d/cman start
Starting cluster:
Checking if cluster has been disabled at boot... [ OK ]
Checking Network Manager... [ OK ]
Global setup... [ OK ]
Loading kernel modules... [ OK ]
Mounting configfs... [ OK ]
Starting cman... [ OK ]
Waiting for quorum... Timed-out waiting for cluster
[FAILED]I try to start the cluster manager on the node:
Code:
root@loki:~# /etc/init.d/cman status
fenced is stopped
root@loki:~# /etc/init.d/cman start
Starting cluster:
Checking if cluster has been disabled at boot... [ OK ]
Checking Network Manager... [ OK ]
Global setup... [ OK ]
Loading kernel modules... [ OK ]
Mounting configfs... [ OK ]
Starting cman... [ OK ]
Waiting for quorum... Timed-out waiting for cluster
[FAILED]Ok, lets make sure multicast is working. I run ssmpingd on the node loki:
Code:
root@loki:~# ssmpingd
Code:
root@proxmox:/# asmping 224.0.2.1 10.118.60.5
asmping joined (S,G) = (*,224.0.2.234)
pinging 10.118.60.5 from 10.118.60.1
unicast from 10.118.60.5, seq=1 dist=0 time=67.314 ms
multicast from 10.118.60.5, seq=1 dist=0 time=126.482 ms
unicast from 10.118.60.5, seq=2 dist=0 time=20.631 ms
multicast from 10.118.60.5, seq=2 dist=0 time=38.674 ms
unicast from 10.118.60.5, seq=3 dist=0 time=375.337 ms
multicast from 10.118.60.5, seq=3 dist=0 time=446.936 ms
unicast from 10.118.60.5, seq=4 dist=0 time=22.709 ms
Code:
root@loki:~# ssmpingd
received request from 10.118.60.1
received request from 10.118.60.1
received request from 10.118.60.1Just to be exhaustive lets do the reverse and run ssmpingd on the master and asmping on the node:
Code:
root@proxmox:/# ssmpingd
Code:
root@loki:~# asmping 224.0.2.1 10.118.60.1
asmping joined (S,G) = (*,224.0.2.234)
pinging 10.118.60.1 from 10.118.60.5
unicast from 10.118.60.1, seq=1 dist=0 time=78.835 ms
multicast from 10.118.60.1, seq=1 dist=0 time=96.192 ms
unicast from 10.118.60.1, seq=2 dist=0 time=22.973 ms
multicast from 10.118.60.1, seq=2 dist=0 time=45.023 ms
unicast from 10.118.60.1, seq=3 dist=0 time=44.340 ms
multicast from 10.118.60.1, seq=3 dist=0 time=72.089 ms
Code:
root@proxmox:/# ssmpingd
received request from 10.118.60.5
received request from 10.118.60.5
received request from 10.118.60.5
received request from 10.118.60.5
received request from 10.118.60.5Edit: I also tested multicast with omping:
Code:
root@loki:~# omping 10.118.60.1 10.118.60.510.118.60.1 : waiting for response msg
10.118.60.1 : waiting for response msg
10.118.60.1 : waiting for response msg
10.118.60.1 : joined (S,G) = (*, 232.43.211.234), pinging
10.118.60.1 : unicast, seq=1, size=69 bytes, dist=0, time=83.576ms
10.118.60.1 : multicast, seq=1, size=69 bytes, dist=0, time=143.066ms
10.118.60.1 : unicast, seq=2, size=69 bytes, dist=0, time=55.412ms
10.118.60.1 : multicast, seq=2, size=69 bytes, dist=0, time=116.422ms
10.118.60.1 : unicast, seq=3, size=69 bytes, dist=0, time=49.987ms
10.118.60.1 : multicast, seq=3, size=69 bytes, dist=0, time=106.437ms
10.118.60.1 : unicast, seq=4, size=69 bytes, dist=0, time=52.979ms
Code:
root@proxmox:/# omping 10.118.60.1 10.118.60.510.118.60.5 : waiting for response msg
10.118.60.5 : joined (S,G) = (*, 232.43.211.234), pinging
10.118.60.5 : unicast, seq=1, size=69 bytes, dist=0, time=84.582ms
10.118.60.5 : multicast, seq=1, size=69 bytes, dist=0, time=104.628ms
10.118.60.5 : unicast, seq=2, size=69 bytes, dist=0, time=29.353ms
10.118.60.5 : multicast, seq=2, size=69 bytes, dist=0, time=91.524ms
10.118.60.5 : unicast, seq=3, size=69 bytes, dist=0, time=22.349ms
10.118.60.5 : multicast, seq=3, size=69 bytes, dist=0, time=84.465ms
10.118.60.5 : unicast, seq=4, size=69 bytes, dist=0, time=27.698ms
10.118.60.5 : multicast, seq=4, size=69 bytes, dist=0, time=86.316ms
Last edited: