One question about clustering in Proxmox 2.x

ayron24

New Member
May 10, 2012
2
0
1
Hello, I new here and I would like say thank to the team of PVE for an fantastic virtualization system.

I have a question about clustering in Proxmox 2.x:

Currently, I have 4 nodes on cluster with Proxmox VE 2.1. One of them had a break in the hard disk and the had to change the disk for a new one, so I had to install the system in de new disk.

I tried to join the cluster again

pvecm add IP-MASTER-CLUSTER

and gives me this error and there does not continue

Loading kernel modules ... [OK]
Mounting configfs ... [OK]
Starting cman ... [OK]
Waiting for quorum ... Timed-out waiting for cluster
[FAILED]
waiting for quorum ...


I probe also with

pvecm add IP-MASTER-CLUSTER -force

and gives me the same above error

In Proxmox 1.x this was easy to do, and I read the proxmox 2 wiki and it say, there is no way to do that.

So, there is any way to add that node again in the cluster with the same name that had previously???



thanks for the helps
 
do you have any issue regarding cluster communications? does you IP network supports IP multicast?
 
I have the same problem with a fresh installation and two servers.

I'm using OpenVPN in order to have Multicast working.

Server 2 to Server 1 :

asmping 239.192.218.143 192.168.2.1 asmping joined (S,G) = (*,239.192.218.234)
pinging 192.168.2.1 from 192.168.2.2
unicast from 192.168.2.1, seq=1 dist=0 time=3.126 ms
multicast from 192.168.2.1, seq=1 dist=0 time=43.073 ms
unicast from 192.168.2.1, seq=2 dist=0 time=1.283 ms
multicast from 192.168.2.1, seq=2 dist=0 time=1.849 ms

Server 1 to Server 2

asmping 239.192.218.143 192.168.2.2
asmping joined (S,G) = (*,239.192.218.234)
pinging 192.168.2.2 from 192.168.2.1
multicast from 192.168.2.2, seq=1 dist=0 time=1.402 ms
unicast from 192.168.2.2, seq=1 dist=0 time=42.298 ms
unicast from 192.168.2.2, seq=2 dist=0 time=1.343 ms
multicast from 192.168.2.2, seq=2 dist=0 time=1.951 ms

Any ideas ?
 
Hi, i have 2 servers in OVH. I think that supports IP multicast.

Server 1

ssmpingd
received request from 94.23.32.x
received request from 94.23.32.x
received request from 94.23.32.x
received request from 94.23.32.x
received request from 94.23.32.x
received request from 94.23.32.x
received request from 94.23.32.x
received request from 94.23.32.x


Server 2


asmping 239.192.36.74 94.23.6.x
asmping joined (S,G) = (*,239.192.36.234)
pinging 94.23.6.x from 94.23.32.x
unicast from 94.23.6.x, seq=1 dist=1 time=0.330 ms
unicast from 94.23.6.x, seq=2 dist=1 time=0.254 ms
unicast from 94.23.6.x, seq=3 dist=1 time=0.286 ms
unicast from 94.23.6.x, seq=4 dist=1 time=0.271 ms
unicast from 94.23.6.x, seq=5 dist=1 time=0.245 ms
unicast from 94.23.6.x, seq=6 dist=1 time=0.224 ms
unicast from 94.23.6.x, seq=7 dist=1 time=0.257 ms
unicast from 94.23.6.x, seq=8 dist=1 time=0.238 ms
^C
--- 94.23.6.x statistics ---
8 packets transmitted, time 7863 ms
unicast:
8 packets received, 0% packet loss
rtt min/avg/max/std-dev = 0.224/0.263/0.330/0.032 ms
multicast:
0 packets received, 100% packet loss



Is this ok?
 
no ok. try udpu instead of IP multicast for corosync transport method.

add transport="udpu" to your cluster.conf. And not forget to increase (config_version number).

Code:
nano /etc/pve/cluster.conf

Code:
<?xml version="1.0"?>
<cluster config_version="48" name="yourclustername">
  <cman keyfile="/var/lib/pve-cluster/corosync.authkey" [COLOR=#ff0000]transport="udpu"[/COLOR]/>
...
 
Is there anything to do to apply the new cluster configuration after editing the xml ?
 
Is there anything to do to apply the new cluster configuration after editing the xml ?

It requires a cluster-wide restart to change transport mechanisms. I suggest you test this change in your network before you apply it to your live systems.
 
and also add all your cluster nodes in /etc/hosts
 
And would you have any idea why the Quroum step of "pvecm add ip" time out ? (considering the multicast is working)
 
And would you have any idea why the Quroum step of "pvecm add ip" time out ? (considering the multicast is working)

if there is a timeout there seems to be a communication problem. I cannot see this problem here.
 
I just found what my problem was. Corosync log showed me that it was using the wrong ip (it needed to use the VPN ip, but it didn't).

I forgot to add "pvelocalhost" at the end of "xxx.xxx.xxx.xxx myserver2.vpn.local myserver2 pvelocalhost" in "/etc/hosts".

Now it uses the right ip (the vpn ip) so the multicast work and the quorum step doesn't timeout !

:)
 
no ok. try udpu instead of IP multicast for corosync transport method.

add transport="udpu" to your cluster.conf. And not forget to increase (config_version number).

Code:
nano /etc/pve/cluster.conf

Code:
<?xml version="1.0"?>
<cluster config_version="48" name="yourclustername">
  <cman keyfile="/var/lib/pve-cluster/corosync.authkey" [COLOR=#ff0000]transport="udpu"[/COLOR]/>
...



After change cluster.conf in Server 1 (with VM's running), restart this, reinstall proxmox in Server 2 and execute in Server 2 the command
Code:
pvecm add ip_of_server_1
the result is the same:

Code:
copy corosync auth keystopping pve-cluster service
Stopping pve cluster filesystem: pve-cluster.
backup old database
Starting pve cluster filesystem : pve-cluster.
Starting cluster:
   Checking if cluster has been disabled at boot... [  OK  ]
   Checking Network Manager... [  OK  ]
   Global setup... [  OK  ]
   Loading kernel modules... [  OK  ]
   Mounting configfs... [  OK  ]
   Starting cman... [  OK  ]
   Waiting for quorum... Timed-out waiting for cluster
[FAILED]
waiting for quorum...

Any more ideas?
 
But Tom said that use udp protocol instead multicast, am i in certain?
 
Oh you want to use Udp, ok, i didn't understood that.
Then i can't help you sorry. If you can't get it working after some time, consider using a VPN.
 
I understand use udp unicast instead multicast, but i am not sure.

@Tom, am i in certain?
 
did you add all members in /etc/hosts (on all nodes)?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!