Proxmox v4 - lost quorum

[xpanisello]

Hi,

I updated (really new installation) from 3.4 to 4, but now I'm having problems with quorum. There are, by now, only 3 HP Proliant DL360 gen 8. I installed v4 without problems two days ago, and today the quorum has lost. I don't know why, I have been configuring Netapp iSCSI and NFS 4, during this process there has been some errors and reconfigurations about it, but finally it works, but the quorum has lost. I tryed to stop and start all nodes, but no change.

How can I restore the quorum?

 

[RobFantini]

from each node can you ping all the other nodes?

 

[xpanisello]

Hi, yes, all the nodes has four 10GbaseT, 2 for LAN and 2 for SAN (using LACP), moreover the Ilo connection. I can ping from all the lans the other nodes. All the nodes exists in /etc/hosts and dns resolution is working. Do you know which connection uses Proxmox for heartbeat? Can I specify it?

 

[RobFantini]

Try something like this to isolate pve network from vm's:

# for pve hosts
auto eth0  
iface eth0  inet static
        address  10.1.10.42
        netmask  255.255.255.0
        gateway  10.1.10.1

# for vm's	
iface eth1 inet manual

auto vmbr0
iface vmbr0 inet manual
        bridge_ports eth1
        bridge_stp off
        bridge_fd 0

We had extra nics around , so I'm using 3 vmbr's per host and splitting up traffic.

I have a lot of network traffic on vm's which sometimes caused cluster to break. Using different nics along with vlans fixed the issue so far.

 

[xpanisello]

Yes, but pve hosts are connected to two lans. I have 4 nics on each, 2 LACP groups, one for lan (172.22.0.0, eth0 and eth2) and one for SAN (172.18.0.0, eth1 and eth3). Which interface uses Proxmox for heartbeat (I guess that uses multicast)? Where can I found this configuration? In order to ensure the heartbeat interface has activated multicast and quality connection.

 

[RobFantini]

as far as i know multicast traffic is only on the nic that the pve host IP address uses. I'm not an expert!

also there is a multicast wiki page - in case you want testing tips.

 

[xpanisello]

Yes, but which ip address, the pve hosts has two ip addresses.

Thanks, I'm trying to ensure that multicast is permited in both lans (LAN and SAN), the swith support team is wotking to ensure it. About the traffic, the switches are Extreme Summit, I hope they can support more, more, a lot more, traffic than actual, by now I just have two VM and I'm preparing to import ten more that actually are in another VMware cluster.

 

[RobFantini]

ping the node name to get the address.

 

[xpanisello]

UUFFF... solved!!!

The problem was the multicast. It was enabled in the switches, but in order to work the switch needs an IP address in each vlan, and there was NOT. When the support team has configured the IP addresses the quorum has reached, and now is well working.

Thanks for all RobFantini!!

 

[RobFantini]

which type of switch is being used? can your support team post the make / model and solution?

we're using netgear gsm7300 series and i'd like to see the setting to compare settings.

 

[xpanisello]

Hi, sorry for no reply before, but i had spam problems, someone in my lan has been doing spam, a lot of spam... but solved by now.

Our switches are two Extreme Summit X670 with 48x10Gbase-T ports, plus 4 SFP+...

The switches had only one ip address configured for administration, this interface is in the administration vlan. When I asked them for multicast, they activated the multicast in the LAN vlan and in the SAN vlan but it didn't work, until they saw in the switches manual that it is necessary, for the multicast Works, the switches to have an IP adddress in the vlan in which you want to enable multicast, when the ip was assigned the quòrum was restablished in seconds.

Do you need more details?
I can send you the commands to enable multicast and for assign the ip address but probably Netgear has another commands set.

 

[RobFantini]

that is good ,thank you.