3.4 clsuter install failing - multicast problems

J

jstableau

New Member
Apr 10, 2015
8
0
1
I've tried several times over the past week to install 3.4 pve cluster on 4 nodes and keep running into a problem where nodes stop receiving multicast traffic, while continuing to accept unicast traffic. In the output below you'll see the output of omping showing the 01 node cut out of communication.

My install process was as follows. I installed the base image and set up /etc/hosts. Ran "pvecm create pxmx-c01", then ssh'd from 01 into each of the other three nodes and ran "pvevm add pxmx01" on each.

The nodes are all in the same vlan, IGMP snooping is enabled and all cluster members are added into /etc/hosts.

From 1001:

pxmxv1002 : unicast, xmt/rcv/%loss = 17/17/0%, min/avg/max/std-dev = 0.062/0.142/0.196/0.033
pxmxv1002 : multicast, xmt/rcv/%loss = 17/0/100%, min/avg/max/std-dev = 0.000/0.000/0.000/0.000
pxmxv1003 : unicast, xmt/rcv/%loss = 15/15/0%, min/avg/max/std-dev = 0.058/0.142/0.169/0.031
pxmxv1003 : multicast, xmt/rcv/%loss = 15/0/100%, min/avg/max/std-dev = 0.000/0.000/0.000/0.000
pxmxv1004 : unicast, xmt/rcv/%loss = 12/12/0%, min/avg/max/std-dev = 0.057/0.138/0.166/0.034
pxmxv1004 : multicast, xmt/rcv/%loss = 12/0/100%, min/avg/max/std-dev = 0.000/0.000/0.000/0.000

root@pxmxv1001:~# pvecm nodes
Node Sts Inc Joined Name
1 M 8 2015-04-11 18:56:42 pxmxv1001
2 X 0 pxmxv1002
3 X 0 pxmxv1003
4 X 0 pxmxv1004

root@pxmxv1001:~# pvecm status
Version: 6.2.0
Config Version: 4
Cluster Name: pxmx-c01
Cluster Id: 22949
Cluster Member: Yes
Cluster Generation: 8
Membership state: Cluster-Member
Nodes: 1
Expected votes: 4
Total votes: 1
Node votes: 1
Quorum: 3 Activity blocked
Active subsystems: 1
Flags:
Ports Bound: 0
Node name: pxmxv1001
Node ID: 1
Multicast addresses: 239.192.89.254
Node addresses: 10.194.0.50


1002:
239.192.89.254 : response message never received
pxmxv1001 : unicast, xmt/rcv/%loss = 18/18/0%, min/avg/max/std-dev = 0.075/0.144/0.174/0.024
pxmxv1001 : multicast, xmt/rcv/%loss = 18/0/100%, min/avg/max/std-dev = 0.000/0.000/0.000/0.000
pxmxv1003 : unicast, xmt/rcv/%loss = 19/19/0%, min/avg/max/std-dev = 0.067/0.137/0.168/0.030
pxmxv1003 : multicast, xmt/rcv/%loss = 19/19/0%, min/avg/max/std-dev = 0.084/0.149/0.184/0.028
pxmxv1004 : unicast, xmt/rcv/%loss = 15/15/0%, min/avg/max/std-dev = 0.058/0.139/0.163/0.030
pxmxv1004 : multicast, xmt/rcv/%loss = 15/15/0%, min/avg/max/std-dev = 0.067/0.150/0.181/0.032


root@pxmxv1002:~# pvecm nodes
Node Sts Inc Joined Name
1 X 0 pxmxv1001
2 M 4 2015-04-10 19:22:43 pxmxv1002
3 M 12 2015-04-11 17:59:41 pxmxv1003

root@pxmxv1002:~# pvecm status
Version: 6.2.0
Config Version: 3
Cluster Name: pxmx-c01
Cluster Id: 22949
Cluster Member: Yes
Cluster Generation: 12
Membership state: Cluster-Member
Nodes: 2
Expected votes: 3
Total votes: 2
Node votes: 1
Quorum: 2 Activity blocked
Active subsystems: 1
Flags: Error
Ports Bound: 0
Node name: pxmxv1002
Node ID: 2
Multicast addresses: 239.192.89.254
Node addresses: 10.194.0.51


From 1003:
239.192.89.254 : response message never received
pxmxv1002 : unicast, xmt/rcv/%loss = 17/17/0%, min/avg/max/std-dev = 0.066/0.146/0.173/0.024
pxmxv1002 : multicast, xmt/rcv/%loss = 17/15/11% (seq>=3 0%), min/avg/max/std-dev = 0.145/0.169/0.191/0.015
pxmxv1003 : unicast, xmt/rcv/%loss = 18/18/0%, min/avg/max/std-dev = 0.061/0.142/0.164/0.030
pxmxv1003 : multicast, xmt/rcv/%loss = 18/17/5% (seq>=2 0%), min/avg/max/std-dev = 0.075/0.160/0.179/0.024
pxmxv1001 : unicast, xmt/rcv/%loss = 14/14/0%, min/avg/max/std-dev = 0.057/0.144/0.172/0.029
pxmxv1001 : multicast, xmt/rcv/%loss = 14/0/100%, min/avg/max/std-dev = 0.000/0.000/0.000/0.000

root@pxmxv1003:~# pvecm nodes
Node Sts Inc Joined Name
1 X 0 pxmxv1001
2 M 8 2015-04-10 19:30:24 pxmxv1002
3 M 4 2015-04-10 19:30:23 pxmxv1003


root@pxmxv1003:~# pvecm status
Version: 6.2.0
Config Version: 3
Cluster Name: pxmx-c01
Cluster Id: 22949
Cluster Member: Yes
Cluster Generation: 12
Membership state: Cluster-Member
Nodes: 2
Expected votes: 3
Total votes: 2
Node votes: 1
Quorum: 2
Active subsystems: 5
Flags: Error
Ports Bound: 0
Node name: pxmxv1003
Node ID: 3
Multicast addresses: 239.192.89.254
Node addresses: 10.194.0.52


1004:

pxmxv1002 : unicast, xmt/rcv/%loss = 17/17/0%, min/avg/max/std-dev = 0.066/0.146/0.173/0.024
pxmxv1002 : multicast, xmt/rcv/%loss = 17/15/11% (seq>=3 0%), min/avg/max/std-dev = 0.145/0.169/0.191/0.015
pxmxv1003 : unicast, xmt/rcv/%loss = 18/18/0%, min/avg/max/std-dev = 0.061/0.142/0.164/0.030
pxmxv1003 : multicast, xmt/rcv/%loss = 18/17/5% (seq>=2 0%), min/avg/max/std-dev = 0.075/0.160/0.179/0.024
pxmxv1001 : unicast, xmt/rcv/%loss = 14/14/0%, min/avg/max/std-dev = 0.057/0.144/0.172/0.029
pxmxv1001 : multicast, xmt/rcv/%loss = 14/0/100%, min/avg/max/std-dev = 0.000/0.000/0.000/0.000

root@pxmxv1004:~# pvecm nodes
Node Sts Inc Joined Name
1 X 0 pxmxv1001
2 M 12 2015-04-11 17:59:42 pxmxv1002
3 M 12 2015-04-11 17:59:41 pxmxv1003
4 M 4 2015-04-11 17:59:40 pxmxv1004

root@pxmxv1004:~# pvecm status
Version: 6.2.0
Config Version: 4
Cluster Name: pxmx-c01
Cluster Id: 22949
Cluster Member: Yes
Cluster Generation: 12
Membership state: Cluster-Member
Nodes: 3
Expected votes: 4
Total votes: 3
Node votes: 1
Quorum: 3
Active subsystems: 5
Flags:
Ports Bound: 0
Node name: pxmxv1004
Node ID: 4
Multicast addresses: 239.192.89.254
Node addresses: 10.194.0.53
 
Linux pxmxv1001 2.6.32-37-pve #1 SMP Wed Feb 11 10:00:27 CET 2015 x86_64 GNU/Linux - direct install from ISO. No bonding - single 10GbE. No querier configured on the switch.
 
You must log in or register to reply here.
Top Bottom