interesting issue with cluster

[Techie]

I just created a cluster about a month or so ago and noticed a few things that seemed rather off.

the address of what is to be my master machine is 192.168.0.4

Version: 6.2.0Config Version: 1
Cluster Id: 7732
Cluster Member: Yes
Cluster Generation: 92
Membership state: Cluster-Member
Nodes: 1
Expected votes: 1
Total votes: 1
Node votes: 1
Quorum: 1
Active subsystems: 5
Flags:
Ports Bound: 0
Node name: server1
Node ID: 1
Multicast addresses: 239.192.30.82
Node addresses: 192.168.0.4

vmbr1     Link encap:Ethernet  HWaddr 00:30:67:ac:32:13
          inet addr:192.168.0.4  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::230:67ff:feac:3213/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:23155 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14773 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:6614056 (6.3 MiB)  TX bytes:3664297 (3.4 MiB)

but when i log in to the gui and go to status: server address: 192.168.0.3

so for obvious reasons my other machine @192.168.0.2 is unable to properly communicate. i've tried everything to get this to point at the right place but i'm just not able to find where this address is living. i've cleared the cluster configs off both boxes, rebooted, recreated the cluster. still points over to .3

i do have a 3rd nic installed that i did have on .3 but it's currently disabled until i can figure out what's going on with the storage issue so i can get the virtual firewall installed.

 

[udo]

I just created a cluster about a month or so ago and noticed a few things that seemed rather off.

the address of what is to be my master machine is 192.168.0.4

Version: 6.2.0Config Version: 1
Cluster Id: 7732
Cluster Member: Yes
Cluster Generation: 92
Membership state: Cluster-Member
Nodes: 1
Expected votes: 1
Total votes: 1
Node votes: 1
Quorum: 1
Active subsystems: 5
Flags:
Ports Bound: 0
Node name: server1
Node ID: 1
Multicast addresses: 239.192.30.82
Node addresses: 192.168.0.4

vmbr1     Link encap:Ethernet  HWaddr 00:30:67:ac:32:13
          inet addr:192.168.0.4  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::230:67ff:feac:3213/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:23155 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14773 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:6614056 (6.3 MiB)  TX bytes:3664297 (3.4 MiB)

but when i log in to the gui and go to status: server address: 192.168.0.3

so for obvious reasons my other machine @192.168.0.2 is unable to properly communicate. i've tried everything to get this to point at the right place but i'm just not able to find where this address is living. i've cleared the cluster configs off both boxes, rebooted, recreated the cluster. still points over to .3

i do have a 3rd nic installed that i did have on .3 but it's currently disabled until i can figure out what's going on with the storage issue so i can get the virtual firewall installed.

Hi,
in short:
1. all nodes in a pve 2.x cluster are "master" - there aren't the master/client thing like in pve1.x.
2. your cluster has only one node?!
3. you should use vmbr0 for clustercommunication instead of vmbr1

4. If you connect to with an browser to .4 and reach .3 look at your network with tcpdump! (or in /etc/hosts - perhaps there are wrong values?!)

Udo

 

[Techie]

Hi Udo,

thanks for the reply!

2 node cluster
both nodes have bonded NICs, the bond on server1 is using vmbr1

vmbr0 on server1 is my 3rd nic that i don't intend on using for the cluster management/internal networking at all. that one is going to be my WAN(eventually). for the sake of testing, i just changed the bond over to vmbr0, changed my IP scheme over to .4 and rebooted. server2 is still seeing server1 on .3. rebooted server2...server1 still listed as .3 in the gui.

there's nothing about the IPs in the cluster.conf

when i created the cluster, I did

pvecm add 192.168.0.4 -force

did -force because it gave me some issues without it.

think i'm just going to pull the backups on the drives and start from scratch. clean slate...won't be too big of an issue considering neither box has any VMs running yet

 

[udo]

Hi Udo,

thanks for the reply!

2 node cluster
both nodes have bonded NICs, the bond on server1 is using vmbr1

Hi,
again - cluster-communication should be done with vmbr0!

vmbr0 on server1 is my 3rd nic that i don't intend on using for the cluster management/internal networking at all. that one is going to be my WAN(eventually).

The assignment of nics to bridges can be changed on /etc/network/interfaces and/or /etc/udev/rules.d/70-persistent-net.rules

for the sake of testing, i just changed the bond over to vmbr0, changed my IP scheme over to .4 and rebooted. server2 is still seeing server1 on .3. rebooted server2...server1 still listed as .3 in the gui.

there's nothing about the IPs in the cluster.conf

when i created the cluster, I did

pvecm add 192.168.0.4 -force

did -force because it gave me some issues without it.

think i'm just going to pull the backups on the drives and start from scratch. clean slate...won't be too big of an issue considering neither box has any VMs running yet

are the right entrys in /etc/hosts?
Does multicast work? See here http://pve.proxmox.com/wiki/Multicast_notes

Udo