Migration between nodes brings down network

[da5id]

Hello all,

I have two nodes in a cluster.
The main node has a public IP address 198.0.x.x
The joined node has an internal IP address 10.1.x.x

When I try and migrate a VM from the main node it causes my entire network to crater.

My network is set up thusly:
Comcast modem - switch - node1 node2

I looked through logs but I couldn't find anything really telling. I am wondering if this is caused by traversing different subnets. Any help would be appreciated.

 

[e100]

From your description it is not entirely clear how your network is configured.
What are the contents of /etc/network/interfaces on both servers?

 

[da5id]

Node1
# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
address 198.0.x.x
netmask 255.255.255.248
gateway 198.0.x.x
bridge_ports eth0
bridge_stp off
bridge_fd 0

Node2

auto lo
iface lo inet loopback

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
address 10.1.10.221
netmask 255.255.255.0
gateway 10.1.10.1
bridge_ports eth1
bridge_stp off
bridge_fd 0

From your description it is not entirely clear how your network is configured.
What are the contents of /etc/network/interfaces on both servers?

 

[nethfel]

Ok, unless I'm misreading this - it looks like the two nodes are on completely separate networks - how are you even getting them within the same cluster to be able to try a migration? Also I see on Node2 you have eth0 marked as inet manual, but no apparent configuration... what is that?

 

[da5id]

I was able to cluster the nodes just fine. I did have to use UDPU instead of multicast.

The modem is a Comcast Business box, and I of course have static IP addresses.

The eth0 config is me being terrible. I have now removed it.

I'm assuming my troubles stem from being on two different subnets and the modem/router is having a stroke.

I appreciate the assistance.

Ok, unless I'm misreading this - it looks like the two nodes are on completely separate networks - how are you even getting them within the same cluster to be able to try a migration? Also I see on Node2 you have eth0 marked as inet manual, but no apparent configuration... what is that?

 

[e100]

My suggestion is to do one of the following:

1.
Use a 2nd switch and 2nd network port for the proxmox cluster communications.
You have to edit /etc/hosts so it has IP address of the interface you want proxmox to use.
If you only have two nodes for now you could use a cat5 cable as your 2nd "switch" for this other network port.
If your interface is 1000M, any cat5 cable will do, if 10/100M you need a crossover cable.

2.
If you have a managed switch setup vlans.
Put your comcast network on one vlan and your proxmox cluster communications on another vlan.
This can all run through one network port with the proper network configuration.

3. if your comcast router is actually doing NAT (most likely is) then simply put both nodes on the same 192.0.x.x network and avoid any further complexities.
Make sure whatever static IP address you assign in proxmox in not going to be assigned to something else via DHCP from your comcast router.

 

[da5id]

Thanks for your assistance!

I will set up a second network port for cluster communications as suggested.

I put both machines on the same network and everything is working as intended, now. Again, I appreciate the help.

My suggestion is to do one of the following:

1.
Use a 2nd switch and 2nd network port for the proxmox cluster communications.
You have to edit /etc/hosts so it has IP address of the interface you want proxmox to use.
If you only have two nodes for now you could use a cat5 cable as your 2nd "switch" for this other network port.
If your interface is 1000M, any cat5 cable will do, if 10/100M you need a crossover cable.

2.
If you have a managed switch setup vlans.
Put your comcast network on one vlan and your proxmox cluster communications on another vlan.
This can all run through one network port with the proper network configuration.

3. if your comcast router is actually doing NAT (most likely is) then simply put both nodes on the same 192.0.x.x network and avoid any further complexities.
Make sure whatever static IP address you assign in proxmox in not going to be assigned to something else via DHCP from your comcast router.