No connection between 2 VMs from 2 nodes of the same cluster

[Lamia]

Hello,

I have a cluster with 2 nodes. Both of them belong to the same LAN. I have a gateway to reach the internet. I do use NAT so that I can create as many VMs as I want, without giving them addresses from the LAN pool.

My problem is that a VM from the first server doesn't ping a VM from the second server and vice-versa. How to fix this? I added a VLAN to both nodes thinking that putting them in the same VLAN would allow communication, but I am not quite sure of what I am doing and I dont't want to mess up my configuration.

Here is the content of /etc/network/interfaces

***
auto lo
iface lo inet loopback

iface enp1s0 inet manual

auto enp1s0.99
iface enp1s0.99 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.10.246/24
gateway 192.168.10.254
bridge-ports enp1s0
bridge-stp off
bridge-fd 0

auto vmbr1
iface vmbr1 inet static
address 10.10.10.1/24
bridge-ports none
bridge-stp off
bridge-fd 0

post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE

***

Many thanks,

Lamia

 

[Pifouney]

Hey

Hum ... Yours nodes are together in NAT mode ? If yes, it's normal that one VM cannot ping an another VM on the 2nd node.

In your case, NAT mode allow only that:
VMs node1 can ping NODE2, BUT NOT THESES VMs
VMs node2 can ping NODE1, BUT NOT THESES VMs too

Create another bridge on each node without given them an hardware connected interfaces cannot help you without a really complex firewalling/routage.

If you want to try easyly that you seems to try, try that:

First, powerOff all VM on each node.

On node 1, we 're gonna use your 1rst virtual network: 10.10.10.0/24
For my part, i prefer to call x.x.x.254 for my gateway, but make that you want.
node 1 vmbr1 IP: 10.10.10.254

On node 2, you need to modify the network name. Here, gonna take 10.20.20.0/24
node 2 vmbr1 IP: 10.20.20.254

On each node, you need a little bit FW rules:
node1
iptables -t filter -A forward -s 10.10.10.0/24 -j ACCEPT
iptables -t filter -A forward -d 10.10.10.0/24 -j ACCEPT
iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
route add -net 10.20.20.0/24 gw IP_NODE2-on-yourLannetwork

node2:
iptables -t filter -A forward -s 10.20.20.0/24 -j ACCEPT
iptables -t filter -A forward -d 10.20.20.0/24 -j ACCEPT

iptables -t nat -A POSTROUTING -s '10.20.20.0/24' -o vmbr0 -j MASQUERADE
route add -net 10.10.10.0/24 gw IP_NODE1-on-yourLannetwork

Up VMs and modify theirs network configuration. After it, it normally does to work ( but, it's not really clean ^^)

 

[Lamia]

Thanks a lot for your response.

 

[Lamia]

Hi,

I have changed my interfaces /etc/network/interfaces according to the response given above and it didn't solve the problem (I also deleted the VLAN I created since it was not useful).

My questions are:
1. Should I change the /etc/pve/nodes/<nodename>/host.fw file?
2. /etc/pve/nodes/<nodename>/host.fw doesn't exist. Should I create it manually?
3. Should I put the 4 lines in the firewall file or should I put those 2 lines related to the NAT and the route in the /etc/network/interfaces file?

Thanks,

Lamia