No connection between 2 VMs from 2 nodes of the same cluster

Lamia

Member
Jun 25, 2020
24
0
6
32
Hello,

I have a cluster with 2 nodes. Both of them belong to the same LAN. I have a gateway to reach the internet. I do use NAT so that I can create as many VMs as I want, without giving them addresses from the LAN pool.

My problem is that a VM from the first server doesn't ping a VM from the second server and vice-versa. How to fix this? I added a VLAN to both nodes thinking that putting them in the same VLAN would allow communication, but I am not quite sure of what I am doing and I dont't want to mess up my configuration.

Here is the content of /etc/network/interfaces

***
auto lo
iface lo inet loopback

iface enp1s0 inet manual

auto enp1s0.99
iface enp1s0.99 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.10.246/24
gateway 192.168.10.254
bridge-ports enp1s0
bridge-stp off
bridge-fd 0

auto vmbr1
iface vmbr1 inet static
address 10.10.10.1/24
bridge-ports none
bridge-stp off
bridge-fd 0

post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE

***

Many thanks,

Lamia
 

Pifouney

Member
Oct 17, 2021
105
6
18
33
Hey :)

Hum ... Yours nodes are together in NAT mode ? If yes, it's normal that one VM cannot ping an another VM on the 2nd node.

In your case, NAT mode allow only that:
VMs node1 can ping NODE2, BUT NOT THESES VMs
VMs node2 can ping NODE1, BUT NOT THESES VMs too

Create another bridge on each node without given them an hardware connected interfaces cannot help you without a really complex firewalling/routage.

If you want to try easyly that you seems to try, try that:

First, powerOff all VM on each node.

On node 1, we 're gonna use your 1rst virtual network: 10.10.10.0/24
For my part, i prefer to call x.x.x.254 for my gateway, but make that you want.
node 1 vmbr1 IP: 10.10.10.254

On node 2, you need to modify the network name. Here, gonna take 10.20.20.0/24
node 2 vmbr1 IP: 10.20.20.254

On each node, you need a little bit FW rules:
node1

iptables -t filter -A forward -s 10.10.10.0/24 -j ACCEPT
iptables -t filter -A forward -d 10.10.10.0/24 -j ACCEPT
iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
route add -net 10.20.20.0/24 gw IP_NODE2-on-yourLannetwork

node2:
iptables -t filter -A forward -s 10.20.20.0/24 -j ACCEPT
iptables -t filter -A forward -d 10.20.20.0/24 -j ACCEPT

iptables -t nat -A POSTROUTING -s '10.20.20.0/24' -o vmbr0 -j MASQUERADE
route add -net 10.10.10.0/24 gw IP_NODE1-on-yourLannetwork

Up VMs and modify theirs network configuration. After it, it normally does to work ( but, it's not really clean ^^)
 

Lamia

Member
Jun 25, 2020
24
0
6
32
Hi,

I have changed my interfaces /etc/network/interfaces according to the response given above and it didn't solve the problem (I also deleted the VLAN I created since it was not useful).

My questions are:
1. Should I change the /etc/pve/nodes/<nodename>/host.fw file?
2. /etc/pve/nodes/<nodename>/host.fw doesn't exist. Should I create it manually?
3. Should I put the 4 lines in the firewall file or should I put those 2 lines related to the NAT and the route in the /etc/network/interfaces file?

Thanks,

Lamia
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!