Public and Private IPs

[zoltanb]

What is the best way to let the VMs on different nodes to communicate to each other either via their private IPs or via their public IPs?
How to achieve this with a 3 node cluster? All the 3 nodes have 1 public IP each + public IPs to be used on VMs.
Let's suppose I will have 5-10 VMs on each node using CEPH and HA and a public IP for each. I would like to setup private IPs to each VM and let the communicate together through the private IPs.
What is the best way to achieve this setup?

 

[oktay]

I think you'd need a switch, a virtual switch , a real cable between the machines or a vpn to be able to do this.

 

[zoltanb]

I got a vlan tag from my hosting provider, supposedly my 2 nodes are added to vlan tag 2001.

Server1: /etc/network/interfaces:
auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

auto vmbr0
iface vmbr0 inet static
address 100.101.102.159/24
address 192.168.0.1
gateway 100.101.102.1
bridge-ports eno1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2001

Server2: /etc/network/interfaces:
auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

auto vmbr0
iface vmbr0 inet static
address 100.101.102.121/24
address 192.168.0.2
gateway 100.101.102.1
bridge-ports eno1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2001

I can ping the nodes through their public IPs, but can't ping them through their private IPs (192.168.0.1 and 192.168.0.2). What am I doing wrong?