VM's can't reach VM's on same host using external IP host

[Schoonmoeder]

Hello,

Situation. 4 hosts in 1 cluster.
VMs on host 1 can talk to any VMs on other hosts by using the external IP of the assigned host (for example: VM1 on host 1 can connect to VM2 on host 2 using host 2 external IP)
VMs on host 1 can talk to VMs on host 1 using the Linux bridge IP. (10.0.0.x/24)
VMs on host 1 can NOT talk to VMs on host 1 using the external IP of host 1.

Could someone give me pointers on why VMs can use the external IP of their host to get to VMs on the same machine?

cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno8303 inet manual
        post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o vmbr0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s 10.0.0.0/24 -o vmbr0 -j MASQUERADE
        post-up   iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
        post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

iface eno8403 inet manual

auto vmbr0
iface vmbr0 inet static
        address 100.101.102.114/27
        gateway 100.101.102.126
        bridge-ports eno8303
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
        address 10.0.0.1/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0

auto vmbr2
iface vmbr2 inet static
        address 10.0.40.0/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0

auto vmbr3
iface vmbr3 inet static
        address 10.0.100.1/24
        bridge-ports eno8403
        bridge-stp off
        bridge-fd 0

source /etc/network/interfaces.d/*

 

[shanreich]

(for example: VM1 on host 1 can connect to VM2 on host 2 using host 2 external IP)

Do you mean directly by using the IP of host2 - or by using host2 as a gateway? In the first case, do you have any DNAT rules in place for those hosts that aren't available on host1?

 

[Schoonmoeder]

Our issue is that on 1 host we have 4 VMs
these 4 VMs can talk using the internal 10.0.0.x/24
ISSUE: these 4 VMs can't talk using the external IP from their host

situation (all on 1 host)
VM 1: listens to port 13001
VM 2: listens to port 13002
VM 3: listens to port 13003
VM 4: listens to port 13004

Working: VM 1 connecting to 10.0.0.12:13002 (VM 1 curl VM 2 over internal IP and port)
Working: VM 2 connecting to 10.0.0.11:13001 (VM 2 curl VM 1 over internal IP and port)
Working: Any other machine connecting to VM 2 by using External host IP. ( curl 100.101.102.114:13002)
Not Working: VM 1 connecting to VM 2 by External IP. (curl 100.101.102.114:13002)