Proxmox 5.1 - containers / VMs cannot communicate with each other using the public IP

[Anjan]

Hello,

I have Dedicated Server on Hetzner with a single NIC and a single public IP on it. There are quite a few discussions about setting up Proxmox networking in such a situation. Specifically, I have followed the following guides:

• forum <dot> proxmox <dot> com/threads/proxmox-network-config-on-hetzner.36245/
  
• lowendtalk <dot> com/discussion/103656/hetzner-server-with-proxmox-and-1-ip-number
• raymii <dot> org/s/tutorials/Proxmox_VE_One_Public_IP.html

My /etc/network/interfaces file: pastebin <dot> com/XYiaujKj
My /etc/sysctl.conf: pastebin <dot> com/KN9drab7

To summarize:

• The NIC on the host has the public IP assigned on the interface named `eno1`.
• There is a separate bridge `vmbr2` which has a private IP series (192.168.22.x) assigned on it.
• All containers and VMs will be connected to `vmbr2` and have an IP in the same private IP series. The IP of vmbr2 will be the gateway for all the VMs and containers. Internet access works from all the VMs and containers.
• I have one VM on `192.168.22.5` which is setup as a reverse proxy. 3 ports from the Proxmox host (2222, 80, 443) will be forwarded to this VM. This has been setup using iptables pre-routing.
• On this reverse-proxy VM, using HAProxy running on 80 and 443, based on the hostname of the request, I forward the request to the appropriate VM / container's (private) IP.
• So, from the outside world, suppose I make a request to `abcd.example.com`, it gets routed correctly to the VM with IP `192.168.22.25`.

All of the above works as expected.

Now, to the problem I am facing: If I make the same request as above, to `abcd <dot> example <dot> com`, from one of the other VMs / containers, the request fails.

Say, I SSH in to `192.168.22.10` and run the below command:

$ curl abcd <dot> example <dot> com -v
    * Rebuilt URL to: abcd <dot> example <dot> com/
    *   Trying 145.250.76.40...
    * connect to 145.250.76.40 port 80 failed: Connection refused
    * Failed to connect to abcd <dot> example <dot> com port 80: Connection refused
    * Closing connection 0
    curl: (7) Failed to connect to abcd <dot> example <dot> com port 80: Connection refused

I have checked the reverse-proxy server and the intended target VM. Neither of them receive the request at all. For some reason, the host is not forwarding requests if it originates from the internal network.

Could somebody tell me what I am missing here?

 

[Bruno SINOU]

Hello,

Sorry to dig this thread out but I have exactly the same problem and lost my hair on it this afternoon...
Any hint anyone?

Thanks!

 

[Andrew Hart]

If you have a private network like 192.168.22.x and you want to talk to the outside world then you'll need a NAT.
A bridge is the equivalent of a network switch and doesn't do that.
You crazy cat.
(sorry it felt like a poem)

 

[Bruno SINOU]

Just for anyone hitting the same problem...

The answer comes with "Response Policy Zones": the idea is to overwrite public DNS with a local one (typically bind9 does the job) when you are trying to reach a machine that is in your local private VLAN using its public name.

This post explains it quite clearly: www*redpill-linpro*com/sysadvent/2015/12/08/dns-rpz*html (replace * by points, I cannot post link as a new user...)

Little quote from there:
> Overriding DNS could in itself be considered ugly. But there are some cases where it’s very useful:
> ...
> - accessing servers when you and the server are both on the inside of a NATed firewall/router

@Andrew Hart , thanks for the poetic answer but this does not solve the problem: it, in fact, comes from NAT...

 

[Andrew Hart]

Hi,
On a switch I think the correct technology is called "hairpinning" on a NAT it is called "NAT reflection"
Overriding DNS can work (Big companies do it too) but DNS over HTTPS might be trouble for the future.
Andrew