Accessing web url of container from within container

[Soban Khalid]

Hi,

I have setup containers with NAT with the following network configuration:

auto vmbr0
iface vmbr0 inet static
address 51.254.45.138
netmask 255.255.255.0
network 51.254.45.0
broadcast 51.254.45.255
gateway 51.254.45.254
bridge_ports eth0
bridge_stp off
bridge_fd 0

auto vmbr2
#private sub network for the containers
iface vmbr2 inet static
address 10.1.1.254
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_fd 0

post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s '10.1.1.0/24' -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.1.1.0/24' -o vmbr0 -j MASQUERADE

For each container, I add bridge vmbr2 and assign an IP address from 10.1.1./24. For internet access, the container uses NAT/MASQUERADE and goes through vmbr0, which is the public facing interface.

To redirect requests from outside to the containers, I use iptables rules like:

iptables -t nat -A PREROUTING -i vmbr0 -p tcp -d 51.254.45.138 --dport 44324 -j DNAT --to 10.1.1.24:443

This is redirecting a request on port 44324 on vmbr0 to port 443 of a container on its internal IP 10.1.1.24

When i send the request from outside to the URL to access a website on the container (PUBLIC-IP:44324) the request goes through fine and reaches 10.1.1.24:443 and I can access the webpage. But when I initiate the same request from within the container to the same URL, PUBLIC-IP:44324, it times out.

I have tried testing with telnet and nc but they end up on connection timeout as well.

# nc 51.254.45.138 44324 -vvvv
Ncat: Version 6.40
libnsock nsi_new2(): nsi_new (IOD #1)
libnsock msevent_new(): msevent_new (IOD #1) (EID #8)
libnsock nsock_connect_tcp(): TCP connection requested to 51.254.45.138:44324 (IOD #1) EID 8
libnsock nsp_add_event(): NSE #8: Adding event
libnsock nsock_loop(): nsock_loop() started (no timeout). 1 events pending
libnsock nsock_trace_handler_callback(): Callback: CONNECT TIMEOUT for EID 8 [51.254.45.138:44324]
Ncat: Connection timed out.

Can anyone help me figure out what could be the issue here?

 

[Richard]

But when I initiate the same request from within the container to the same URL, PUBLIC-IP:44324, it times out.

The request packet will be sent like this:

Source:10.1.1.24 Dest:51.254.45.138:44324 from Container to Host
after DNAT
Source:10.1.1.24 Dest:10.1.1.24:443 from Host to Container


For the response in the container it would be routed to itself (10.1.1.24 to 10.1.1.24) via lo, but the request came from eth0 (or any other NIC), which would be an "asymmetric routing" - this is avoided by default (regardless of having a container, VM or bare metal machine) in Linux. Can be changed by disabling rp_filter, but not recommended. In principle such a connection is useless (why not connecting to "localhost" directly?).

 

[Soban Khalid]

Hi Richard,

Thank you for the response.

There is a backend application running on the container on 443. And there is another frontend application that needs to connect to this backend application with its URL 51.254.45.138:44324 . If I give the URL like localhost:443, then every hit from the browser goes to localhost, which fails obviously.