A problem with network forwarding after I finally got everything working

[aleksdj]

Hi there,

I have a Proxmox V7 configured in a barebone server with a public IP. To lower the costs, I'm trying to configure containers without the need to buy new public IPs. And by now, all seems working pretty well using Nginx Proxy Manager and the corresponding iptables forwarding rules. I can assign a ip in 10.0.0.x network to any container (using vmbr1), then configure it in NPM and right have outgoing and incoming traffic perfectly fine (even for ssh using NPM streams).

The problem has arose today when I needed to configure a new container having a public IP that I have bought. I have created the container using vmbr0 bridge, set the public IP (and the mac provided by ISP) and now the problem is that, as all traffic in vmbr0 with destination port 80:443 is routing to 10.0.0.100, every time I try to reach external web servers with port 80 (like in a `apt update`) it doesn't work.

So the question is: what is the best practice to, (having this setup that allow me to use the 1 public ip for n guests), be able to also create containers with their own public ip? My guesses are to either create e new vmbr interface (but I cannot get it working or I don't know how to do it correctly because eno1 port and host public IP is already in use by vmbr0 ), or making the DNAT rules only obey to real external traffic (and not traffic from guest hosts). Or even something with VLAN? I don't know...

This is my current host network configuration:

auto lo
iface lo inet loopback
iface eno1 inet manual
iface eno2 inet manual

auto vmbr0
iface vmbr0 inet static
    address x.x.100.125/24
    gateway x.x.100.1
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0
    hwaddress fc:15:b4:0f:5e:e8
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 80:443 -j DNAT --to-destination 10.0.0.100
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 80:443 -j DNAT --to-destination 10.0.0.100

auto vmbr1
iface vmbr1 inet static
    address 10.0.0.1/24
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up   iptables -t nat -A POSTROUTING -s '10.0.0.1/24' -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.0.0.1/24' -o vmbr0 -j MASQUERADE
    post-up   iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
    post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

This is how I was trying to create the new container (with public IP masked):


And this is my problem when trying to update aptitude (pings are working tho):

W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal/InRelease  Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1360:8001::24). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1360:8001::23). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (91.189.91.38), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.39), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.88.152), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.88.142), connection timed out

 

[aleksdj]

could somebody help me please?

 

[aleksdj]

I think I got it, I just have to set the destination IP of the "incoming traffic" DNAT to the public ip of the host.

post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp -d x.x.100.125 --dport 80:443 -j DNAT --to-destination 10.0.0.100

so that way, it uses the vmbr0 "public" bridge, but only for real external incoming traffic to host public ip (and not affecting other guests public ips using vmbr0 bridge).

Is this the best practice/solution?