Multiple Public IPs with NAT configuration

Bueddl · Oct 12, 2014

I am currently trying to get proxmox to work with multiple public IPs with NAT. I have multiple VMs and the basic NAT setup works.
So I can use them all through the "proxmox" IP used for vmbr0. This is my current configuration:

Code:

VM100 has the internal IP 10.0.2.100 and is bridged to vmbr3.
VM101 has the internal IP 10.0.2.101 and is bridged to vmbr3.
etc...

My NAT configuration currently only forwards 212.xx.xx.88:80 to one vm and forwards 40022, 40122, 40222 to the ssh ports of the VMs.

The network config /etc/network/interfaces of the proxmox host:

Code:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static

auto vmbr0
iface vmbr0 inet static
    address  212.xx.xx.88
    netmask  255.255.255.0
    gateway  212.x.xx.1
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

auto vmbr3
iface vmbr3 inet static
    address  10.0.2.254
    netmask  255.255.255.0
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward

    post-up      iptables -t nat -A POSTROUTING -s '10.0.2.0/24' -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.0.2.0/24' -o vmbr0 -j MASQUERADE

    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 40022 -j DNAT --to 10.0.2.100:22
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 40022 -j DNAT --to 10.0.2.100:22
    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 40122 -j DNAT --to 10.0.2.101:22
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 40122 -j DNAT --to 10.0.2.101:22
    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 40222 -j DNAT --to 10.0.2.102:22
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 40222 -j DNAT --to 10.0.2.102:22

    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 80    -j DNAT --to 10.0.2.101:80
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 80    -j DNAT --to 10.0.2.101:80

This configuration currently works as expected. I am now trying to add a secondary Public IP and use that IP for some of the VMs, but not for all.
I was trying to simple do this to "route" VM100 through the secondary IP:

Code:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static

auto vmbr0
iface vmbr0 inet static
    address  212.xx.xx.88
    netmask  255.255.255.0
    gateway  212.x.xx.1
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

auto vmbr1
iface vmbr1 inet static
    address  212.xx.xx.98
    netmask  255.255.255.0
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

auto vmbr3
iface vmbr3 inet static
    address  10.0.2.254
    netmask  255.255.255.0
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward

    # VM100
    post-up      iptables -t nat -A POSTROUTING -s '10.0.2.100' -o vmbr1 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.0.2.100' -o vmbr1 -j MASQUERADE
    post-up   iptables -t nat -A PREROUTING -i vmbr1 -p tcp --dport 40022 -j DNAT --to 10.0.2.100:22
    post-down iptables -t nat -D PREROUTING -i vmbr1 -p tcp --dport 40022 -j DNAT --to 10.0.2.100:22

    # others
    post-up      iptables -t nat -A POSTROUTING -s '10.0.2.0/24' -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.0.2.0/24' -o vmbr0 -j MASQUERADE

    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 40122 -j DNAT --to 10.0.2.101:22
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 40122 -j DNAT --to 10.0.2.101:22
    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 40222 -j DNAT --to 10.0.2.102:22
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 40222 -j DNAT --to 10.0.2.102:22

    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 80    -j DNAT --to 10.0.2.101:80
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 80    -j DNAT --to 10.0.2.101:80

But this does not work at all. The VM100 is not reachable anymore - the other VMs are working.

How do I get this to work as expected?

Thanks,
Sebastian

Bueddl · Oct 12, 2014

May this be related to vlans?

Bueddl · Oct 14, 2014

Any Idea on this?

kotakomputer · Oct 30, 2014

You have setup vmbr0 (external IP) paired to vmbr3 (internal IP) and works fine.
What you need to do is setup vmbr1 (external IP) paired to vmbr4 (internal IP).

I suggest you give ordered vmbrX name, so will easy to remember, such as:
vmbr0 (external IP) paired to vmbr1 (internal IP) -- NAT1
vmbr2 (external IP) paired to vmbr3 (internal IP) -- NAT2
vmbr4 (external IP) paired to vmbr5 (internal IP) --NAT3

And give each internal IP with different SUBNET, i.e 10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24

Bueddl · Oct 30, 2014

Many thanks for your reply, I am still working on this. Going to giving your attempt tomorrow a try!

Thanks,
Sebastian

Bueddl · Nov 1, 2014

I configured it according to your description, but I won't get it to work.
As of the three external IPs are of the same subnet, I am not able to assign the gateway to all three "external" bridges - vmbr0, vmbr2 and vmbr4.

Any ideas?

kotakomputer · Nov 2, 2014

How do you add additional IPs to eth0? Do you use more than 1 eth?
You may copy again /etc/network/interfaces here.

Bueddl · Nov 2, 2014

This is what I got so far. Thought I did what you suggested. Or were I misunderstanding you?
For now, I only have extenral services running on the 10.0.2.0/24 subnet, but this will definitl change as soon as I got this working.

Code:

# network interface settings
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static

iface eth1 inet manual

# vmbr0 -> vmbr1
# xxx.xxx.26.88 -> 10.0.1.0/24
auto vmbr0
iface vmbr0 inet static
    address  xxx.xxx.26.88
    netmask  255.255.255.0
    gateway  xxx.xxx.26.1
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

auto vmbr1
iface vmbr1 inet static
    address  10.0.1.254
    netmask  255.255.255.0
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up   iptables -t nat -A POSTROUTING -s '10.0.1.0/24' -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.0.1.0/24' -o vmbr0 -j MASQUERADE
    # no services defined here (for now)
 
# vmbr2 -> vmbr3
# xxx.xxx.26.98 -> 10.0.2.0/24
auto vmbr2
iface vmbr2 inet static
    address  xxx.xxx.26.98
    netmask  255.255.255.255
    bridge_ports none
    bridge_stp off
    bridge_fd 0

auto vmbr3
iface vmbr3 inet static
    address  10.0.2.254
    netmask  255.255.255.0
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up   iptables -t nat -A POSTROUTING -s '10.0.2.0/24' -o vmbr2 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.0.2.0/24' -o vmbr2 -j MASQUERADE
    post-up   iptables -t nat -A PREROUTING -i vmbr2 -p tcp --dport 40022 -j DNAT --to 10.0.2.100:22
    post-down iptables -t nat -D PREROUTING -i vmbr2 -p tcp --dport 40022 -j DNAT --to 10.0.2.100:22
    post-up   iptables -t nat -A PREROUTING -i vmbr2 -p tcp --dport 40122 -j DNAT --to 10.0.2.101:22
    post-down iptables -t nat -D PREROUTING -i vmbr2 -p tcp --dport 40122 -j DNAT --to 10.0.2.101:22
    post-up   iptables -t nat -A PREROUTING -i vmbr2 -p tcp --dport 40222 -j DNAT --to 10.0.2.102:22
    post-down iptables -t nat -D PREROUTING -i vmbr2 -p tcp --dport 40222 -j DNAT --to 10.0.2.102:22
    post-up   iptables -t nat -A PREROUTING -i vmbr2 -p tcp --dport 80    -j DNAT --to 10.0.2.101:80
    post-down iptables -t nat -D PREROUTING -i vmbr2 -p tcp --dport 80    -j DNAT --to 10.0.2.101:80
    post-up   iptables -t nat -A PREROUTING -i vmbr2 -p tcp --dport 20    -j DNAT --to 10.0.2.101:20
    post-up   iptables -t nat -A PREROUTING -i vmbr2 -p tcp --dport 21    -j DNAT --to 10.0.2.101:21
    post-down iptables -t nat -D PREROUTING -i vmbr2 -p tcp --dport 20    -j DNAT --to 10.0.2.101:20
    post-down iptables -t nat -D PREROUTING -i vmbr2 -p tcp --dport 21    -j DNAT --to 10.0.2.101:21

# vmbr4 -> vmbr5
# xxx.xxx.26.99 -> 10.0.3.0/24
auto vmbr4
iface vmbr4 inet static
    address  xxx.xxx.26.99
    netmask  255.255.255.255
    bridge_ports none
    bridge_stp off
    bridge_fd 0

auto vmbr5
iface vmbr5 inet static
    address  10.0.3.254
    netmask  255.255.255.0
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up   iptables -t nat -A POSTROUTING -s '10.0.3.0/24' -o vmbr4 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.0.3.0/24' -o vmbr4 -j MASQUERADE
    # no services defined here (for now)

Thanks again!

Sebastian

Search

Search

Multiple Public IPs with NAT configuration

Bueddl

New Member

Bueddl

New Member

Bueddl

New Member

kotakomputer

Renowned Member

Bueddl

New Member

Bueddl

New Member

kotakomputer

Renowned Member

Bueddl

New Member