Dedicated Server: Single NIC. Use 2nd IP for VM traffic

gregg098

Well-Known Member
Apr 5, 2018
39
5
48
43
Update: Working now. Working configuration shown at the bottom.

Looking for some assistance to see what Im doing wrong in this setup.

Dedicated server with ColoCrossing with 2 public IPs. Single network interface enp3s0.

The following setup works just fine. VMs are setup with an IP of 10.0.0x and gateway of 10.0.0.1. Traffic originates from <IP1>
Code:
auto lo
iface lo inet loopback

iface eno1 inet manual

auto enp3s0
iface enp3s0 inet static
    address <IP1>
    gateway <GW>

auto vmbr0
iface vmbr0 inet manual
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    address  10.0.0.1/24
    post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up   iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o enp3s0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o enp3s0 -j MASQUERADE

I would like to make it so all traffic on vmbr0 uses the 2nd IP address. Essentially making <IP1> for Proxmox management only. <IP2> to be the source of all VM traffic.

From research on forums and documentation, it looks like I should change my NAT rules to something like this:
Code:
post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o enp3s0 -j SNAT --to-source <IP2>
post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o enp3s0 -j SNAT --to-source <IP2>

But when I do this, I dont get internet in any of the VMs. VMs are still set to using IPs from the range 10.0.0.0/24, with gateway 10.0.0.1.

What am I doing wrong?

As an additional test, I setup an OPNsense VM tied to the bridge with the first NAT setup. I created a virtual IP alias of <IP2> and an outgoing NAT rule to use <IP2>, and it works just fine. So the second IP is routed properly. I just cant get it to work in a Proxmox only environment (no OPNsense as a middle man for the VMs). I would prefer not to manage the OPNsense VM as well just for this.

Thank you.

UPDATE1:

I did get it working finally after messing around for awhile. Below is my working config. I think what really made it work was adding enp3s0's mac address to vmbr0 due to it being a hosting providers network.
If anyone sees anything wrong with this configuration, please let me know.

Code:
auto enp3s0
iface enp3s0 inet static
    address <IP1>
    gateway <GW>
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward

auto vmbr0
iface vmbr0 inet manual
    hwaddress <mac address of enp3s0>
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    address  10.0.0.1/24
    up ip route add <IP2> dev vmbr0
    post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up   iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o enp3s0 -j SNAT --to-source <IP2>
    post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o enp3s0 -j SNAT --to-source <IP2>

    post-up   iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
    post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!