Create additional Masquerading (NAT) bridge

[alex1452]

Dear all,

I have a PC with 4 physical ethernet ports behind a router that I connected with a bridge, so that I can connect additional ethernet equipment. For proxmox, I would like to create an additional "internal" brigde for my VMs (KVM and LXC) with private networking that are then routed. I do not want to router to see these additional virtual machines. According to the manual I have tried the following:

auto lo
iface lo inet loopback
iface enp1s0 inet manual
iface enp2s0 inet manual
iface enp3s0 inet manual
iface enp4s0 inet manual

auto br0
iface br0 inet static
    address 192.168.178.71/24
    gateway 192.168.178.1
    bridge-ports enp1s0 enp2s0 enp3s0 enp4s0
    bridge-stp off
    bridge-fd 0

iface br0 inet6 auto
    accept_ra 2

auto vmbr0
#private sub network
iface vmbr0 inet static
        address  10.10.10.1/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0

        post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o br0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o br0 -j MASQUERADE

source /etc/network/interfaces.d/*

Unfortunately this does not work! When I connect LXC-containers to br0, I have internet access. However, when I try to connect them to vmbr0 (using dhcp or static) I have no internet access. What am I doing wrong?

• What is the correct setup for the additional bridge?
• What do I need to do to have DHCP on this bridge?

Any pointers on how I can debug this? Ideally my VMs would get an IP via DHCP like 10.10.10.100 and the traffic gets routed to my dsl router at 192.168.178.1. Can I provide additional information to help debugging?

 

[alimmi]

I wanted to know exactly the same and this setup just started working for me:
Maybe the part making problems is the multiple interfaces in the br0. I tried something similar before and that didn't work in combination with NAT from a second bridge, but I can't tell exactly what was the problem, after hours of trial and error.

auto lo
iface lo inet loopback

# Main Interface
auto enp0s25
iface enp0s25 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.178.5/24
        gateway 192.168.178.1
        bridge-ports enp0s25
        bridge-stp off
        bridge-fd 0
        post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr0/proxy_arp
#Bridged local network

auto vmbr1
iface vmbr1 inet static
    address 172.16.0.1/24
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up   iptables -t nat -A POSTROUTING -s '172.16.0.0/24' -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '172.16.0.0/24' -o vmbr0 -j MASQUERADE
#Internal net behind NAT

source /etc/network/interfaces.d/*

When I ping my Laptop (Connected to local network) from a VM with IP 172.16.0.10 on vmbr1 and monitor ICMP messages on the laptop, you can see they are masqueraded with the IP 192.168.178.5, the main interface of the Proxmox host:

$ sudo tcpdump -i en0 icmp and icmp[icmptype]=icmp-echo
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en0, link-type EN10MB (Ethernet), capture size 262144 bytes
01:54:40.525976 IP 192.168.178.5 > vm.fritz.box: ICMP echo request, id 2, seq 1, length 64
01:54:41.568512 IP 192.168.178.5 > vm.fritz.box: ICMP echo request, id 2, seq 2, length 64
01:54:42.592570 IP 192.168.178.5 > vm.fritz.box: ICMP echo request, id 2, seq 3, length 64

Be aware that for this setup alone, machines on the vmbr1 subnet need their IP set up manually:

If you want DHCP, you could set up dnsmasq, which is already installed on Proxmox natively.

 

[alimmi]

Regarding the DHCP, that was very straight forward to set up (For a simple case like mine)

Install dnsmasq

apt update
apt install dnsmasq

Edit at least the following lines in /etc/dnsmasq.conf
(Edit the corresponding IP's and ranges to your needs.)

interface=vmbr1
dhcp-range=172.16.0.10,172.16.0.254,48h

The "48h" is the lease time, 48 hours.

Then

sudo service dnsmasq restart