[SOLVED] SDN with 2+ vnet subnet specify SNAT to WAN IP

[w2vy]

I have 5 IP addresses from my ISP and need to setup a VNET for each IP with a pfSense firewall to provide UPNP support for each IP.

My test machine has a an internal Ethernet port and a USB Dongle. I am using the dongle (enx22) for the WAN

The network looks like this:

enx22 <--> vmbr1 <--> VNET184 <--> pfSense VM <--> vmbr185 <--> clients
enx22 <--> vmbr2 <--> VNET184 <--> pfSense VM <--> vmbr185 <--> clients

The error I am running into is that you can't have more than one bridge port enslaved to the same interface (enx22)

As I found in other posts, I created the bridges and SDN zone(1) and VNETs (2) with IPs and added SNAT rules manually

If I only map one (or the other) bridge port to enx22 each vnet works fine, suggesting that I need a physical port for each WAN IP, which I would prefer not to do.

Any ideas how to do this with a single interface? (yes the WAN IP's are from a single subnet)

My current configuration is below.

For anyone wanting to replicate this the only config that I manually entered is the post-up/post-down rules and those lines were taken from checking the SNAT box in the VNET when I was testing a single VNET (which works fine BTW)

Thanks!
Tom

/etc/network/interfaces

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual

auto enx22
iface enx22 inet manual

auto vmbr0
iface vmbr0 inet static
    address 192.168.8.40/24
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0

auto vmbr1
iface vmbr1 inet static
    address 1.2.34.184/24
    gateway 1.2.34.1
    bridge-ports enx22 # Comment out to use 185
    bridge-stp off
    bridge-fd 0

auto vmbr2
iface vmbr2 inet static
    address 1.2.34.185/24
    gateway 1.2.34.1
#    bridge-ports enx22 # Comment out to use 184
    bridge-stp off
    bridge-fd 0

auto vmbr185
iface vmbr185 inet manual
    bridge-ports none
    bridge-stp off
    bridge-fd 0

auto vmbr184
iface vmbr184 inet manual
    bridge-ports none
    bridge-stp off
    bridge-fd 0

iface vnet184
    post-up iptables -t nat -A POSTROUTING -s '10.10.184.0/24' -o vmbr1 -j SNAT --to-source 1.2.34.184
    post-down iptables -t nat -D POSTROUTING -s '10.10.184.0/24' -o vmbr1 -j SNAT --to-source 1.2.34.184
    post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
    post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

iface vnet185
    post-up iptables -t nat -A POSTROUTING -s '10.10.185.0/24' -o vmbr2 -j SNAT --to-source 1.2.34.185
    post-down iptables -t nat -D POSTROUTING -s '10.10.185.0/24' -o vmbr2 -j SNAT --to-source 1.2.34.185
    post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
    post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

source /etc/network/interfaces.d/*

/etc/network/interfaces.d/sdn

#version:47

auto vnet184
iface vnet184
    address 10.10.184.1/24
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    ip-forward on

auto vnet185
iface vnet185
    address 10.10.185.1/24
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    ip-forward on

 

[w2vy]

Sometimes the simplest solutions are the best!

I replaced my external switch with a WAN Bridge (vbmr1) with 2 physical ports (pass through for other existing equipment) and then added OPNsense VMs for each unused WAN IP with the WAN interface attached to the WAN Bridge

I then assigned the WAN IP to each VM and I was on my way!

I created vbmr185 for the LAN side that used that WAN IP and attached all the '185 clients, same for other WAN IPs

No advanced NAT (SNAT/DNET) routing etc... really simple

Thanks
Tom