I have two sites connected via a Unifi Site MagicVPN. Working great. I'd like to simplify the process of moving an LXC container back and forth between sites so I'm using a VxLAN overlay network and and placing the container on the overlay net. Looks like this:
I want to avoid routing all traffic out of a single site and instead route traffic for a container out the gateway for that site, hopefully improving reliability when the site-to-site VPN goes down. Instead of using a router VM on each Proxmox server I modified the /etc/network/interfaces.d/sdn configuration and assigned an overlay network IP address as well. Before the change:
And after the change:
I also createe a static router on the Unifi gateway to forward traffic, keeping NAT translation on the gateway. With this configuration, traffic originating from containers on Site A are routed out of Site A's internet interface, and the same for Site B. The only downside is having to modify the gateway setting for the container as I move it between sites.
Looking for feedback if there are better solutions available. I'd expect that if the GUI modifies the SDN configuration in the future I'll lose the current configuration, so there's some risk there as well.
Thanks.
Site A Site B ┌────────────────────┐ ┌────────────────────┐ │ UniFi Gateway (A) │ │ UniFi Gateway (B) │ │ LAN: 192.168.10.1 │ │ LAN: 192.168.20.1 │ │ WAN: (ISP-A) │ │ WAN: (ISP-B) │ │ Static Route: │ │ Static Route: │ │ 10.42.0.0/24 --> │ │ 10.42.0.0/24 --> │ │ 192.168.10.2 │◀───────────────VPN───────────▶│ 192.168.20.2 │ └─────────┬──────────┘ └─────────┬──────────┘ │ │ (LAN switch) (LAN switch) │ │ ┌──────┴───────┐ ┌─────┴───────┐ │ Proxmox (A) │ │ Proxmox (B) │ │ LAN IP: │ │ LAN IP: │ │ 192.168.10.2│ │ 192.168.20.2│ │ VXLAN br: │ │ VXLAN br: │ │ 10.42.0.1/24│◀─────────── overlay (VXLAN) ─────────│ 10.42.0.2/24│ └──────┬───────┘ └─────┬───────┘ │ │ LXC veth→vmbr-vxlan LXC veth→vmbr-vxlan (CT GW = 10.42.0.1) (CT GW = 10.42.0.2)Internet egress at A: CT(10.42) → GW 10.42.0.1 (Proxmox A) → 192.168.10.1 (UniFi A) → ISP-AInternet egress at B: CT(10.42) → GW 10.42.0.2 (Proxmox B) → 192.168.20.1 (UniFi B) → ISP-BI want to avoid routing all traffic out of a single site and instead route traffic for a container out the gateway for that site, hopefully improving reliability when the site-to-site VPN goes down. Instead of using a router VM on each Proxmox server I modified the /etc/network/interfaces.d/sdn configuration and assigned an overlay network IP address as well. Before the change:
#version:1auto corpiface corp bridge_ports vxlan_corp bridge_stp off bridge_fd 0 mtu 1342auto vxlan_corpiface vxlan_corp vxlan-id 42 vxlan_remoteip 192.168.20.2 mtu 1342And after the change:
#version:1auto corpiface corp inet static address 10.42.0.2/24 bridge_ports vxlan_corp bridge_stp off bridge_fd 0 mtu 1342auto vxlan_corpiface vxlan_corp vxlan-id 42 vxlan_remoteip 192.168.20.2 mtu 1342I also createe a static router on the Unifi gateway to forward traffic, keeping NAT translation on the gateway. With this configuration, traffic originating from containers on Site A are routed out of Site A's internet interface, and the same for Site B. The only downside is having to modify the gateway setting for the container as I move it between sites.
Looking for feedback if there are better solutions available. I'd expect that if the GUI modifies the SDN configuration in the future I'll lose the current configuration, so there's some risk there as well.
Thanks.