[SOLVED] How to configure 2 default gateways for 2 public IPs

[cmonty14]

Hi!

PVE 4.x is running on server with 2 NICs.
eth0 is connected directly to ISP providing public IPv4.
eth1 is connected to Fritz!Box home network.

The attached graphic could help to understand the architecture though:

For any NIC eth0 and eth1 a Linux-Bridge is defined: vmbr0 and vmbr1.

Question:
How can I ensure that all clients (KVM and containers) that are connected to vmbr0 are routed to gateway from ISP B, but all clients that are connected to vmbr1 are routed to gateway from Fritz!Box?

I have found this article when searching for "two default gateways" explaining the network configuration with prerequisite of package "iproute2".

Executing the following commands will create the demanded second gateway:

ip route add 192.168.178.0/24 dev vmbr1 src 192.168.178.10 table rt2
ip route add default via 192.168.178.1 dev vmbr1 table rt2
ip rule add from 192.168.178.10/32 table rt2
ip rule add to 192.168.178.10/32 table rt2

This is resulting routing table:

route
Kernel-IP-Routentabelle
Ziel            Router          Genmask         Flags Metric Ref    Use Iface
default         HSI-KBW-217-008 0.0.0.0         UG    0      0        0 eth0
10.0.0.0        *               255.255.255.0   U     0      0        0 vmbr0
192.168.0.0     *               255.255.255.0   U     0      0        0 vmbr1
217.8.50.64     *               255.255.255.192 U     0      0        0 eth0

To ensure that the rules as well as the routing entries are working as expected:

ip rule show
0:      from all lookup local
32764:  from all to 192.168.178.10 lookup rt2
32765:  from 192.168.178.10 lookup rt2
32766:  from all lookup main
32767:  from all lookup default
ip route list table rt2
default via 192.168.178.1 dev vmbr1
192.168.178.0/24 dev vmbr1  scope link  src 192.168.178.10

Making the configuration permanent I have modified the networking configuration /etc/network/interfaces accordingly.

# The loopback network interface
auto lo
iface lo inet loopback

## unitymedia business static IPv4
auto eth0
iface eth0 inet dhcp
        post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp

auto eth1
iface eth1 inet manual

## LAN by pc4-gigabyte (Proxmox VE)
auto vmbr0
iface vmbr0 inet static
        address  10.0.0.1
        netmask  255.255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fd 0
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth0 -j MASQUERADE

## LAN net1-fb6490 mit zweiter Routingtabelle
auto vmbr1
iface vmbr1 inet static
        address  192.168.178.10
        netmask  255.255.0.0
        gateway  192.168.178.1
        bridge_ports eth1
        bridge_stp off
        bridge_fd 0
        post-up ip route add 192.168.178.0/24 dev vmbr1 src 192.168.178.10 table rt2
        post-up ip route add default via 192.168.178.1 dev vmbr1 table rt2
        post-up ip rule add from 192.168.178.10/32 table rt2
        post-up ip rule add to 192.168.178.10/32 table rt2

THX

 

[wolfgang]

Hi,
You have to add for each network a route to the uplink(Gateway).

add some think like this to your network conf

up route add -net 192.168.1.0 netmask 255.255.255.0 gw <IP of GW>
down route delete -net 192.168.1.0 netmask 255.255.255.0 gw <IP of GW>

 

[cmonty14]

Hi,
You have to add for each network a route to the uplink(Gateway).

add some think like this to your network conf

up route add -net 192.168.1.0 netmask 255.255.255.0 gw <IP of GW>
down route delete -net 192.168.1.0 netmask 255.255.255.0 gw <IP of GW>

Hello Wolfgang,

thanks for your reply.

I was doing some investigation and found at least 2 alternative options that sounds interesting:
- 2 default gateways in 1 system using iproute2
- Asymmetric routing, multiple default gateways on Linux with haproxy

Option 1 looks like a straight forward approach using standard tools.
Option 2 offers some enhanced functionalities that I have considered anyway (e.g. reverse proxy).

Apart of the additional effort for configuration, do you see any downside of the 2 options, e.g. in regards to network security?
Would you recommend to refrain from installing additional packages like haproxy or isc-dhcp-server on Proxmox VE server?

THX