Proxmox cluster with dhcp from SDN possible to have different default gateway on VM's?

[sverker]

I have a proxmox cluster with two nodes. Each has a physical ethernet interface on public internet and on the same physical interface a VLAN tagged interface (tag=4000) for internal network. Each of the hosts have one IP address on the internal network and have masquerading set up. The VM's currently receives their ip addresses from an ISC dhcpd running on one of the hosts.

I understand that with SDN in Proxmox I can get dhcp functionality from there, but there is a twist. I want that the VM gets the ip of the host they are running on as default gateway so that the traffic can go directly to internet without having to be sent over the internal network to the other host. Is this possible to achieve?

I am able to get the desired functionality by adding each host as static entries in dhcpd.conf but I would like it to be handled automatically so that e.g. if a VM is migrated to the other host it would automatically get the correct default gateway.

 

[spirit]

I think you should look at evpn zone with anycast gateway (same ip on each vnet on each host).


They are no way to change gateway ip inside the vm when your are migrating it. (because it's transparent for the guest os).


(Note that dhcp is only implemented on single zone currently, other zones support should be available soon)

 

[sverker]

Ok, I had already started looking at evpn zone but the doc is not very clear and there are no examples.

 

[sverker]

I have set up an evpn zone but it doesn't seem to do what I want. For starters, as you wrote it doesn't support dhcp so my usecase is toast already there. Second, if I configure static IP addresses on vm's on both hosts I can reach the gateway ip which is the same on each host but I can't reach the hosts directly nor can I reach VM's on the other host. It's not clear if any ports needs to be open between the Proxmox hosts for evpn to work.

I found another very simple solution though. This is the current vlan interface on the Proxmox hosts (one of them, the other has address 10.208.1.2:

auto vmbr0.4000
iface vmbr0.4000 inet static
        address 10.208.1.1/24
        post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.208.1.0/24' ! -d '10.208.1.0/24' -o vmbr0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.208.1.0/24' ! -d '10.208.1.0/24' -o vmbr0 -j MASQUERADE
        post-up   iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
        post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

I then added this on both Proxmox hosts:

# Anycast IP
auto lo:1
iface lo:1 inet static
        address 10.208.1.254/32

In dhcpd.conf I then set option routers 10.208.1.254;

So far everything works as expected, outbound traffic is routed via the Proxmox host the vm is running on, all vm's can reach each other and all hosts. Only thing I need to correct is the routing to my vpn which terminates on one of the hosts and set up failover for dhcpd