Suggestions for Integrating Proxmox into an Out of Band Management Network

[Ela@Blach]

Hi,

I am looking to replace some ESXI hypervisors with Proxmox in light of the Broadcom acquisition, and I have to figure out how to make this work: I need to be able to access Proxmox from two separate networks, and both need to be able to say "Hey this traffic came in on this interface, send it back there where it came from". My first thought was to create a vrf, virtual bridge in this vrf with the OOB network port. Doing this I was able to ping the hypervisor from both networks successfully, however the two things I really need to work (ssh and the PVE UI) were not.... listening, bound.... to the network in the vrf. After I thought about it, This made sense and I tried to come up with a different solution.

I also thought about trying to mark a (session?) when it is established on a certain interface, so I could then use that mark to guide that traffic back out the correct interface, but I am not sure how to do this in Linux.

I also thought about creating a separate routing table for the out of band, but am not sure how to go about that. I would really like to figure something out for this, and I think I need a nudge in the right direction to get me started.

This OOB network does not need to be particularly performant, just needs to be A: Fairly simple to set up on many hypervisors, and B: there in case we need to access one of our hypervisors and the primary vpn is down.

(In ESXI I found a simple workaround for this, Use a DHCP client on a port group attached to a VMK, this allows access on both interfaces through our VPN. I am not a Linux networking wiz, and this has been troubling me for a few weeks now.

Any Help is greatly appreciated <3

Below is a simple diagram of what I am trying to achieve from a resultant functionality level:

 

[Ela@Blach]

I have continued to bash my head against this issue, and have come up with something that works. Just trying to figure out the best way to persist this across reboots, updates, network commits in the GUI, etc.

Create a routing table:
Add: [table#] [TableName] to /etc/iproute2/rt_tables

#I think this may actually not be needed, and might even break it.
Create a route to your oob network in the main routing table: ip route add [gateway of oob network] scope link dev [bridge interface for oob]

Create a routing rule so proxmox knows when to use this new table: ip rule add from [ip address of bridge interface for oob] table [TableName]
Create the default route for this OOB routing table: ip route add default via [gateway of oob network] dev [bridge interface for oob] table [TableName]

Hope this helps someone in the future, but if anyone knows a good way to make this persist across reboots and updates, I would love to know!