Proxmox HA cluster + external load balancer possible?

[leonidas_o]

Hello guys,
I would like to setup a Proxmox HA cluster and as for now, I see multiple approaches for making the VM's accessible when moved to a different node.
For example, having a 3 node Proxmox cluster. Each PVE node has its own public static IP address. The machines will be running inside a vLAN. Having shared nfs storage.
A mailserver VM is running on node1. One day, node1 should be updated and rebooted, therefore either automatically or manually the mailserver is moved to node2. The typical problem, mailserver is now under a different public IP address.

Current DNS settings: A records pointing to node1 (*, mail, www, etc.), CNAME records (autoconfig and autodiscover) + some other records ( don't think they are somehow relevant for this example).

So this will not work that way.
Now I see a few possible approaches. Also if there are some other, better approaches, please let me know.

- Approach1: Hetzner for example, has a vSwitch and the possibility to order a subnet, which has a pretty high setup fee + a monthly fee and somehow an outgoing Limit of 1TB. So it doesn't sound that interesting.

- Approach2: Hetzner is providing a simple round robin (cloud) load balancer, which can be setup to load balance between the dedicated servers. Pricing seems better than approach1, a traffic limit of 20TB for the smallest load balancer. But wouldn't work because only round robin and least connections available. So if the mailserver moved to node2, the load balancer has to be aware of that and only send the requests to the available VM.

- Approach3: A cloud flare load balancer. Smallest package cheaper than Hetzners solution, has no public static IP address. How I understand it, you have to update your CNAME records to point to cloudflares load balancer. And here I don't have a lot of experience with DNS Settings, to evaluate how or even if this could work. I guess I would have to remove the above mentioned A and CNAME records, set a CNAME record to redirect everything to the load balancer and inside the load balancer, setup the records again like mentioned above. At least I hope this is possible that way.

And the overall question is, can the load balancer approach even be used, instead of approach1 (subnet)? I would say approach3, the cloud flare load balancer seems to be the best approach especially as it seems to provide the functionality to detect an unresponsive server and not redirecting to it. Means, you would set it up like the mailserver is running on all of the three nodes. The load balancer on the other hand would always see only one available instance and always redirect to that one.
Has anyone experience with cloud flare or done something like that?

 

[LnxBil]

Due to this limitations, we are not running HA in hetzner and went to a housing provider that enables us to have a "real" subnet that is shared among all nodes so that you can have VMs that have also public IPs and are reachable from every host.

 

[alexskysilk]

I havent used Hetzner, but it seems like they do offer multiple vlan support
.
https://community.hetzner.com/tutorials/hyperconverged-proxmox-cloud

Once you have an internal lan, all your vm's attach to that subnet and you use a router vm (eg pfsense) to route traffic.

 

[leonidas_o]

yes, hetzner does support that, as I said in approach1, but the picture in your link @alexskysilk must be old or they have different prices according the server package size, amount etc. one is having. If I go into that menu for the subnets, the monthly fee actually doubled and the setup fee is starting at 180 Euro and goes up to 724 Euro. Which absolutely does not make any sense for me, as I basically need it just for a little mail server. That's why I was asking if that could be achieved with a load balancer (approach3).

 

[alexskysilk]

as I basically need it just for a little mail server.

Ah thats the part I guess I missed.

so really, none of those are really serving your needs, at least not for incoming mail. For outgoing, any number of vps servers running smtp with cloudflare in front would do the trick as long as you set up your dkim/vsf/dmarc correctly- so yes, option 3 would serve.

for incoming, the whole point of HA is that you have multiple logical servers, so you really wouldnt be benefitting from proxmox VE here as it will allow you to have a single logical server via multiple hypervisors. I hear proxmox has a mail server tho might want to post your query on the appropriate board for that.

 

[leonidas_o]

I mean, there are few other VM's on the nodes as well, but it's not that critical and wouldn't make sense to pay such setup fees. I just think the mail server is the most complex part to have a load balancer for, compared to a website, git etc. Also I would like to avoid being locked in by a provider with lots of customisations. Maybe you are not satisfied with the provider, want to switch to another and then the struggle begins. I had that with Scaleway so I'll be moving to Hetzner I guess.

No, it's just a Proxmox Mail Gateway, not a whole server. I'll post in the cloud flare forum, maybe someone have used a load balancer in front of a mailserver which can be either on node1, node2 or node3.

 

[LnxBil]

We had our own outwards-facing mailserver for years but it got harder and harder to have correctly delivered mails so we changed to a kind of proxy mail server from a huge provider and just load all mail from them into our own "local" mailserver via fetchmai every minute and send all via smart host to that ISP provided mailserver. With this, we have only the mail part we care about, hosted in-house and not directly reachable from the internet. The ISP is responsible for graylisting etc. Maybe that is also a way to go for you?

 

[leonidas_o]

I don't have any issues with the sending, receiving, filtering (spam) mails. So the tooling and its configuration actually is working very well for me. It's just the optimisation to a high availability system. Anyway, thanks for the ideas. I think I have to simply try it, order cloudflares load balancer and just test it to see if it is possible. In Cloudflares forum, unfortunately nobody answered my post.