HA usefulness without network-storage ?

[toxic]

Hello,
I use pve to virtualize my opnSense router, I'm about to install and configure a second machine with another pve and another virtual router that will use CARP to do failover of the first virtual router.
I feel there is no need for me to use pve cluster features, especially since both virtual router should be 2 distinct VM running each on a distinct pve host.
No other VM should be automatically migrated from one pve host to the other when it fails...

Actually, I have one CT that is running traefik for me and is my loadbalancer/proxy to answer sollicitation from WAN clients, this CT would benefit from keeping running.
But in fact, this CT cannot be the same on both pve hosts since the failover host is much lower power/RAM.

So what I should really do is to have another CT running a smaller version of traefik on my failover pve, and spin this up when the main pve is down, just to keep some proxying working but not the full extent of what my traefik/docker CT is doing on my main pve.

I'm not sure having a pve cluster is really necessary especially since I have no physical fencing and I would turn to a raspberry pi to be the service to achieve corum.

I tend towards setting up a VIP for my traefik host and keep both the main and the failover CT running all the time.

Would welcome any advice on why I should be using pve clustering.

I see the raspberry qdevice as the main advantage, but I'm not sure that a pve cluster is well suited to spin up a different VM because the first has been shutdown on the main host or the host is down...

As you can see I'm still pretty new to this, thanks for any pointer.

 

[aaron]

A PVE cluster can help to manage multiple nodes and to migrate guests easily between them. You don't necessarily need to use HA. In fact, if you can manage HA on an application layer, it will usually be better as even with the PVE HA stack you will have a few short minutes until the guest is started on a remaining node.

Now, without any shared network storage you still have one way to use the PVE HA stack. Using ZFS as guest storage and the VM Replication. Since the replication is run in intervals (the shortest possible is every minute), you will have to decide if some dataloss since the last replication run is okay for you.

For example, I do run a small 2 node cluster and have VMs configured as HA with ZFS replication. They are smaller systems that will still work even if they do not start with the most current data. For example DNS and DHCP servers.

If you have a 2 node cluster I can really recommend to set up the QDevice.

I hope that helped you to decide

 

[toxic]

Thanks for the feedback yes !

The key in your answer for me is "if you can manage HA on an application layer, it will usually be better".

I'll keep my investigations of proxmox clusters for another time and will first find a way to failover 2 traefik instances, then I'll come back to proxmox cluster for the fun.

Because I understand that a cluster of 2 nodes needs something to break the tie, but nontheless in my case it would make an old raspberry pi 1 as a qdevice become something of a spof in my setup... Probably I'll still have other issues with other solutions as it's inherent to having 2 nodes only... We'll see.

Thanks for the ideas, will report back here in a few weeks when I sorted things out in my head and tried a few things in my setup.