<50% & Quorum?

[SlothCroissant]

Hey all,

I have a cluster of 7 SFF PCs running as a PVE HA cluster backed by NFS storage (on an 8th machine, also in the cluster). I'm on "time-of-use" electricity costs, and would love to be able to kill off many of the SFF nodes during "peak" hours to save on energy costs, or if compute power isn't needed for some time.

However, due to Quorum rules, I need 5 machines running to obtain quorum (50% +1). Once I shut down more than 3 of the SFF PCs, quorum is lost and all heck breaks loose. So I have a few asks:

1. are there recommendations on how one could safely reduce the number of nodes active in a cluster beyond 50%? Everything about PVE HA works great for me (shutdown initiates migration to other nodes, etc), except for this one main feature that is a blocker of sorts. One option that I've seen that I'd like to consider is last_man_standing feature of votequorum, or manually setting expected_votes but haven't seen it much when searching specifically related to PVE (or haven't seen it referenced in years).
2. Is it expected that everything stops working when quorum is lost? It seems that once quorum is lost, all hosts, even those running perfectly fine on healthy alive nodes, stop working. Even if I pre-migrate all my VMs to alive nodes, then shut down the old, now-empty nodes, everything grinds to a halt. I expect this is expected behavior but would like to confirm.
3. If I'm out of options, is there any other recommendation on how I can achieve this?

 

[Matthias.]

1: You can use an external vote device https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_corosync_external_vote_support
Do NOT set expected_votes if you are not 100% certain that you know what you're doing

2: yes, that's how it's supposed to work

3: see the link in 1

 

[fabian]

an external vote device won't really help in this situation though (it's for providing a tie-breaker in clusters with an even node count). PVE isn't really made for such dynamic scaling of the cluster itself unfortunately.

with HA completely disabled, migrating guests to the partition of the cluster that will remain up (without quorum) before shutting down the remaining nodes should work - you won't be able to change anything while the cluster is not quorate (including starting/stopping guests, creating backups, ..), but already running guests should continue running.. with HA enabled that won't work of course, since losing quorum means fencing