Extra node votes in cluster (homelab)

[ott]

I have three servers. I want to be able to shut down the servers whenever without every VM dropping and having to set pvecm expected 1 to log in.

Is it ok to give the the servers 2 votes each, expected votes 2, or are there any gotchas? I was thinking something like this:

pve1 - votes = 2: mainly runs firewall (pfSense), and a few bits and bobs (unifi, xClarity admin,...).
pve2 - votes = 2: Is a nested proxmox instance on my truenas scale box. I use TNS for storage and provides nfs for proxmox over 10G.
pve3 - votes = 2: on and off server for experiments, AND nice to have when I need to update TNS/proxmox since I can live migrate whatever is running.

- All VMs run "local" on each server, no HA, but some use NFS storage
- pfSense runs on local storage, but backups to NFS

 

[fabian]

no, that makes no sense. in a cluster, you always need > 50% of the votes to be there to do any "modifying" action. giving each node more votes doesn't change this. manually overriding how much votes are expected is dangerous and shouldn't done unless you deeply understand how things work under the hood (which by the nature of your question is not the case - no offense intended!).

in a three node cluster at most one node should be down at any given time (so ideally no node is down, so that you can take the "hit" of one node going down without causing problems).

 

[ott]

no offense intended!

None taken!

What happens to VMs in a two node cluster (default votes etc) if one of the two goes down? Assuming no HA is used for any VMs.

(I know this is fringe cases, but I have a lab to break things )

 

[fabian]

nothing happens to the VM per se - you just can't modify any settings, or take any actions on the VM (like starting/stopping/taking a backup/..)

 

[UdoB]

What happens to VMs in a two node cluster (default votes etc) if one of the two goes down?

You can add a "QDevice": https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_corosync_external_vote_support to reduce the problem level of such a situation.

@fabian : any reasoning against this? You did not mention it...

Best regards

 

[fabian]

the original post was about a three node cluster - a qdevice doesn't help there (since it would only go from 3 to 4, which actually means the quorum now requires 3 nodes and only allows one node to go down, jsut like with 3/2 - and there are other problems as well with odd-numbered clusters and qdevices )

 

[mfed]

I actually run this exact configuration with each node having two votes for quite some time. As Fabian said it will not make any difference regarding the cluster quorum. My reason for this configuration was that sometimes I may add a temporary test node to the cluster and want to make sure that I can bring down this temporary node and one other node of the cluster without impact.

 

[ott]

I actually run this exact configuration with each node having two votes for quite some time. As Fabian said it will not make any difference regarding the cluster quorum. My reason for this configuration was that sometimes I may add a temporary test node to the cluster and want to make sure that I can bring down this temporary node and one other node of the cluster without impact.

ok, do you mean:
2 nodes with 2 votes each? And sometimes have the third temporary node with 1 vote?
Or, 3 nodes with 2 votes each?

 

[mfed]

ok, do you mean:
2 nodes with 2 votes each? And sometimes have the third temporary node with 1 vote?
Or, 3 nodes with 2 votes each?

I have 3 permanent nodes with 2 votes each. On rare occasions I added fourth node with default 1 vote. The whole cluster becomes 7 votes, so you need 4 votes for quorum. That means any permanent 2 nodes should be enough.