how to set quorum 3 on 6 nodes cluster ?

[sterua32]

Hi all.
I have a pve 9.1 cluster of 6 nodes.
3 nodes are in server room 1 and 3 others nodes are in server room 2.
For now, i need 4 nodes for quorum for my cluster (when i do corosync-quorumtool -s, i have Quorum: 4)
Monday, i need to halt all nodes in server room 1 because there will be a power outage (due to electrical work), and so i wish to set quorum to 3 to have my cluster still working with the 3 others nodes that will be still online ( i will migrate all vm of the 3 nodes that will be shuted down of course).
Today, for test, i try to shutdown the 3 nodes of server room 1, and i lost access to all my cluster, and all vms have halted (i see all vm was halted when i reboot the 3 nodes and had access to my cluster again).
Is there a tip, a solution to my problem?
i try to do pvecm expected 3 but i got an error : Unable to set expected votes: CS_ERR_INVALID_PARAM
Can anyone help me?
Thanks in advance.
Have a nice day.

 

[leesteken]

Changing the quorum to less or equal to 50% is a bad idea. Since you have an even number of nodes the recommendation is to add a QDevice: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_corosync_external_vote_support

There are quite a few threads on this forum about people (getting into trouble with) changing the quorum setting, running even numbered clusters (mostly 2 but 6 also applies), QDevices and running clusters that are in separate locations (and the problem they can cause).

 

[sterua32]

Thanks @leesteken for your answer.
So, if i understand correctly, i can, on any other server in my server room 2, install corosync-qnetd (and on all my 6 cluster nodes corosync-qdevice
), and then setup it by pvecm qdevice setup <QDEVICE-IP> so like that monday, my cluster will remain working with the 3 nodes halted. is that correct?
Can i add more than 1 qdevice? for example, in the future, if i add a qdevice in my server room 1, it will be possible? and, for example, if in that case, my server room 2 will be shut down (so 3 nodes shut down and 1 qdevice shut down), my cluster with the 3 remains nodes and the second qdevice will still work? or did the qdevice work as node, and so if i have 6 nodes and 2 qdevice, i will need 5 votes to have quorum?
thanks for the help

 

[sterua32]

nvm, we cant add more than 1 qdevice
in any case thanks for the help

 

[UdoB]

Can anyone help me?

You can set "expected N" as soon as only N nodes are left up and running.

I prefer to do it step-by-step: shutdown node 6 and then pvecm expected 5; shudown node 5, pvecm expected 4; shutdown node 4, pvecm expected 3.

we cant add more than 1 qdevice

Correct!

 

[SteveITS]

The Qdevice can be located anywhere accessible by all servers. It’s a 7th vote.

 

[alexskysilk]

The Qdevice can be located anywhere accessible by all servers. It’s a 7th vote.

In order for it to be a true quorum provider, it has to be accessible to all nodes but NOT PHYSICALLY LOCATED at either site. The whole point is should one of the sites drop, the quorum provider is not affected.

 

[VictorSTS]

Be mindful that if HA is enabled, around 2 minutes after losing quorum HA will reset the no-longer-in-quorum hosts. If you plan on using pvecm expected N, do it gradually as @UdoB explained above instead of setting it to 3 directly when the whole DC is off, as you will have little time to do so.

The long term solution would be to place a QDevice in a third location accessible by both DC1 and DC2, as pointed out just above.
Remember that:

• QDevice isn't restricted by Corosync's latency requirements, so it is location is quite flexible, but plan properly on what may go wrong during a DC failure to ensure QDevice reachability during such event.
• All nodes must be up and in quorum when deploying/undeploying a QDevice: plan for a QDevice in advance!