2 Node HA with external QDevice

F

fluppy

New Member
Jul 10, 2023
12
1
1
We are currently running a single proxmox server to host our VMs, and I would like to build some redundancy. I was thinking of ordering an identical spec server to basically have a mirror. We also have some old servers laying around we can use for the quorum as I've read that you need 3 servers to achieve HA in Proxmox, however we don't want the VMs to run on this 3rd server as it will be old hardware.

For this 3rd server, do we have to run Proxmox on it? Or can we use it as a QDevice? I'm not sure what this means, would really appreciate an explanation and pointers.
 
In Proxmox cluster a software package called Corosync is used to maintain the cluster membership and decision making. A normal Proxmox node, which is a member of a cluster, runs a full Corosync service.
Qdevice is a light-weight vote caster designed to work within a Corosync cluster. Qdevice service does not need to run on an equally powerful server, or even same OS as the Proxmox. You can run multiple Qdevice services on single server to service multiple PVE clusters. You can run Qdevice service on Raspberry device.

Yes you can run Qdevice on an old server. To do so you need to follow instructions https://pve.proxmox.com/wiki/Cluste...External Vote Support,-This section describes

You can also run normal PVE software on the old server to get 3 members, and restrict HA to only 2 primary servers.

Good luck

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
Oh you can involve Pis? That's great I got a load of them lying around.

Is 2 servers + 1 PI sufficient? Or should we be using 2 Pis?
 
Ok I see, I'll probably just do a single Pi then and have an extra one as a cold spare ready to swap in if the Pi itself dies.
 
  • Like
Reactions: Johannes S
For anyone landing here later, the short version of what worked for me with two nodes: the QDevice is the right call, and it must live on hardware that is not one of the two cluster nodes (a Pi or any always-on Linux box is fine). If you run it as a VM or container on a node, that node going down takes its own vote and the QDevice's vote with it and you are back to square one.

The setup is three commands:

# on the external box (Debian/Ubuntu/Raspberry Pi OS)
apt install corosync-qnetd
# on EVERY cluster node
apt install corosync-qdevice
# on ONE cluster node (root SSH to the QDevice box must work)
pvecm qdevice setup 192.168.10.5

Then pvecm status should show 3 total votes with "Qdevice" listed. Two more things from running this at home: keep the nodes wired (Corosync wants consistent single-digit-millisecond latency, and Wi-Fi jitter makes a node flap in and out of the cluster), and know the emergency lever pvecm expected 1 for when one node is truly dead and you need the survivor working now. It is a lever, not a fix: two nodes that each think they are the whole cluster will write conflicting configs.

Disclosure: this is from my book Homelab the Right Way (chapter 2 is this two-node setup), https://payhip.com/b/mJKnI
 
CharlesHartmann said:
For anyone landing here later, the short version of what worked for me with two nodes: the QDevice is the right call, and it must live on hardware that is not one of the two cluster nodes (a Pi or any always-on Linux box is fine). If you run it as a VM or container on a node, that node going down takes its own vote and the QDevice's vote with it and you are back to square one.
Click to expand...
Thanks for sharing @CharlesHartmann . The steps are a match to the official guide, and I would always recommend checking there over a forum post that may get stale.

Cheers.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
@bbgeek17 Fair point, thanks. The official Proxmox docs (the Corosync External Vote Support section of the Cluster Manager chapter) are the place to check first, since commands and package details can change between releases. My post is just the short version for anyone who lands here from search.
 
You must log in or register to reply here.
Top Bottom
Back