[SOLVED] QDevice Deployment
- Thread starter IceManJB
- Start date
Hi @IceManJB , welcome to the forum.
A Qdevice is a lightweight component of the Corosync package that provides just enough functionality to act as a tie-breaking vote in a cluster. It can run on an isolated or dedicated VM, container, or bare-metal node.
The Qdevice can be located in a different rack, room, datacenter, or even town. However, because it participates in cluster quorum, the underlying requirements are the same as for any other cluster member: fast communication and low latency.
Hope this helps.
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
A Qdevice is a lightweight component of the Corosync package that provides just enough functionality to act as a tie-breaking vote in a cluster. It can run on an isolated or dedicated VM, container, or bare-metal node.
The Qdevice can be located in a different rack, room, datacenter, or even town. However, because it participates in cluster quorum, the underlying requirements are the same as for any other cluster member: fast communication and low latency.
Hope this helps.
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
Glad I could help.
If you are all set, feel free to mark this thread as Solved by editing the original post and selecting Solved from the drop down near subject.
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
If you are all set, feel free to mark this thread as Solved by editing the original post and selecting Solved from the drop down near subject.
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
I believe this is a myth
Let's take a look. I've added a QDev to a four node virtual test-cluster. Normal operation:
Code:
~# pvecm status
Cluster information
-------------------
Name: pn2
Config Version: 5
Transport: knet
Secure auth: on
Quorum information
------------------
Date: Thu Oct 2 14:23:55 2025
Quorum provider: corosync_votequorum
Nodes: 4
Node ID: 0x00000001
Ring ID: 1.183
Quorate: Yes
Votequorum information
----------------------
Expected votes: 5
Highest expected: 5
Total votes: 5
Quorum: 3
Flags: Quorate Qdevice
Membership information
----------------------
Nodeid Votes Qdevice Name
0x00000001 1 A,V,NMW 10.2.192.21 (local)
0x00000002 1 A,V,NMW 10.2.192.22
0x00000003 1 A,V,NMW 10.2.192.24
0x00000004 1 A,V,NMW 10.2.192.25
0x00000000 1 QdeviceIf I stop it that vote vanishes immediately. (As intended.)
Now what about latency. On my virtual test-cluster I can insert artificial delays on the VM with the Quorum Device:
Code:
~# tc qdisc add dev ens18 root netem delay 10ms
To confirm the delay is active just "ping" it from a cluster node:
64 bytes from 10.2.140.141: icmp_seq=85 ttl=64 time=0.182 ms
64 bytes from 10.2.140.141: icmp_seq=86 ttl=64 time=0.251 ms
64 bytes from 10.2.140.141: icmp_seq=87 ttl=64 time=0.310 ms
64 bytes from 10.2.140.141: icmp_seq=88 ttl=64 time=10.3 ms
64 bytes from 10.2.140.141: icmp_seq=89 ttl=64 time=10.4 ms
64 bytes from 10.2.140.141: icmp_seq=90 ttl=64 time=10.3 msI *want* to see a malfunction, delay set to 1000 ms:
Code:
~# tc qdisc del dev ens18 root; tc qdisc add dev ens18 root netem delay 1000ms
Code:
~# date; pvecm status | grep -A 5 Vote
Thu Oct 2 02:34:19 PM CEST 2025
Votequorum information
----------------------
Expected votes: 5
Highest expected: 5
Total votes: 5
Quorum: 3
--
Nodeid Votes Qdevice Name
0x00000001 1 A,V,NMW 10.2.192.21 (local)
0x00000002 1 A,V,NMW 10.2.192.22
0x00000003 1 A,V,NMW 10.2.192.24
0x00000004 1 A,V,NMW 10.2.192.25
0x00000000 1 QdeviceIs working with 10 full seconds even possible? ssh is really a *pain* in this mode ;-)
Code:
~# tc qdisc del dev ens18 root; tc qdisc add dev ens18 root netem delay 10000ms
64 bytes from 10.2.140.141: icmp_seq=29 ttl=64 time=10001 ms
64 bytes from 10.2.140.141: icmp_seq=30 ttl=64 time=10001 ms
From 10.2.192.21 icmp_seq=38 Destination Host Unreachable
From 10.2.192.21 icmp_seq=39 Destination Host Unreachable
From 10.2.192.21 icmp_seq=40 Destination Host Unreachable
64 bytes from 10.2.140.141: icmp_seq=31 ttl=64 time=10001 ms
64 bytes from 10.2.140.141: icmp_seq=32 ttl=64 time=10001 ms
64 bytes from 10.2.140.141: icmp_seq=33 ttl=64 time=10000 msFor this test this is a successful one:
Code:
~# date; pvecm status | grep -A 5 Vote
Thu Oct 2 02:41:20 PM CEST 2025
Votequorum information
----------------------
Expected votes: 5
Highest expected: 5
Total votes: 4
Quorum: 3
--
Nodeid Votes Qdevice Name
0x00000001 1 A,NV,NMW 10.2.192.21 (local)
0x00000002 1 A,NV,NMW 10.2.192.22
0x00000003 1 A,NV,NMW 10.2.192.24
0x00000004 1 A,NV,NMW 10.2.192.25
0x00000000 0 Qdevice (votes 1)Stepping back to 1000ms brings back the Quorum Device's vote without any additional/manual action.
I have no idea where the maximum latency is specified or documented. While the recommendation for low latency is a good one, it works well with some hundred milliseconds.
Last edited:
I think you may be testing a somewhat unrelated scenario. The purpose of a qdevice vote is to break ties. It could be slow or unresponsive 364 days a year, but on the one day when your 4-node cluster splits in half (e.g. 2 nodes go down hard), that’s when the qdevice matters.
The OP’s original question was whether the qdevice/vote can be remote. Technically, yes, it can. But the more important question is "why" you would do that. If all nodes are in the same datacenter, there’s no benefit to placing the qdevice remotely.
If your cluster is split across two rooms or datacenters, then the qdevice should be placed in a third independent location. Otherwise, it won’t provide effective tie-breaking.
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
I was asking the remote question for the QDevice to be something similar to how a 2-node ROBO vSan cluster is with VMware. I am also not sure if a QDevice can be shared with another cluster as it seems the QDevice needs to be a one-to-one ratio. Right now it would seem for best practice that the QDevice be local and deployed on something small like raspberry PI device.
It’s designed to support multiple clusters and is almost configuration and state free
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox