QDevice Deployment

I

IceManJB

New Member
Sep 29, 2025
2
0
1
I am looking for information on how can a QDevice be deployed. Can it be deployed as a remote witness? Like a witness in VMware can be deployed remotely.
 
Hi @IceManJB , welcome to the forum.

A Qdevice is a lightweight component of the Corosync package that provides just enough functionality to act as a tie-breaking vote in a cluster. It can run on an isolated or dedicated VM, container, or bare-metal node.

The Qdevice can be located in a different rack, room, datacenter, or even town. However, because it participates in cluster quorum, the underlying requirements are the same as for any other cluster member: fast communication and low latency.

Hope this helps.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
  • Like
Reactions: Johannes S and IceManJB
bbgeek17 said:
the underlying requirements are the same as for any other cluster member: fast communication and low latency.
Click to expand...

I believe this is a myth :-)


Let's take a look. I've added a QDev to a four node virtual test-cluster. Normal operation:
Code: 
~# pvecm status

Cluster information
-------------------
Name:             pn2
Config Version:   5
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Thu Oct  2 14:23:55 2025
Quorum provider:  corosync_votequorum
Nodes:            4
Node ID:          0x00000001
Ring ID:          1.183
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   5
Highest expected: 5
Total votes:      5
Quorum:           3
Flags:            Quorate Qdevice

Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000001          1    A,V,NMW 10.2.192.21 (local)
0x00000002          1    A,V,NMW 10.2.192.22
0x00000003          1    A,V,NMW 10.2.192.24
0x00000004          1    A,V,NMW 10.2.192.25
0x00000000          1            Qdevice

If I stop it that vote vanishes immediately. (As intended.)

Now what about latency. On my virtual test-cluster I can insert artificial delays on the VM with the Quorum Device:
Code: 
~# tc qdisc add dev ens18 root netem delay  10ms

To confirm the delay is active just "ping" it from a cluster node:
64 bytes from 10.2.140.141: icmp_seq=85 ttl=64 time=0.182 ms
64 bytes from 10.2.140.141: icmp_seq=86 ttl=64 time=0.251 ms
64 bytes from 10.2.140.141: icmp_seq=87 ttl=64 time=0.310 ms
64 bytes from 10.2.140.141: icmp_seq=88 ttl=64 time=10.3 ms
64 bytes from 10.2.140.141: icmp_seq=89 ttl=64 time=10.4 ms
64 bytes from 10.2.140.141: icmp_seq=90 ttl=64 time=10.3 ms

I *want* to see a malfunction, delay set to 1000 ms:
Code: 
~# tc qdisc del dev ens18 root;  tc qdisc add dev ens18 root netem delay  1000ms
Code: 
~# date; pvecm status | grep -A 5 Vote
Thu Oct  2 02:34:19 PM CEST 2025
Votequorum information
----------------------
Expected votes:   5
Highest expected: 5
Total votes:      5
Quorum:           3
--
    Nodeid      Votes    Qdevice Name
0x00000001          1    A,V,NMW 10.2.192.21 (local)
0x00000002          1    A,V,NMW 10.2.192.22
0x00000003          1    A,V,NMW 10.2.192.24
0x00000004          1    A,V,NMW 10.2.192.25
0x00000000          1            Qdevice
Even five minutes later the vote does NOT vanish!

Is working with 10 full seconds even possible? ssh is really a *pain* in this mode ;-)
Code: 
~# tc qdisc del dev ens18 root;  tc qdisc add dev ens18 root netem delay  10000ms

64 bytes from 10.2.140.141: icmp_seq=29 ttl=64 time=10001 ms
64 bytes from 10.2.140.141: icmp_seq=30 ttl=64 time=10001 ms
From 10.2.192.21 icmp_seq=38 Destination Host Unreachable
From 10.2.192.21 icmp_seq=39 Destination Host Unreachable
From 10.2.192.21 icmp_seq=40 Destination Host Unreachable
64 bytes from 10.2.140.141: icmp_seq=31 ttl=64 time=10001 ms
64 bytes from 10.2.140.141: icmp_seq=32 ttl=64 time=10001 ms
64 bytes from 10.2.140.141: icmp_seq=33 ttl=64 time=10000 ms

For this test this is a successful one:
Code: 
~# date; pvecm status | grep -A 5 Vote
Thu Oct  2 02:41:20 PM CEST 2025
Votequorum information
----------------------
Expected votes:   5
Highest expected: 5
Total votes:      4
Quorum:           3
--
    Nodeid      Votes    Qdevice Name
0x00000001          1   A,NV,NMW 10.2.192.21 (local)
0x00000002          1   A,NV,NMW 10.2.192.22
0x00000003          1   A,NV,NMW 10.2.192.24
0x00000004          1   A,NV,NMW 10.2.192.25
0x00000000          0            Qdevice (votes 1)

Stepping back to 1000ms brings back the Quorum Device's vote without any additional/manual action.

I have no idea where the maximum latency is specified or documented. While the recommendation for low latency is a good one, it works well with some hundred milliseconds.
 
Last edited:
UdoB said:
Stepping back to 1000ms brings back the Quorum Device's vote without any additional/manual action.
Click to expand...
I think you may be testing a somewhat unrelated scenario. The purpose of a qdevice vote is to break ties. It could be slow or unresponsive 364 days a year, but on the one day when your 4-node cluster splits in half (e.g. 2 nodes go down hard), that’s when the qdevice matters.

The OP’s original question was whether the qdevice/vote can be remote. Technically, yes, it can. But the more important question is "why" you would do that. If all nodes are in the same datacenter, there’s no benefit to placing the qdevice remotely.

If your cluster is split across two rooms or datacenters, then the qdevice should be placed in a third independent location. Otherwise, it won’t provide effective tie-breaking.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
  • Like
Reactions: Johannes S
You must log in or register to reply here.
Top Bottom