I run a 4 node + qdevice cluster. When all nodes + qdevice are online expected votes are 5 and quorum is 3.
I currently have 2 nodes offline, making it a (temporary) 2 node cluster + qdevice. Quorum remains 3, but (unclear to me) the expected votes are set to 4.
When I reboot the qdevice my nodes reboot, since quorum is lost. I believe this is expected.
If I am not wrong, a 2 node cluster + qdevice would have a default quorum of 2.
Could I temporarily set the quorum to 2? (until my offline nodes come back from maintenance (weeks, worst case moths))
From reading the manual, it doesn't seem like
All 4 nodes + qdevice online:
2 nodes + qdevice online:
This is what happens to the nodes when the qdevice reboot:
I currently have 2 nodes offline, making it a (temporary) 2 node cluster + qdevice. Quorum remains 3, but (unclear to me) the expected votes are set to 4.
When I reboot the qdevice my nodes reboot, since quorum is lost. I believe this is expected.
If I am not wrong, a 2 node cluster + qdevice would have a default quorum of 2.
Could I temporarily set the quorum to 2? (until my offline nodes come back from maintenance (weeks, worst case moths))
From reading the manual, it doesn't seem like
pvecm expected 2 is what I want to use, but maybe it's just a bit ambiguous to me since I am not very familiar with corosync. How would I change it? Or would I change expected votes from 4 to 3 with pvecm expected 3 and PVE calculates the quorum (of 2) from it?All 4 nodes + qdevice online:
Code:
pvecm status
Cluster information
-------------------
Name: msc
Config Version: 5
Transport: knet
Secure auth: on
Quorum information
------------------
Date: Thu Jun 19 10:26:21 2025
Quorum provider: corosync_votequorum
Nodes: 4
Node ID: 0x00000003
Ring ID: 1.32d
Quorate: Yes
Votequorum information
----------------------
Expected votes: 5
Highest expected: 5
Total votes: 5
Quorum: 3
Flags: Quorate Qdevice
Membership information
----------------------
Nodeid Votes Qdevice Name
0x00000001 1 A,V,NMW 10.10.1.150
0x00000002 1 A,V,NMW 10.10.1.160
0x00000003 1 A,V,NMW 10.10.1.170 (local)
0x00000004 1 A,V,NMW 10.10.1.180
0x00000000 1 Qdevice2 nodes + qdevice online:
Code:
pvecm status
Cluster information
-------------------
Name: msc
Config Version: 5
Transport: knet
Secure auth: on
Quorum information
------------------
Date: Thu Jun 19 10:15:32 2025
Quorum provider: corosync_votequorum
Nodes: 2
Node ID: 0x00000001
Ring ID: 1.325
Quorate: Yes
Votequorum information
----------------------
Expected votes: 4
Highest expected: 4
Total votes: 3
Quorum: 3
Flags: Quorate Qdevice
Membership information
----------------------
Nodeid Votes Qdevice Name
0x00000001 1 A,V,NMW 10.10.1.150 (local)
0x00000003 1 A,V,NMW 10.10.1.170
0x00000000 1 QdeviceThis is what happens to the nodes when the qdevice reboot:
Code:
ms03 corosync[1463]: [QUORUM] This node is within the non-primary component and will NOT provide any services.
ms03 corosync[1463]: [QUORUM] Members[2]: 1 3
ms03 pmxcfs[1358]: [status] notice: node lost quorum
ms03 corosync-qdevice[1484]: Can't connect to qnetd host. (-5986): Network address not available (in use?)
ms03 pve-ha-lrm[1535]: lost lock 'ha_agent_ms03_lock - cfs lock update failed - Permission denied
ms03 pve-ha-crm[1523]: lost lock 'ha_manager_lock - cfs lock update failed - Permission denied
ms03 pve-ha-lrm[1535]: status change active => lost_agent_lock
ms03 pve-ha-crm[1523]: status change master => lost_manager_lock
ms03 pve-ha-crm[1523]: watchdog closed (disabled)
ms03 pve-ha-crm[1523]: status change lost_manager_lock => wait_for_quorum
ms03 corosync-qdevice[1484]: Connect timeout
ms03 corosync-qdevice[1484]: Can't connect to qnetd host. (-5986): Network address not available (in use?)
ms03 corosync-qdevice[1484]: Connect timeout
ms03 corosync-qdevice[1484]: Can't connect to qnetd host. (-5986): Network address not available (in use?)
ms03 corosync-qdevice[1484]: Connect timeout
ms03 corosync-qdevice[1484]: Can't connect to qnetd host. (-5986): Network address not available (in use?)
ms03 corosync-qdevice[1484]: Connect timeout
ms03 corosync-qdevice[1484]: Can't connect to qnetd host. (-5986): Network address not available (in use?)
ms03 corosync-qdevice[1484]: Connect timeout
ms03 corosync-qdevice[1484]: Can't connect to qnetd host. (-5986): Network address not available (in use?)
ms03 corosync-qdevice[1484]: Connect timeout
ms03 corosync-qdevice[1484]: Can't connect to qnetd host. (-5986): Network address not available (in use?)
ms03 watchdog-mux[1188]: client watchdog expired - disable watchdog updates
ms03 pvescheduler[2889801]: jobs: cfs-lock 'file-jobs_cfg' error: no quorum!
ms03 pvescheduler[2889800]: replication: cfs-lock 'file-replication_cfg' error: no quorum!
@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@2025-06-19T>
ms03 kernel: [ 0.000000] Linux version 6.8.12-11-pve (build@proxmox) (gcc (Debian 12.2.0-14+deb12u1) 12.2.0, GNU ld (GNU
Last edited: