NR status for quorum nodes in three node cluster

zandhaas · Tuesday at 14:00

Hello all,

I have had a two node cluster and a qdevice as thrid quorum device. Last week I added a third node to the cluster and sunce I see below "pvecm status" overview.
Two of the node have a NR (Not registered" quorum status. The clsuetr itself seem work as expected but.......

Is there a way two get the NR quorums registered??

Code:

Cluster information
-------------------
Name:             thuis-Cluster
Config Version:   9
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Tue Sep  2 13:53:55 2025
Quorum provider:  corosync_votequorum
Nodes:            3
Node ID:          0x00000003
Ring ID:          1.2f4
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      3
Quorum:           2 
Flags:            Quorate

Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000001          1         NR 192.168.0.57
0x00000002          1  NA,NV,NMW 192.168.0.58
0x00000003          1         NR 192.168.0.56 (local)
0x00000000          0            Qdevice (votes 0)

dakralex · Tuesday at 15:08

Hi!

How did you add the node to the cluster? Note that it is required to remove the QDevice before adding (or deleting for that matter) a node to the cluster [0]. Also note that a QDevice in a odd numbered cluster is discouraged, see [1] for more information.

[0] https://pve.proxmox.com/pve-docs/chapter-pvecm.html#_frequently_asked_questions
[1] https://pve.proxmox.com/pve-docs/chapter-pvecm.html#_supported_setups

zandhaas · 2025-09-03T07:55:54+0200

I added the third node from the commandline before I was aware of the fact that I first had to remove the qdevice.
I removed the qdevice after adding the third node with above result.

My ultimate goal is to remove node2 from the cluster (Old laptop) and have again a two node cluster with a qdevice. But with the current status I'm a bit afraid things go wrong because node 2 is the ony node with a, for me, normal quorum status.

It looks like there is still a qdevice but when I again try to delete it I get below anwser.

Code:

root@proxmox2:~# pvecm  qdevice remove
No QDevice configured!

root@proxmox2:~# pvecm status
Cluster information
-------------------
Name:             thuis-Cluster
Config Version:   9
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Wed Sep  3 07:54:28 2025
Quorum provider:  corosync_votequorum
Nodes:            3
Node ID:          0x00000003
Ring ID:          1.2f4
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      3
Quorum:           2
Flags:            Quorate

Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000001          1         NR 192.168.0.57
0x00000002          1  NA,NV,NMW 192.168.0.58
0x00000003          1         NR 192.168.0.56 (local)
0x00000000          0            Qdevice (votes 0)

zandhaas · 2025-09-03T09:12:09+0200

zandhaas said:
I added the third node from the commandline before I was aware of the fact that I first had to remove the qdevice.
I removed the qdevice after adding the third node with above result.

My ultimate goal is to remove node2 from the cluster (Old laptop) and have again a two node cluster with a qdevice. But with the current status I'm a bit afraid things go wrong because node 2 is the ony node with a, for me, normal quorum status.

It looks like there is still a qdevice but when I again try to delete it I get below anwser.

Code:

root@proxmox2:~# pvecm qdevice remove No QDevice configured! root@proxmox2:~# pvecm status Cluster information ------------------- Name: thuis-Cluster Config Version: 9 Transport: knet Secure auth: on Quorum information ------------------ Date: Wed Sep 3 07:54:28 2025 Quorum provider: corosync_votequorum Nodes: 3 Node ID: 0x00000003 Ring ID: 1.2f4 Quorate: Yes Votequorum information ---------------------- Expected votes: 3 Highest expected: 3 Total votes: 3 Quorum: 2 Flags: Quorate Membership information ---------------------- Nodeid Votes Qdevice Name 0x00000001 1 NR 192.168.0.57 0x00000002 1 NA,NV,NMW 192.168.0.58 0x00000003 1 NR 192.168.0.56 (local) 0x00000000 0 Qdevice (votes 0) [/QUOTE]

zandhaas · 2025-09-03T09:21:46+0200

OK.

I did some more research on the corosync related deamons and found out that the corosync-qdevice deamon on all three active nodes was in a failed status. So I decided to stop those deamons on all three nodes with

Bash:

systemctl stop corosync-qdevice.service

.

now the output of pvecm status gives below result

Code:

root@proxmox:~# pvecm status
Cluster information
-------------------
Name:             thuis-Cluster
Config Version:   9
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Wed Sep  3 09:10:42 2025
Quorum provider:  corosync_votequorum
Nodes:            3
Node ID:          0x00000002
Ring ID:          1.2f4
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      3
Quorum:           2
Flags:            Quorate Qdevice

Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000001          1  NA,NV,NMW 192.168.0.57
0x00000002          1  NA,NV,NMW 192.168.0.58 (local)
0x00000003          1  NA,NV,NMW 192.168.0.56
0x00000000          0            Qdevice (votes 0)

I do not know if it's normal that I still see the Qdevice line in the output.

Next step will be removing node (192.168.0.58) from the cluster and I hope I can add the qdevice again.

zandhaas · 2025-09-03T14:40:03+0200

zandhaas said:
OK.

I did some more research on the corosync related deamons and found out that the corosync-qdevice deamon on all three active nodes was in a failed status. So I decided to stop those deamons on all three nodes with

Bash:

systemctl stop corosync-qdevice.service

.

now the output of pvecm status gives below result

Code:

root@proxmox:~# pvecm status Cluster information ------------------- Name: thuis-Cluster Config Version: 9 Transport: knet Secure auth: on Quorum information ------------------ Date: Wed Sep 3 09:10:42 2025 Quorum provider: corosync_votequorum Nodes: 3 Node ID: 0x00000002 Ring ID: 1.2f4 Quorate: Yes Votequorum information ---------------------- Expected votes: 3 Highest expected: 3 Total votes: 3 Quorum: 2 Flags: Quorate Qdevice Membership information ---------------------- Nodeid Votes Qdevice Name 0x00000001 1 NA,NV,NMW 192.168.0.57 0x00000002 1 NA,NV,NMW 192.168.0.58 (local) 0x00000003 1 NA,NV,NMW 192.168.0.56 0x00000000 0 Qdevice (votes 0)

I do not know if it's normal that I still see the Qdevice line in the output.

Next step will be removing node (192.168.0.58) from the cluster and I hope I can add the qdevice again.

A last short update.

I Succesfully removed the old laptop from the cluster and everything looks still OK for now.

Search

Search

NR status for quorum nodes in three node cluster

zandhaas

New Member

dakralex

Proxmox Staff Member

zandhaas

New Member

zandhaas

New Member

zandhaas

New Member

zandhaas

New Member

We value your privacy