Added new host without disabling qdevice, now can't remove the qdevice

[Cylindric]

Hi folks. I have a Proxmox cluster that I've had up for a while now, consisting of two nodes and a pi running as a QDevice.
I recently aquired a new server so added that as a cluster node without realising that I was supposed to remove the qdevice first.

I now have a cluster with a qdevice that I can't seem to remove. I'm also not sure why my new node is showing with an "NR" code and the others have "A,V,NMW" - I couldn't find any information on those codes anywhere.

I'm currently experiencing a problem whereby rebooting one cluster node (in today's case, #1) caused the whole cluster to crap itself and halt all VMs on all hosts. I thought removing the qdevice and returning to a 'supported' three-node configuration would be a good start, but it's proving tricky.

pvecm qdevice remove:

No QDevice configured!

pvecm status:

Cluster information
-------------------
Name:             home
Config Version:   7
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Thu Aug 26 17:02:46 2021
Quorum provider:  corosync_votequorum
Nodes:            3
Node ID:          0x00000003
Ring ID:          1.1cf
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   4
Highest expected: 4
Total votes:      3
Quorum:           3
Flags:            Quorate

Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000001          1    A,V,NMW 172.29.14.155
0x00000002          1    A,V,NMW 172.29.14.153
0x00000003          1         NR 172.29.14.154 (local)
0x00000000          0            Qdevice (votes 1)

All proxmox nodes are running pve-manager/7.0-11/63d82f4e.

 

[Cylindric]

I rebooted my first prox server and then on my third, the HA status changed to "no quorum"

A quick pvecm status (on prox2 in this example) shows:

Cluster information
-------------------
Name:             home
Config Version:   7
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Sat Aug 28 13:51:32 2021
Quorum provider:  corosync_votequorum
Nodes:            1
Node ID:          0x00000001
Ring ID:          1.1e3
Quorate:          No

Votequorum information
----------------------
Expected votes:   4
Highest expected: 4
Total votes:      1
Quorum:           3 Activity blocked
Flags:            Qdevice

Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000001          1   A,NV,NMW 172.29.14.155 (local)
0x00000000          0            Qdevice (votes 1)

 

[cccdemon]

Same Problem here.

Linux 5.11.22-3-pve #1 SMP PVE 5.11.22-7 (Wed, 18 Aug 2021 15:06:12 +0200
pve-manager/7.0-11/63d82f4e

root@core1:~# pvecm status
Cluster information
-------------------
Name:             ea02
Config Version:   2
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Thu Oct  7 10:20:30 2021
Quorum provider:  corosync_votequorum
Nodes:            2
Node ID:          0x00000001
Ring ID:          1.3b
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   2
Highest expected: 2
Total votes:      2
Quorum:           2
Flags:            Quorate Qdevice

Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000001          1  NA,NV,NMW fd00:2380:ea02:47::1%32698 (local)
0x00000002          1         NR fd00:2380:ea02:47::2%32698
0x00000000          0            Qdevice (votes 0)

 

[cccdemon]

Solved with:

pvecm updatecerts #on all pve nodes
# retry the qdevice installation

pvecm qdevice setup fd00:

#works now

 

[freddyvdh]

Thanks, worked for me too