Removed Qdevice to add server, still has flag Qdevice

[SteveITS]

I had previously created a Qdevice. Today I removed it:

# pvecm qdevice remove
Synchronizing state of corosync-qdevice.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable corosync-qdevice
Removed "/etc/systemd/system/multi-user.target.wants/corosync-qdevice.service".
Synchronizing state of corosync-qdevice.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable corosync-qdevice
Removed "/etc/systemd/system/multi-user.target.wants/corosync-qdevice.service".
Synchronizing state of corosync-qdevice.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable corosync-qdevice
Removed "/etc/systemd/system/multi-user.target.wants/corosync-qdevice.service".
Synchronizing state of corosync-qdevice.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable corosync-qdevice
Removed "/etc/systemd/system/multi-user.target.wants/corosync-qdevice.service".
Reloading corosync.conf...
Done

Removed Qdevice.

I then added a fifth node. If I look at the status it still shows a Qdevice flag and Qdevice column?

# pvecm status
Cluster information
-------------------
Name:             clustername
Config Version:   8
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Tue May  6 15:44:34 2025
Quorum provider:  corosync_votequorum
Nodes:            5
Node ID:          0x00000001
Ring ID:          1.5d
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   5
Highest expected: 5
Total votes:      5
Quorum:           3
Flags:            Quorate Qdevice

Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000001          1  NA,NV,NMW 10.1.1.107 (local)
0x00000002          1         NR 10.1.1.104
0x00000003          1  NA,NV,NMW 10.1.1.102
0x00000004          1         NR 10.1.1.103
0x00000005          1         NR 10.1.1.108
0x00000000          0            Qdevice (votes 0)

I was not expecting to see anything about Qdevice. Did it not fully remove? Does it display there permanently now?

Of note, .103 and .104 were restarted, and .108 is new. Do I need to restart the other two?

My next goal was to decommission an old server and add a Qdevice again...

 

[SInisterPisces]

I'm assuming it's not throwing any errors since you didn't mention it. Everything is working?

I've not set up a cluster yet. But I've watched some videos on how to do it, and from what I've learned so far, I think this is okay? Even if it doesn't disappear when you reboot all the nodes, it doesn't look to be causing a problem.

Votequorum information
----------------------
Expected votes:   5
Highest expected: 5
Total votes:      5
Quorum:           3
Flags:            Quorate Qdevice

You have 5 nodes. It knows this.

Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000001          1  NA,NV,NMW 10.1.1.107 (local)
0x00000002          1         NR 10.1.1.104
0x00000003          1  NA,NV,NMW 10.1.1.102
0x00000004          1         NR 10.1.1.103
0x00000005          1         NR 10.1.1.108
0x00000000          0            Qdevice (votes 0)

The Qdevice has 0 votes. It knows it's no longer eligible to vote, so it shouldn't try to use it--whether it's supposed to be there or not.

I'm not sure if your current configuration is correct, but it looks like it's not causing an obvious problem.

 

[SteveITS]

Hi, thanks for the reply. I found no errors.

As it happens though, the PDU in our data center rack was acting up so I had to shut everything down tonight and move power cords (yay!). After rebooting all servers, cluster status is clean again...no mention of a Qdevice. So I guess it was just leftover info.

 

[SInisterPisces]

Hi, thanks for the reply. I found no errors.

As it happens though, the PDU in our data center rack was acting up so I had to shut everything down tonight and move power cords (yay!). After rebooting all servers, cluster status is clean again...no mention of a Qdevice. So I guess it was just leftover info.

Glad it's working and that you survived the PDU cable fix.

Every time I have to mess with my cables, I pull my back and inhale 5 pounds of carpet dust.

 

[SteveITS]

5 pounds of carpet dust.

One nice thing about a large data center...no dust. It is horrifically noisy though...I learned early to wear earplugs. Forgot them last night and my ears were ringing after being there a half hour.

We just carefully migrated everything to Proxmox VE and three weeks in we need to shut down the entire cluster. On the plus side we were there to notice the error message.