quorum device setup issue "corosync-qdevice Default-Start contains no runlevels"

G

Giovanni

Renowned Member
Apr 1, 2009
112
12
83
Hi,

I'm having some trouble setting up the qdevice following the documentation [1]

Code: 
# pvecm qdevice setup 172.16.0.69 -f
/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/bin/ssh-copy-id: WARNING: All keys were skipped because they already exist on the remote system.
                (if you think this is a mistake, you may want to use -f option)


INFO: initializing qnetd server
Certificate database (/etc/corosync/qnetd/nssdb) already exists. Delete it to initialize new db

INFO: copying CA cert and initializing on all nodes

node 'VE-H310': Creating /etc/corosync/qdevice/net/nssdb
password file contains no data
node 'VE-H310': Creating new key and cert db
node 'VE-H310': Creating new noise file /etc/corosync/qdevice/net/nssdb/noise.txt
node 'VE-H310': Importing CA
node 'VE-SSD': Creating /etc/corosync/qdevice/net/nssdb
password file contains no data
node 'VE-SSD': Creating new key and cert db
node 'VE-SSD': Creating new noise file /etc/corosync/qdevice/net/nssdb/noise.txt
node 'VE-SSD': Importing CA
INFO: generating cert request
Creating new certificate request


Generating key.  This may take a few moments...

Certificate request stored in /etc/corosync/qdevice/net/nssdb/qdevice-net-node.crq

INFO: copying exported cert request to qnetd server

INFO: sign and export cluster cert
Signing cluster certificate
Certificate stored in /etc/corosync/qnetd/nssdb/cluster-LAX.crt

INFO: copy exported CRT

INFO: import certificate
Importing signed cluster certificate
Notice: Trust flag u is set automatically if the private key is present.
pk12util: PKCS12 EXPORT SUCCESSFUL
Certificate stored in /etc/corosync/qdevice/net/nssdb/qdevice-net-node.p12

INFO: copy and import pk12 cert to all nodes

node 'VE-H310': Importing cluster certificate and key
node 'VE-H310': pk12util: PKCS12 IMPORT SUCCESSFUL
node 'VE-SSD': Importing cluster certificate and key
node 'VE-SSD': pk12util: PKCS12 IMPORT SUCCESSFUL
INFO: add QDevice to cluster configuration

INFO: start and enable corosync qdevice daemon on node 'VE-H310'...
Synchronizing state of corosync-qdevice.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable corosync-qdevice
update-rc.d: error: corosync-qdevice Default-Start contains no runlevels, aborting.
command 'ssh -o 'BatchMode=yes' -lroot 192.168.255.151 systemctl enable corosync-qdevice' failed: exit code 1
root@VE-SSD:~#

I've logged onto the PVE nodes and tried to run the systemctl enable command and both of them fail. Not sure if there's a new requirement or something I missed.


[1] https://pve.proxmox.com/wiki/Cluster_Manager
 
On the PVE nodes I edited the startup script and added some info to the runlevels. Command now works but forcing a re-setup fails, seems like it thinks the qdevice was setup properly and must be removed.

/etc/init.d/corosync-qdevice

### BEGIN INIT INFO
# Provides: corosync-qdevice
# Required-Start: $remote_fs $syslog corosync
# Required-Stop: $remote_fs $syslog corosync
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Corosync Qdevice daemon
# Description: Starts and stops Corosync Qdevice daemon.
### END INIT INFO
Click to expand...

Hmmm....

Code: 
root@VE-SSD:~# pvecm qdevice remove
Synchronizing state of corosync-qdevice.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable corosync-qdevice
update-rc.d: error: no runlevel symlinks to modify, aborting!
command 'ssh -o 'BatchMode=yes' -lroot 192.168.255.151 systemctl disable corosync-qdevice' failed: exit code 1
 
D'oh forgot to regen the init.d scripts after the edit this will do:

# update-rc.d -f corosync-qdevice remove
# update-rc.d -f corosync-qdevice defaults

After that I remove the qdevice settings
# pvecm qdevice remove

and re-try
Code: 
# pvecm qdevice setup 172.16.0.69 -f
/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/bin/ssh-copy-id: WARNING: All keys were skipped because they already exist on the remote system.
                (if you think this is a mistake, you may want to use -f option)


INFO: initializing qnetd server
Certificate database (/etc/corosync/qnetd/nssdb) already exists. Delete it to initialize new db

INFO: copying CA cert and initializing on all nodes

node 'VE-H310': Creating /etc/corosync/qdevice/net/nssdb
password file contains no data
node 'VE-H310': Creating new key and cert db
node 'VE-H310': Creating new noise file /etc/corosync/qdevice/net/nssdb/noise.txt
node 'VE-H310': Importing CA
node 'VE-SSD': Creating /etc/corosync/qdevice/net/nssdb
password file contains no data
node 'VE-SSD': Creating new key and cert db
node 'VE-SSD': Creating new noise file /etc/corosync/qdevice/net/nssdb/noise.txt
node 'VE-SSD': Importing CA
INFO: generating cert request
Creating new certificate request


Generating key.  This may take a few moments...

Certificate request stored in /etc/corosync/qdevice/net/nssdb/qdevice-net-node.crq

INFO: copying exported cert request to qnetd server

INFO: sign and export cluster cert
Signing cluster certificate
Certificate stored in /etc/corosync/qnetd/nssdb/cluster-LAX.crt

INFO: copy exported CRT

INFO: import certificate
Importing signed cluster certificate
Notice: Trust flag u is set automatically if the private key is present.
pk12util: PKCS12 EXPORT SUCCESSFUL
Certificate stored in /etc/corosync/qdevice/net/nssdb/qdevice-net-node.p12

INFO: copy and import pk12 cert to all nodes

node 'VE-H310': Importing cluster certificate and key
node 'VE-H310': pk12util: PKCS12 IMPORT SUCCESSFUL
node 'VE-SSD': Importing cluster certificate and key
node 'VE-SSD': pk12util: PKCS12 IMPORT SUCCESSFUL
INFO: add QDevice to cluster configuration

INFO: start and enable corosync qdevice daemon on node 'VE-H310'...
Synchronizing state of corosync-qdevice.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable corosync-qdevice
Created symlink /etc/systemd/system/multi-user.target.wants/corosync-qdevice.service -> /lib/systemd/system/corosync-qdevice.service.

INFO: start and enable corosync qdevice daemon on node 'VE-SSD'...
Synchronizing state of corosync-qdevice.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable corosync-qdevice
Created symlink /etc/systemd/system/multi-user.target.wants/corosync-qdevice.service -> /lib/systemd/system/corosync-qdevice.service.
Reloading corosync.conf...
Done
root@VE-SSD:~# pvecm status
Quorum information
------------------
Date:             Wed Oct 30 01:36:29 2019
Quorum provider:  corosync_votequorum
Nodes:            2
Node ID:          0x00000002
Ring ID:          1/16
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      3
Quorum:           2
Flags:            Quorate Qdevice

Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000001          1    A,V,NMW 192.168.255.151
0x00000002          1    A,V,NMW 192.168.255.150 (local)
0x00000000          1            Qdevice
root@VE-SSD:~#

I would recommend we add a note to documentation about this issue, I was able to find a fix in [1] https://bugs.launchpad.net/ubuntu/+source/corosync-qdevice/+bug/1809682
 
  • Like
Reactions: idic
You must log in or register to reply here.
Top Bottom