Proxmox 8.2.2 - Qdevice

S

Stumpy6891

New Member
Apr 27, 2024
5
0
1
Afternoon all

First time post so be kind please :)

Using proxmox since 7.4 and used a qdevice with no issues upto a mess up on my part needing a complete reinstall of 2 nodes

Im currently on 8.2.2 and for love nor money can get my qdevice to work (everything set up on Pi4 iv even removed & reinstalled on the Pi)

everything correct on PM too any ideas please before i loose more of my ever loosing hair line here is the logs

pvecm qdevice setup 192.168.68.65
/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '192.168.68.65 (192.168.68.65)' can't be established.
ED25519 key fingerprint is SHA256:u4xo847nrpbVZE4d05/pKbs8gBWtB6OE8ZIXpnyaKnE.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.68.65's password:

Number of key(s) added: 1

Now try logging into the machine, with: "ssh 'root@192.168.68.65'"
and check to make sure that only the key(s) you wanted were added.


INFO: initializing qnetd server
Certificate database (/etc/corosync/qnetd/nssdb) already exists. Delete it to initialize new db

INFO: copying CA cert and initializing on all nodes

node 'ProxmoxDell': Creating /etc/corosync/qdevice/net/nssdb
password file contains no data
node 'ProxmoxDell': Creating new key and cert db
node 'ProxmoxDell': Creating new noise file /etc/corosync/qdevice/net/nssdb/noise.txt
node 'ProxmoxDell': Importing CAHost key verification failed.

INFO: generating cert request
Creating new certificate request


Generating key. This may take a few moments...

Certificate request stored in /etc/corosync/qdevice/net/nssdb/qdevice-net-node.crq

INFO: copying exported cert request to qnetd server

INFO: sign and export cluster cert
Signing cluster certificate
Certificate stored in /etc/corosync/qnetd/nssdb/cluster-Olympic-Cluster.crt

INFO: copy exported CRT

INFO: import certificate
Importing signed cluster certificate
Notice: Trust flag u is set automatically if the private key is present.
pk12util: PKCS12 EXPORT SUCCESSFUL
Certificate stored in /etc/corosync/qdevice/net/nssdb/qdevice-net-node.p12

INFO: copy and import pk12 cert to all nodes

node 'ProxmoxDell': Importing cluster certificate and key
node 'ProxmoxDell': pk12util: PKCS12 IMPORT SUCCESSFULHost key verification failed.
command 'ssh -o 'BatchMode=yes' -lroot 192.168.68.68 corosync-qdevice-net-certutil -m -c /etc/pve/qdevice-net-node.p12' failed: exit code 255
root@ProxmoxDell:~# pvecm status
Cluster information
-------------------
Name: Olympic-Cluster
Config Version: 2
Transport: knet
Secure auth: on

Quorum information
------------------
Date: Sat Apr 27 14:56:32 2024
Quorum provider: corosync_votequorum
Nodes: 2
Node ID: 0x00000001
Ring ID: 1.9
Quorate: Yes

Votequorum information
----------------------
Expected votes: 2
Highest expected: 2
Total votes: 2
Quorum: 2
Flags: Quorate

Membership information
----------------------
Nodeid Votes Name
0x00000001 1 192.168.68.67 (local)
0x00000002 1 192.168.68.68
root@ProxmoxDell:~#

If this has been resolved with 8.X.X then please point me in the right direction

Many Thanks
 
I have exactly the same error on version 8.2.2 in the same configuration. I also updated the certificates, nothing changed. Did you find any answer?
 
Hi,

I'm having the same problem with 8.2.2
when I try to set it up on node 1 i get:
Code: 
pvecm qdevice setup 10.1.1.14 --force
....
INFO: copy and import pk12 cert to all nodes

node 'pve1': Importing cluster certificate and key
node 'pve1': pk12util: PKCS12 IMPORT SUCCESSFULHost key verification failed.
command 'ssh -o 'BatchMode=yes' -lroot 10.1.1.18 corosync-qdevice-net-certutil -m -c /etc/pve/qdevice-net-node.p12' failed: exit code 255

I tried then to set it up on the second node (as I understand its not important from which node you set up the pvecm qdevice) I get:

Code: 
pvecm qdevice setup 10.1.1.14  --force
/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/bin/ssh-copy-id: WARNING: All keys were skipped because they already exist on the remote system.
                (if you think this is a mistake, you may want to use -f option)


INFO: initializing qnetd server
Certificate database (/etc/corosync/qnetd/nssdb) already exists. Delete it to initialize new db

INFO: copying CA cert and initializing on all nodes
Host key verification failed.
Host key verification failed.

INFO: generating cert request
Certificate database doesn't exists. Use /sbin/corosync-qdevice-net-certutil -i to create it
command 'corosync-qdevice-net-certutil -r -n Cluster1' failed: exit code 1

I tried ' pvecm updatecerts' on both nodes and tested ssh from both nodes to the QDevice-Host and ssh between both nodes.

"pvecm qdevice remove" is not possible because the device is not yet successfully added.
 
Last edited:
This does work in 8.2.2
JasonNeumann said:
This didn't work for me. If it works for anyone else please let me know. Thanks.
Click to expand...


This actually works for me too in Proxmox 8.2.2. I may have missed an ssh connection on my first attempt. Below are the commands I used. This assumes that you've enabled root login on your Qdevice, which is a requirement. When done, use the pvecm status command to verify all the quorum votes are available to the cluster.

Example:

First, on the Qdevice edit /etc/ssh/sshd_config adding PermitRootLogin yes to the config file. Save and close the file. Reboot or restart sshd, using systemctl restart sshd.

Next, create a 2 node cluster using pve1 and pve2.

Finally, install the qdevice software, then ssh and login as root between all your devices and run pvecm qdevice setup from one of your Proxmox nodes, like so....

Qdevice (debian Linux):
apt install corosync-qdevice corosync-qnetd
ssh root@<pve1 IP>
ssh root@<pve2 IP>

Proxmox pve1:
apt install corosync-qdevice
ssh root@<pve2 IP>
ssh root@<qdevice IP>

Proxmox pve2:
apt install corosync-qdevice
ssh root@<pve1 IP>
ssh root@<qdevice IP>
pvecm qdevice setup <qdevice IP> -f
pvecm status

Good luck everyone.
 
Last edited:
JasonNeumann said:
This does work in 8.2.2



This actually works for me too in Proxmox 8.2.2. I may have missed an ssh connection on my first attempt. Below are the commands I used. This assumes that you've enabled root login on your Qdevice, which is a requirement. When done, use the pvecm status command to verify all the quorum votes are available to the cluster.

...

Good luck everyone.
Click to expand...
Hi!
Sorry, I tried this yesterday and it's not working for me :(

I always get error that corosync-qdevice-net-certutil can't open certificate file.

Best regards
 
carles89 said:
Hi!

There's an upcoming patch: https://bugzilla.proxmox.com/
Sejl said:
Hi!ther thing
Sorry, I tried this yesterday and it's not working for me :(

I always get error that corosync-qdevice-net-certutil can't open certificate file.

Best regards
Click to expand...
It has been reported that this is a bug that's being fixed. Why it finally worked for me is unknown. I did subscribe to the non-subscription repository and updated my PVEs before running the commands.
Click to expand...
 
  • Like
Reactions: Sejl
Last edited:
  • Like
Reactions: carles89
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back