QDevice Setup still harder than necessary :(

[MrPete]

I have succeeded in getting a Raspberry Pi QDevice running, but tore out too much hair along the way.
Starting Point

• Two Proxmox 7.1 nodes in a cluster
• Brand new Raspberry Pi Zero W (hardwired ethernet)

Issue One: Software not installing

• Resolution: Keep at it. Retry, reboot, etc. Clean out any partly-installed packages. Try again.

Issue Two: SSH keys

• I kept getting complaints about SSH keys not working or having been changed
• Resolution:
  • On every box, ensure /etc/hosts has correct entries for self and others
  • Delete stored keys in ~/.ssh/authorized_keys, ~/.ssh/known_hosts, /etc/ssh/ssh_known_hosts (the last one doesn't exist on Rasp Pi)
  • From each box, ssh to the others. approve connections. It should store keys. Exit and try again. Not done until you can ssh login without password

Issue Three: Stored Certs.

• Database errors, or cert problems of various kinds
• Resolution:
  • On the proxmox boxes, rm -r /etc/corosync/qdevice/net/nssdb/

After four days of frustration, it finally worked!

 

[oguz]

hi,

/etc/ssh/ssh_known_hosts (the last one doesn't exist on Rasp Pi)

that only exists when you use that box to ssh to somewhere

I kept getting complaints about SSH keys not working or having been changed

most likely it was an unrelated issue from before?

From each box, ssh to the others. approve connections. It should store keys. Exit and try again. Not done until you can ssh login without password

this is true for any PVE cluster. without your nodes being able to SSH to each other as root (no password and no interaction), your cluster won't work correctly.

On the proxmox boxes, rm -r /etc/corosync/qdevice/net/nssdb/

or pvecm qdevice remove (should wipe the whole /etc/corosync/qdevice dir)

On every box, ensure /etc/hosts has correct entries for self and others

actually not necessary, since you can use the IP address too.

Raspberry Pi Zero W

also make sure this is running the latest bullseye version available (to avoid package version conflicts between corosync-qnetd and corosync-qdevice)

but from my short test now, with a cleanly installed cluster and clean install of debian 11, i didn't have any problems setting up the qdevice for the cluster following the instructions on the wiki [0]

only thing that might be worth noting: edit /etc/ssh/sshd_config on your qdevice before the setup to allow PermitRootLogin yes to be able to ssh with a password.

hope this helps!

[0]: https://pve.proxmox.com/pve-docs/chapter-pvecm.html#_qdevice_net_setup