Permission denied (publickey) after nodes joined cluster

dukotron

New Member
Nov 25, 2023
3
0
1
Hi all, I have a strange issue that I am trying to get to the bottom of when it comes to proxmox clusters and SSH.

I created a cluster using pvecm create <clustername> . Everything works well with that step. The changes are reflected in the proxmox management UI as well. Following this, I used pvecm add <nodeip> where nodeip is the ip of the node on which I created the cluster, I ran the commands on the two nodes that need to join the cluster. I enter the root password, I enter yes to confirm, the node joins the cluster. This went without issues, and again everything looks in-order on all the nodes.

Now the issue is, for the two nodes that joined the cluster, after joining I am unable to SSH into them from my personal computer that I use to manage the servers. I get permission denied (publickey). I have debuged for a few hours, but could not figure out why this is the case. The sshd configs look good (and unchanged), the authorized_keys file also looks correct. There are the three keys related to the nodes in the cluster, and the key used by PC. I have also compared the configurations between node 1 and the two joined nodes, and I can't find a difference there either. But the unusual thing is, I can still SSH into node 1 fine.

In theory, I could ssh into node 1, and from that node ssh into nodes 2 and 3, but that is a hassle that shouldn't be necessary. Does anyone have any ideas?
 
Hi,

There is nothing in the syslog/journalctl?


Could you please run `pvecm updatecerts -f` command on node 1?
 
  • There is nothing in the syslog/journalctl?
    • Sadly not, the only log / output I get is: sshd[151517]: Connection closed by authenticating user root 10.183.2.6 port 51036 [preauth]
    • Previously, I did also try to do ssh -vvv to gain some insights, but it does not say much more than permission denied (public key), but in a longer format.
  • Could you run pvecm updatecerts -fon node 1
    • Just tried it, it runs successfully but there is no change in terms of SSH auth.
 
Thank you!

Now I would compare the sshd_config file on all nodes:
Bash:
cat /etc/ssh/sshd_config  | egrep -iv "#"