[SOLVED] LXC SSH issue after reinstalling a node and using same hostname and IP

[hellotom]

Hi all,

Medium time lurker first time poster - and Proxmox newbie, so apologies in advance for the gaps in understanding…

I have been establishing my small Proxmox cluster of three nodes, which has involved a couple of reinstalls to fix silly things I didn’t think about, like hostname, IP, etc. Most recently, I reinstalled Proxmox on a node that has a single disk so I could use ZFS (and try out replication, possibly HA), and for the first time after a reinstall added the node back to the cluster with the same hostname and IP address as before. After getting everything back up and running again I noticed that I can no longer use the console to access any of the LXCs on one node - the most recently reinstalled one. I’m getting this error message:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:4Wt/PzlN+qS4BccSlPBpYuu72SC/vY9KCauGD5Sf2yI.
Please contact your system administrator.
Add correct host key in /root/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /root/.ssh/known_hosts:3
remove with:
ssh-keygen -f "/root/.ssh/known_hosts" -R "192.168.0.20"
Host key for 192.168.0.20 has changed and you have requested strict checking.
Host key verification failed.

I searched for the answer and found a lot of posts where people confidently posted that I simply needed to run the code listed in the message (ssh-keygen -f "/root/.ssh/known_hosts" -R "192.168.0.20"), or try pvecm updatecerts (possibly with —-force), but those didn’t work for me. As I got deeper into the rabbit hole looking for a solution, I found suggestions to apply patches, disable host key checking, and more. I started to worry that I was going to do more harm than good following these riskier solutions without understanding what they were doing/why.

I also saw there there was a bugfix/improvement that seems to be related to this in PVE 8.2, which is more recent than almost all of the posts offering fixes. All three of my nodes are running PVE 8.2, but all of them were installed from live media that predates 8.2, and then updated.

Is this error related to the SSH issue that was fixed in 8.2 - caused by reinstalling a node and using the same hostname and IP address? How can I fix it? Could I have made things worse by running the ssh-keygen and pve updatecerts commands already?

Thanks in advance for any advice

 

[Kingneutron]

ssh-keygen -R 192.168.0.20

Should absolutely work, I've done it countless times. You need to do it on the PC you're ssh'ing from

And you shouldn't be ssh'ing as the root id if you can at all help it, unless you're doing it for backups. Adduser for non-root ssh and use that, it's better for security.

 

[hellotom]

ssh-keygen -R 192.168.0.20

Should absolutely work, I've done it countless times. You need to do it on the PC you're ssh'ing from

And you shouldn't be ssh'ing as the root id if you can at all help it, unless you're doing it for backups. Adduser for non-root ssh and use that, it's better for security.

I worked this out myself probably just as you were writing this message, and I'm back in! I wondered if it would only be a temporary fix, but I'm glad to hear I somehow stumbled into the right answer. Thanks so much for replying - the confirmation of why it worked is extremely helpful.

I think because I control all three nodes by connecting to proxmox.domain.tld (via a reverse proxy), I had forgotten that the reverse proxy connects to a specific node - and not the one with the issue. Running ssh-keygen -f "/root/.ssh/known_hosts" -R "192.168.0.20" on the node that the reverse proxy connects to solved it (and probably the shorter version you posted would have too).

Thanks for the reminder about root - I know that I shouldn't be using it as my main account but had rationalised that I was still getting everything set up... I'll set up a proper user account now.