2025 / PVE9.x / WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! [analysis, resolution]

Aug 27, 2021
2
0
21
I got tired of running into this issue on every PVE version. I spent some time deep diving to get this to reproduce consistently.

Related threads:
  1. 2025 - Enable RSA key seems to fix
  2. 2023 - SSH Host key ceritificates
  3. 2022 - Remote host identification has changed
Test Setup:
  • PVE 9.0 / 3 node cluster - Fully updated as of 2025-10-20.
  • Default configuration after installation.
I have no longer seen these warnings and can now consistently - on demand - generate this warning message (even with my previous post in threads above I did start recieving those error message again). I have taken to the following steps on a cluster buildout to be sure I can restore to a known state if needed:
  • Backup /root, /etc/ssh for each node after initial creation.
  • Backup /etc/pve/priv (has additional SSH material - likely for WebUI and cluster).
  • Do not set HostKeyAlgorithms, ProxyCommand in ssh_config
  • Set UpdateHostKeys no in ssh_config

Major results

Explicit HostKeyAlgorithms Definitions Break WebUI (ssh_config)

The WebUI overrides (or injects) additional SSH settings for intra-cluster SSH sessions - causing the dreaded 'WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!' message.​

Workaround: Remove HostKeyAlgorithms in /etc/ssh/ssh_config.

Analysis:​
HostKeyAlgorithms is not explicitly set in /etc/ssh/ssh_config. Implicit configuration ssh -G test ssh -G {OTHER_NODE} provides the following supported list (which are known Debian Trixie defaults):​
Code:
ssh-ed25519-cert-v01@openssh.comecdsa-sha2-nistp256-cert-v01@openssh.com[/INDENT]
[INDENT=3]ecdsa-sha2-nistp384-cert-v01@openssh.com[/INDENT]
[INDENT=3]ecdsa-sha2-nistp521-cert-v01@openssh.com[/INDENT]
[INDENT=3]sk-ssh-ed25519-cert-v01@openssh.com[/INDENT]
[INDENT=3]sk-ecdsa-sha2-nistp256-cert-v01@openssh.com[/INDENT]
[INDENT=3]rsa-sha2-512-cert-v01@openssh.com[/INDENT]
[INDENT=3]rsa-sha2-256-cert-v01@openssh.com[/INDENT]
[INDENT=3]ssh-ed25519,ecdsa-sha2-nistp256[/INDENT]
[INDENT=3]ecdsa-sha2-nistp384[/INDENT]
[INDENT=3]ecdsa-sha2-nistp521[/INDENT]
[INDENT=3]sk-ssh-ed25519@openssh.com[/INDENT]
[INDENT=3]sk-ecdsa-sha2-nistp256@openssh.com[/INDENT]
[INDENT=3]rsa-sha2-512[/INDENT]
[INDENT=3]rsa-sha2-256

Explicitly setting HostKeyAlgorithms in /etc/ssh/ssh_config - even to the default values - result in the following behaviors:​
  1. Console / SSH sessions to other cluster nodes work as expected.
  2. WebUI SSH sessions to other cluster nodes break with 'remote host identification changed' message.

Explicit UpdateHostKeys Definitions Break WebUI (ssh_config)
Enabling this option in /etc/ssh/ssh_config and attempting WebUI SSH sessions to other cluster nodes result in 'link error unable to access /etc/ssh/ssh_known_hosts)'.​
Workaround: Explicitly set UpdateHostKeys no in /etc/ssh/ssh_config.​

Console / SSH connections do not seem affected. Again, this looks like the WebUI overrides (or injects) additional SSH settings for intra-cluster SSH sessions.​

Explicit ProxyCommand Definitions Break WebUI (ssh_config)
Enabling this option in /etc/ssh/ssh_config and attempting WebUI SSH sessions to other cluster nodes result in hung SSH session connections.​
Workaround: Remove ProxyCommand in /etc/ssh/ssh_config.​

Suspect WebUI uses this for SSH connections - I have not tested this enough other than knowing that is causes issues and warrants further investigation.​

Minor results

/etc/ssh/sshd_config - Compression delayed - Deprecated value. Use Compression yes
/etc/ssh/sshd_config - # KerberosGetAFSToken no - Remove or never enable. Debian binary does not suppport.
/etc/ssh/sshd_config - GSSAPIAuthentication yes - Doesn't hurt to enable, but disabled in sshd_config. Recommend GSSAPIAuthentication no
/root/.ssh/config - Root user overrides Ciphers with Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com
/etc/ssh/ssh_config,sshd_config - Maximum value for RequiredRSASize is 2048 - /etc/pve/priv/authkey.key is a 2048bit RSA key
 
Last edited:
Thank you for your analysis and workaround.

I got tired of running into this issue on every PVE version.
Ehhh ... what? I'm installing PVE weekly for at least one decade and never run into this problem and I assume most users also do NOT run into this.
Unless you're changing stuff manually, you will not run into this. There are also other security precautions to limit PVE node SSH exposure if you do not want to mess with the SSH settings, namely proper network seperation and firewalling.

Just to add to your workaround, you could also explicitly set the settings for the PVE cluster nodes, so that you will not mess with the default settings for your nodes but can have anything you want to other nodes if you use e.g. Match Address section in your SSH config.
 
Sorry, but for me it's unclear what the problem is, how to reproduce / when does it happen, and what's different in your setup from a standard PVE installation were there's no issue like those you seem to describe. IIUC, you use/need some custom SSH settings that do not seem compatible with PVE internal use of SSH for some cluster operations. IMHO, you should explain your use case as a bug report[1] so devs can evaluate if/how to support those configs.

[1] https://bugzilla.proxmox.com/describecomponents.cgi?product=pve