proxmox 5.4 passwordless ssh does not work

[bassplayer1]

I created 3 brand new ProxMox 5.4 nodes, clustered them together and the passwordless ssh does not work when ssh'ing from one to another, or viewing the shell from the ProxMox GUI console.

Is this a bug? I did not have to do anything to the keys to get password'less SSH in ProxMox 5.3

Note, I have created a vlan between hosts and am using this rather the management IP that I set ProxMox up with, and I have changed the hosts file in each node to point to this.
I did identical steps in ProxMox 5.3 and did not see any abnormal behavior.

They can all SSH, just the keys do not work anymore after clustering.
You can see other settings on the hosts, so the cluster is working.

Thanks

 

[mira]

Please post your /etc/hosts file and the content of '/etc/pve/.members'.

 

[bassplayer1]

Here it is with just 2 nodes connected:

root@proxmox-02:~# cat /etc/pve/.members
{
"nodename": "proxmox-02",
"version": 4,
"cluster": { "name": "mycluster", "version": 2, "nodes": 2, "quorate": 1 },
"nodelist": {
"proxmox-01": { "id": 1, "online": 1, "ip": "192.168.0.1"},
"proxmox-02": { "id": 2, "online": 1, "ip": "192.168.0.2"}
}
}
------------------------------------------------------------------------------------
127.0.0.1 localhost.localdomain localhost
192.168.0.1 proxmox-01.myplatform.home proxmox-01
192.168.0.2 proxmox-02.myplatform.home proxmox-02

10.0.0.1 pve-01.myplatform.home pve-01
10.0.0.2 pve-02.myplatform.home pve-02
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
-----------------------------------------------------------------------------------

 

[mira]

Please try to connect from node 1 to 2 and 2 to 1 with 'ssh -vv' and post the output. In addition make sure your keys are in the '/etc/pve/priv/authorized_keys' file.

 

[bassplayer1]

here is the output:

root@proxmox-01:~# ssh -vv proxmox-02
OpenSSH_7.4p1 Debian-10+deb9u6, OpenSSL 1.0.2r 26 Feb 2019
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "proxmox-02" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to proxmox-02 [192.168.0.2] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
key_load_public: invalid format
debug1: identity file /root/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u6
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4p1 Debia n-10+deb9u6
debug1: match: OpenSSH_7.4p1 Debian-10+deb9u6 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to proxmox-02:22 as 'root'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2 -nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sh a256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman- group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ex t-info-c
debug2: host key algorithms: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2- 256,ssh-rsa,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v0 1@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@open ssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256 -ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256- cbc
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256 -ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256- cbc
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-25 6-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-6 4@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-25 6-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-6 4@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2 -nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sh a256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman- group14-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp2 56,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256 -ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256 -ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-25 6-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-6 4@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-25 6-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-6 4@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit > compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit > compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:2dfHNKrqePLq+E4EogKsmtDlRUZYKX5RORUUfNIg n6I
debug1: Host 'proxmox-02' is known and matches the RSA host key.
debug1: Found key in /etc/ssh/ssh_known_hosts:4
debug2: set_newkeys: mode 1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 134217728 blocks
debug2: key: /root/.ssh/id_rsa ((nil))
debug2: key: /root/.ssh/id_dsa ((nil))
debug2: key: /root/.ssh/id_ecdsa ((nil))
debug2: key: /root/.ssh/id_ed25519 ((nil))
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-s ha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/id_rsa
Enter passphrase for key '/root/.ssh/id_rsa':
---------------------------------------------------------------------------------------

My /etc/pve/priv/authorized_keys file is blank on the ProxMox 5.4 servers, but there is data in the file on my cluster running 5.3.
I never added anything to that file in the working 5.3 cluster, is this something that should be autopopulated during the clustering process?

 

[Stoiko Ivanov]

debug1: Trying private key: /root/.ssh/id_rsa
Enter passphrase for key '/root/.ssh/id_rsa':

seem the key '/root/.ssh/id_rsa' is password protected? PVE uses this key (and it needs to be without passphrase in order to be able to automatically connect to the other boxes).

Since it's asking for the passphrase this should imply that the corresponding public key is in /root/.ssh/authorized_keys on the other node - however in PVE's standard setup this is a symlink to /etc/pve/priv/authorized_keys (in order to keep the same set of keys for all nodes)

Did you change anything w.r.t. ssh-keys or ssh-setup?

 

[bassplayer1]

I didnt make any changes, however i think my problem has turned out to be something to do with cloning the nodes in virtualbox (proxmox is running in virtualbox). Maybe something to do with them all having the same keys?

 

[Stoiko Ivanov]

Maybe something to do with them all having the same keys?

sounds unlikely.
The passphrase is asked for the key /root/.ssh/id_rsa.
this key has a passphrase, but PVE needs a key without passphrase.

How did you create the cluster? (pvecm and the GUI usually take care of this)!

 

[bassplayer1]

I tried it both ways, with pvecm and the GUI using the assisted join, and got the same result.
My process was as follows:
1.create a vm in virtualbox, with 2 interfaces, one Nat(for the managment interface) and the other as internal network (for the proxmox vms to communicate over).
2.Install proxmox on the VM.
3.clone the VM in virtualbox, so i now have 2 proxmox hosts.
4.change the hostname for the new vm to make it different to the first host
5.add a vlan to /etc/network/interfaces for both proxmox hosts
6.add a virtual bridge on each host and connect the vlan so that the proxmox hosts can talk to each other over it
7.Change the hosts file in each proxmox host to point the main hostname to the new bridge ip address, so that when i create the cluster that ip will be used instead of the management ip.
8.ping test the hostnames
7. Create a cluster via the gui in one host.
8. Join the other host via the gui assist.
9.i can now access both hosts via either hosts' gui, but ssh does not auto logon (regardless of if i use the gui or just ssh using putty from my desktop

 

[Stoiko Ivanov]

Please remove /root/.ssh/id_rsa and /root/.ssh/id_rsa.pub before cloning the machines and installing PVE

 

[cvhideki]

Hi all,
I have same problem but can't upgrade system with scripts pve5to6

FAIL: Unsupported SSH Cipher configured for root in /root/.ssh/config: 3des

my file /root/.ssh/config

Ciphers blowfish-cbc,aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc

 

[Stoiko Ivanov]

my file /root/.ssh/config

do you have a particular reason for keeping this cipherlist in your /root/.ssh/config?

else consider removing it (or at least the Ciphers line) - the shipped defaults are usually quite ok (and safer since they don't contain 3des)

I hope this helps!

 

[cvhideki]

Thanks for helping,
after removing the content in file /root/.ssh/config, I have a problem opening the console VM in web browser.
Later I found information about that file, and changed the contents to next

Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com