[SOLVED] Connection closed by IP port xxxx [preauth]

D

Deleted member 33567

Guest
Hi,

This is the weirdest experience we had with SSH server.

We run a 3 nodes cluster with CEPH.

Recently we have noticed that we are unable to use Web VNC for 2 of the nodes, upon some reading online I have restarted the pve-manager and it seems to help with the Web VNC console.

Now the weird part is that we do not run a firewall at nodes or datacenter level.

And we receive this back for every connection attempt.

Here is my local -vvv ssh debug:

Code: 
Revenger:~ xxxxxxxxx$ ssh nu1 -vvv  
OpenSSH_7.7p1, LibreSSL 2.7.3
debug1: Reading configuration data /Users/xxxxxxxxx/.ssh/config
debug1: /Users/xxxxxxxxx/.ssh/config line 1: Applying options for *
debug1: /Users/xxxxxxxxx/.ssh/config line 131: Applying options for nu1
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: /etc/ssh/ssh_config line 52: Applying options for *
debug2: resolve_canonicalize: hostname xxx.xxx.xxx.xxx is address
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to xxx.xxx.xxx.xxx [xxx.xxx.xxx.xxx] port 22.
debug1: Connection established.
debug1: identity file /Users/xxxxxxxxx/.ssh/id_rsa type 0
debug1: key_load_public: No such file or directory
debug1: identity file /Users/xxxxxxxxx/.ssh/id_rsa-cert type -1
debug1: identity file /Users/xxxxxxxxx/.ssh/id_ed25519 type 3
debug1: key_load_public: No such file or directory
debug1: identity file /Users/xxxxxxxxx/.ssh/id_ed25519-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.7
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4p1 Debian-10+deb9u4
debug1: match: OpenSSH_7.4p1 Debian-10+deb9u4 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to xxx.xxx.xxx.xxx:22 as 'root'
debug3: hostkeys_foreach: reading file "/Users/xxxxxxxxx/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts:5
debug3: load_hostkeys: loaded 1 keys from xxx.xxx.xxx.xxx
debug3: hostkeys_foreach: reading file "/Users/xxxxxxxxx/.ssh/known_hosts2"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts2:5
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts2:20
debug3: load_hostkeys: loaded 2 keys from xxx.xxx.xxx.xxx
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:tgY+lqxka/pH5oTZijsoHi695F+B3b9QCBwG2lOkT3E
debug3: hostkeys_foreach: reading file "/Users/xxxxxxxxx/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts:5
debug3: load_hostkeys: loaded 1 keys from xxx.xxx.xxx.xxx
debug3: hostkeys_foreach: reading file "/Users/xxxxxxxxx/.ssh/known_hosts2"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts2:5
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts2:20
debug3: load_hostkeys: loaded 2 keys from xxx.xxx.xxx.xxx
debug1: Host 'xxx.xxx.xxx.xxx' is known and matches the ECDSA host key.
debug1: Found key in /Users/xxxxxxxxx/.ssh/known_hosts:5
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 134217728 blocks
debug2: key: /Users/xxxxxxxxx/.ssh/id_rsa (0x7fded4505710), explicit
debug2: key: /Users/xxxxxxxxx/.ssh/id_ed25519 (0x7fded4505780), explicit
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: RSA SHA256:AbaZnfke6D4XMpr4W7xelETcI+vmxTl2eGB7x2Fdobc /Users/xxxxxxxxx/.ssh/id_rsa
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug1: Offering public key: ED25519 SHA256:kothDQAB2DSwlm20VcJfy6Fdx12uiRWakIobjM1E+OQ /Users/xxxxxxxxx/.ssh/id_ed25519
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
root@xxx.xxx.xxx.xxx: Permission denied (publickey).
Revenger:~ xxxxxxxxx$ ssh nu1 -vvv
OpenSSH_7.7p1, LibreSSL 2.7.3
debug1: Reading configuration data /Users/xxxxxxxxx/.ssh/config
debug1: /Users/xxxxxxxxx/.ssh/config line 1: Applying options for *
debug1: /Users/xxxxxxxxx/.ssh/config line 131: Applying options for nu1
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: /etc/ssh/ssh_config line 52: Applying options for *
debug2: resolve_canonicalize: hostname xxx.xxx.xxx.xxx is address
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to xxx.xxx.xxx.xxx [xxx.xxx.xxx.xxx] port 22.
debug1: Connection established.
debug1: identity file /Users/xxxxxxxxx/.ssh/id_rsa type 0
debug1: key_load_public: No such file or directory
debug1: identity file /Users/xxxxxxxxx/.ssh/id_rsa-cert type -1
debug1: identity file /Users/xxxxxxxxx/.ssh/id_ed25519 type 3
debug1: key_load_public: No such file or directory
debug1: identity file /Users/xxxxxxxxx/.ssh/id_ed25519-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.7
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4p1 Debian-10+deb9u4
debug1: match: OpenSSH_7.4p1 Debian-10+deb9u4 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to xxx.xxx.xxx.xxx:22 as 'root'
debug3: hostkeys_foreach: reading file "/Users/xxxxxxxxx/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts:5
debug3: load_hostkeys: loaded 1 keys from xxx.xxx.xxx.xxx
debug3: hostkeys_foreach: reading file "/Users/xxxxxxxxx/.ssh/known_hosts2"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts2:5
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts2:20
debug3: load_hostkeys: loaded 2 keys from xxx.xxx.xxx.xxx
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:tgY+lqxka/pH5oTZijsoHi695F+B3b9QCBwG2lOkT3E
debug3: hostkeys_foreach: reading file "/Users/xxxxxxxxx/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts:5
debug3: load_hostkeys: loaded 1 keys from xxx.xxx.xxx.xxx
debug3: hostkeys_foreach: reading file "/Users/xxxxxxxxx/.ssh/known_hosts2"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts2:5
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts2:20
debug3: load_hostkeys: loaded 2 keys from xxx.xxx.xxx.xxx
debug1: Host 'xxx.xxx.xxx.xxx' is known and matches the ECDSA host key.
debug1: Found key in /Users/xxxxxxxxx/.ssh/known_hosts:5
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 134217728 blocks
debug2: key: /Users/xxxxxxxxx/.ssh/id_rsa (0x7fb4cfe00610), explicit
debug2: key: /Users/xxxxxxxxx/.ssh/id_ed25519 (0x7fb4cfc007b0), explicit
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: RSA SHA256:AbaZnfke6D4XMpr4W7xelETcI+vmxTl2eGB7x2Fdobc /Users/xxxxxxxxx/.ssh/id_rsa
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug1: Offering public key: ED25519 SHA256:kothDQAB2DSwlm20VcJfy6Fdx12uiRWakIobjM1E+OQ /Users/xxxxxxxxx/.ssh/id_ed25519
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
root@xxx.xxx.xxx.xxx: Permission denied (publickey).
Revenger:~ xxxxxxxxx$ ssh nu1 -vvv
OpenSSH_7.7p1, LibreSSL 2.7.3
debug1: Reading configuration data /Users/xxxxxxxxx/.ssh/config
debug1: /Users/xxxxxxxxx/.ssh/config line 1: Applying options for *
debug1: /Users/xxxxxxxxx/.ssh/config line 131: Applying options for nu1
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: /etc/ssh/ssh_config line 52: Applying options for *
debug2: resolve_canonicalize: hostname xxx.xxx.xxx.xxx is address
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to xxx.xxx.xxx.xxx [xxx.xxx.xxx.xxx] port 22.
debug1: Connection established.
debug1: identity file /Users/xxxxxxxxx/.ssh/id_rsa type 0
debug1: key_load_public: No such file or directory
debug1: identity file /Users/xxxxxxxxx/.ssh/id_rsa-cert type -1
debug1: identity file /Users/xxxxxxxxx/.ssh/id_ed25519 type 3
debug1: key_load_public: No such file or directory
debug1: identity file /Users/xxxxxxxxx/.ssh/id_ed25519-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.7
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4p1 Debian-10+deb9u4
debug1: match: OpenSSH_7.4p1 Debian-10+deb9u4 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to xxx.xxx.xxx.xxx:22 as 'root'
debug3: hostkeys_foreach: reading file "/Users/xxxxxxxxx/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts:5
debug3: load_hostkeys: loaded 1 keys from xxx.xxx.xxx.xxx
debug3: hostkeys_foreach: reading file "/Users/xxxxxxxxx/.ssh/known_hosts2"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts2:5
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts2:20
debug3: load_hostkeys: loaded 2 keys from xxx.xxx.xxx.xxx
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:tgY+lqxka/pH5oTZijsoHi695F+B3b9QCBwG2lOkT3E
debug3: hostkeys_foreach: reading file "/Users/xxxxxxxxx/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts:5
debug3: load_hostkeys: loaded 1 keys from xxx.xxx.xxx.xxx
debug3: hostkeys_foreach: reading file "/Users/xxxxxxxxx/.ssh/known_hosts2"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts2:5
debug3: record_hostkey: found key type ECDSA in file /Users/xxxxxxxxx/.ssh/known_hosts2:20
debug3: load_hostkeys: loaded 2 keys from xxx.xxx.xxx.xxx
debug1: Host 'xxx.xxx.xxx.xxx' is known and matches the ECDSA host key.
debug1: Found key in /Users/xxxxxxxxx/.ssh/known_hosts:5
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 134217728 blocks
debug2: key: /Users/xxxxxxxxx/.ssh/id_rsa (0x7fcfad904fc0), explicit
debug2: key: /Users/xxxxxxxxx/.ssh/id_ed25519 (0x7fcfad905030), explicit
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: RSA SHA256:AbaZnfke6D4XMpr4W7xelETcI+vmxTl2eGB7x2Fdobc /Users/xxxxxxxxx/.ssh/id_rsa
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug1: Offering public key: ED25519 SHA256:kothDQAB2DSwlm20VcJfy6Fdx12uiRWakIobjM1E+OQ /Users/xxxxxxxxx/.ssh/id_ed25519
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
root@xxx.xxx.xxx.xxx: Permission denied (publickey).
Revenger:~ xxxxxxxxx$

Here is the reply from the server:

Code: 
Sep  3 00:01:58 node01-sxb-xxx sshd[1146516]: Connection closed by xxxx port 35567 [preauth]
Sep  3 00:01:59 node01-sxb-xxx sshd[1146524]: Connection closed by xxxx port 38491 [preauth]
Sep  3 00:02:01 node01-sxb-xxx sshd[1146526]: Connection closed by xxxx port 38784 [preauth]

Here is the SSH config, that was not touched or altered in past months, and the SSH auth was working just fine before.

Code: 
grep -vE "^#|^$" /etc/ssh/sshd_config
PermitRootLogin yes
PubkeyAuthentication yes
AuthorizedKeysFile      .ssh/authorized_keys .ssh/authorized_keys2
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM yes
X11Forwarding yes
PrintMotd no
AcceptEnv LANG LC_*
Subsystem       sftp    /usr/lib/openssh/sftp-serve

Would be really happy if anyone had seen this issues before and found a solution to share with.

Note: The IP and some details were SANITIZED by xxx remarks.

Best,
Cosmin
 
Seems both keys you're trying to use (id_rsa and id_ed25519) are not in '/root/.ssh/authorized_keys' file. Since PasswordAuthentication is disabled you don't get asked for a password.
 
Stoiko Ivanov said:
Seems both keys you're trying to use (id_rsa and id_ed25519) are not in '/root/.ssh/authorized_keys' file. Since PasswordAuthentication is disabled you don't get asked for a password.
Click to expand...

The thing is that the keys are there. That is what worries me after checking all settings the keys file was not altered in months. And the keys also did not changed on my side.
 
For example today, I try to shell from web console on node2 and I get:

Code: 
Permission denied (publickey).

Only node01 works on web shell, node2 and node3 see screenshoots.

After restart of pve-cluster I am able to do web shell to all 3 nodes, see screenshoots.

Major issues I had noticed, and I will repeat myself again to be clear:

- No changes were made to authorised_keys and all the keys are there.
- After 24 hours without any changes on nodes, the web shell will fail for node2 and node3 and will require a pve-cluster reboot in order to work.
- This does not change the fact that my keys are present on the server and I cannot login...

When I try to migrate I get:

Code: 
Permission denied (publickey).
TASK ERROR: command '/usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=node01-sxb-pve01' root@172.18.1.1 pvecm mtunnel -migration_network 172.18.0.0/16 -get_migration_ip' failed: exit code 255

Screen Shot 2018-09-04 at 03.12.26.png Screen Shot 2018-09-04 at 03.13.12.png Screen Shot 2018-09-04 at 03.13.04.png Screen Shot 2018-09-04 at 03.14.16.png Screen Shot 2018-09-04 at 03.14.25.png
 
Could be a problem with your cluster network:
* /root/.ssh/authorized_keys is/should be a symlink to /etc/pve/priv/authorized_keys
* /etc/pve is the proxmox cluster filesystem (pmxcfs)

If pmxcfs is not running on a node the authorized_keys file will not be accessible

Check your logs for problems with corosync or pve-cluster/pmxcfs
 
Stoiko Ivanov said:
Could be a problem with your cluster network:
* /root/.ssh/authorized_keys is/should be a symlink to /etc/pve/priv/authorized_keys
* /etc/pve is the proxmox cluster filesystem (pmxcfs)

If pmxcfs is not running on a node the authorized_keys file will not be accessible

Check your logs for problems with corosync or pve-cluster/pmxcfs
Click to expand...

* /root/.ssh/authorized_keys is/should be a symlink to /etc/pve/priv/authorized_keys The symlink is already there like I said no one altered any settings only upgrades were made regularly.

* /etc/pve is the proxmox cluster filesystem (pmxcfs)
have this in console:
pmxcfs[33229]: [status] notice: received log

Enabled
echo "1" >/etc/pve/.debug

Here is the debug log for a few seconds:

Code: 
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = priv new path = lock (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end priv/lock = 0x5597960fa500 (priv/lock) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr priv/lock (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr priv/lock (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /priv/lock (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /priv/lock/ha_manager_lock (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start priv/lock/ha_manager_lock (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug priv/lock/ha_manager_lock (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = priv new path = lock/ha_manager_lock (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end priv/lock/ha_manager_lock = 0x5597960fa500 (priv/lock/ha_manager_lock) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr priv/lock/ha_manager_lock (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr priv/lock/ha_manager_lock (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /priv/lock/ha_manager_lock (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_utimens /priv/lock/ha_manager_lock (pmxcfs.c:476:cfs_fuse_utimens)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start priv/lock/ha_manager_lock (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug priv/lock/ha_manager_lock (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = priv new path = lock/ha_manager_lock (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end priv/lock/ha_manager_lock = 0x5597960fa500 (priv/lock/ha_manager_lock) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_utimes priv/lock/ha_manager_lock (cfs-plug.c:388:cfs_plug_base_utimens)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: dfsm mode is 2 (dfsm.c:658:dfsm_cpg_deliver_callback)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: process message 7 (length = 58) (dcdb.c:806:dcdb_deliver)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: got valid unlock request message (dcdb.c:840:dcdb_deliver)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: leave dcdb_deliver (1) (dcdb.c:920:dcdb_deliver)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: dfsm mode is 2 (dfsm.c:658:dfsm_cpg_deliver_callback)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: process message 6 (length = 46) (dcdb.c:806:dcdb_deliver)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: dir is locked (memdb.c:917:memdb_mtime)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: leave dcdb_deliver (1) (dcdb.c:920:dcdb_deliver)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_utimens /priv/lock/ha_manager_lock (-13) (pmxcfs.c:490:cfs_fuse_utimens)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process msg:1, size:16 (server.c:168:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process result 0 (server.c:318:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process msg:1, size:16 (server.c:168:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process result 0 (server.c:318:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process msg:5, size:528 (server.c:168:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process result 0 (server.c:318:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process msg:5, size:528 (server.c:168:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process result 0 (server.c:318:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process msg:5, size:528 (server.c:168:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process result 0 (server.c:318:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process msg:1, size:16 (server.c:168:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process result 0 (server.c:318:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process msg:1, size:16 (server.c:168:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process result 0 (server.c:318:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process msg:1, size:16 (server.c:168:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process result 0 (server.c:318:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process msg:1, size:16 (server.c:168:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process result 0 (server.c:318:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /nodes (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start nodes (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug nodes (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = nodes new path = (null) (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end nodes = 0x5597960fa500 (nodes) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr nodes (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr nodes (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /nodes (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /nodes/node01-sxb-pve01 (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start nodes/node01-sxb-pve01 (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug nodes/node01-sxb-pve01 (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = nodes new path = node01-sxb-pve01 (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end nodes/node01-sxb-pve01 = 0x5597960fa500 (nodes/node01-sxb-pve01) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr nodes/node01-sxb-pve01 (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr nodes/node01-sxb-pve01 (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /nodes/node01-sxb-pve01 (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start nodes/node01-sxb-pve01/qemu-server (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug nodes/node01-sxb-pve01/qemu-server (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = nodes new path = node01-sxb-pve01/qemu-server (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end nodes/node01-sxb-pve01/qemu-server = 0x5597960fa500 (nodes/node01-sxb-pve01/qemu-server) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/690001.conf (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start nodes/node01-sxb-pve01/qemu-server/690001.conf (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug nodes/node01-sxb-pve01/qemu-server/690001.conf (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = nodes new path = node01-sxb-pve01/qemu-server/690001.conf (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end nodes/node01-sxb-pve01/qemu-server/690001.conf = 0x5597960fa500 (nodes/node01-sxb-pve01/qemu-server/690001.conf) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/690001.conf (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/690001.conf (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/690001.conf (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/120.conf (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start nodes/node01-sxb-pve01/qemu-server/120.conf (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug nodes/node01-sxb-pve01/qemu-server/120.conf (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = nodes new path = node01-sxb-pve01/qemu-server/120.conf (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end nodes/node01-sxb-pve01/qemu-server/120.conf = 0x5597960fa500 (nodes/node01-sxb-pve01/qemu-server/120.conf) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/120.conf (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/120.conf (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/120.conf (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/9999.conf (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start nodes/node01-sxb-pve01/qemu-server/9999.conf (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug nodes/node01-sxb-pve01/qemu-server/9999.conf (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = nodes new path = node01-sxb-pve01/qemu-server/9999.conf (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end nodes/node01-sxb-pve01/qemu-server/9999.conf = 0x5597960fa500 (nodes/node01-sxb-pve01/qemu-server/9999.conf) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/9999.conf (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/9999.conf (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/9999.conf (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/17002501.conf (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start nodes/node01-sxb-pve01/qemu-server/17002501.conf (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug nodes/node01-sxb-pve01/qemu-server/17002501.conf (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = nodes new path = node01-sxb-pve01/qemu-server/17002501.conf (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end nodes/node01-sxb-pve01/qemu-server/17002501.conf = 0x5597960fa500 (nodes/node01-sxb-pve01/qemu-server/17002501.conf) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/17002501.conf (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/17002501.conf (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/17002501.conf (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/3333.conf (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start nodes/node01-sxb-pve01/qemu-server/3333.conf (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug nodes/node01-sxb-pve01/qemu-server/3333.conf (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = nodes new path = node01-sxb-pve01/qemu-server/3333.conf (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end nodes/node01-sxb-pve01/qemu-server/3333.conf = 0x5597960fa500 (nodes/node01-sxb-pve01/qemu-server/3333.conf) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/3333.conf (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/3333.conf (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/3333.conf (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/103.conf (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start nodes/node01-sxb-pve01/qemu-server/103.conf (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug nodes/node01-sxb-pve01/qemu-server/103.conf (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = nodes new path = node01-sxb-pve01/qemu-server/103.conf (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end nodes/node01-sxb-pve01/qemu-server/103.conf = 0x5597960fa500 (nodes/node01-sxb-pve01/qemu-server/103.conf) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/103.conf (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/103.conf (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/103.conf (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/119.conf (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start nodes/node01-sxb-pve01/qemu-server/119.conf (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug nodes/node01-sxb-pve01/qemu-server/119.conf (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = nodes new path = node01-sxb-pve01/qemu-server/119.conf (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end nodes/node01-sxb-pve01/qemu-server/119.conf = 0x5597960fa500 (nodes/node01-sxb-pve01/qemu-server/119.conf) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/119.conf (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/119.conf (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/119.conf (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/118.conf (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start nodes/node01-sxb-pve01/qemu-server/118.conf (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug nodes/node01-sxb-pve01/qemu-server/118.conf (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = nodes new path = node01-sxb-pve01/qemu-server/118.conf (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end nodes/node01-sxb-pve01/qemu-server/118.conf = 0x5597960fa500 (nodes/node01-sxb-pve01/qemu-server/118.conf) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/118.conf (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/118.conf (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/118.conf (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/105.conf (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start nodes/node01-sxb-pve01/qemu-server/105.conf (pmxcfs.c:103:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug nodes/node01-sxb-pve01/qemu-server/105.conf (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = nodes new path = node01-sxb-pve01/qemu-server/105.conf (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end nodes/node01-sxb-pve01/qemu-server/105.conf = 0x5597960fa500 (nodes/node01-sxb-pve01/qemu-server/105.conf) (pmxcfs.c:110:find_plug)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/105.conf (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr nodes/node01-sxb-pve01/qemu-server/105.conf (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /nodes/node01-sxb-pve01/qemu-server/105.conf (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process msg:1, size:16 (server.c:168:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process result 0 (server.c:318:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process msg:1, size:16 (server.c:168:s1_msg_process_fn)
Sep 04 16:02:07 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process result 0 (server.c:318:s1_msg_process_fn)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: dfsm mode is 2 (dfsm.c:658:dfsm_cpg_deliver_callback)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: process message 7 (length = 58) (dcdb.c:806:dcdb_deliver)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: got valid unlock request message (dcdb.c:840:dcdb_deliver)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: leave dcdb_deliver (1) (dcdb.c:920:dcdb_deliver)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: dfsm mode is 2 (dfsm.c:658:dfsm_cpg_deliver_callback)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: process message 6 (length = 46) (dcdb.c:806:dcdb_deliver)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: dir is locked (memdb.c:917:memdb_mtime)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: leave dcdb_deliver (1) (dcdb.c:920:dcdb_deliver)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process msg:1, size:16 (server.c:168:s1_msg_process_fn)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process result 0 (server.c:318:s1_msg_process_fn)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_utimens /priv/lock/ha_manager_lock (pmxcfs.c:476:cfs_fuse_utimens)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start priv/lock/ha_manager_lock (pmxcfs.c:103:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug priv/lock/ha_manager_lock (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = priv new path = lock/ha_manager_lock (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end priv/lock/ha_manager_lock = 0x5597960fa500 (priv/lock/ha_manager_lock) (pmxcfs.c:110:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_utimes priv/lock/ha_manager_lock (cfs-plug.c:388:cfs_plug_base_utimens)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process msg:1, size:16 (server.c:168:s1_msg_process_fn)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [ipcs] debug: process result 0 (server.c:318:s1_msg_process_fn)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: dfsm mode is 2 (dfsm.c:658:dfsm_cpg_deliver_callback)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: process message 7 (length = 58) (dcdb.c:806:dcdb_deliver)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: got valid unlock request message (dcdb.c:840:dcdb_deliver)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: leave dcdb_deliver (1) (dcdb.c:920:dcdb_deliver)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: dfsm mode is 2 (dfsm.c:658:dfsm_cpg_deliver_callback)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: process message 6 (length = 46) (dcdb.c:806:dcdb_deliver)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: dir is locked (memdb.c:917:memdb_mtime)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [dcdb] debug: leave dcdb_deliver (1) (dcdb.c:920:dcdb_deliver)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_utimens /priv/lock/ha_manager_lock (-13) (pmxcfs.c:490:cfs_fuse_utimens)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr / (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start  (pmxcfs.c:103:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug  (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end  = 0x5597960fa500 () (pmxcfs.c:110:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_base_getattr  (cfs-plug.c:84:cfs_plug_base_getattr)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_plug_base_getattr  (cfs-plug.c:103:cfs_plug_base_getattr)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr / (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /.debug (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start .debug (pmxcfs.c:103:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug .debug (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = .debug new path = (null) (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end .debug = 0x55979610c210 ((null)) (pmxcfs.c:110:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_func_getattr  (cfs-plug-func.c:78:cfs_plug_func_getattr)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /.debug (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_open /.debug (pmxcfs.c:305:cfs_fuse_open)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start .debug (pmxcfs.c:103:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug .debug (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = .debug new path = (null) (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end .debug = 0x55979610c210 ((null)) (pmxcfs.c:110:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_func_open  (cfs-plug-func.c:197:cfs_plug_func_open)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_open /.debug (0) (pmxcfs.c:321:cfs_fuse_open)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_truncate /.debug 0 (pmxcfs.c:382:cfs_fuse_truncate)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start .debug (pmxcfs.c:103:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug .debug (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = .debug new path = (null) (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end .debug = 0x55979610c210 ((null)) (pmxcfs.c:110:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_truncate /.debug (0) (pmxcfs.c:394:cfs_fuse_truncate)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_getattr /.debug (pmxcfs.c:127:cfs_fuse_getattr)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start .debug (pmxcfs.c:103:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug .debug (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = .debug new path = (null) (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end .debug = 0x55979610c210 ((null)) (pmxcfs.c:110:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_func_getattr  (cfs-plug-func.c:78:cfs_plug_func_getattr)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: leave cfs_fuse_getattr /.debug (0) (pmxcfs.c:154:cfs_fuse_getattr)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_fuse_write /.debug 2 0 (pmxcfs.c:359:cfs_fuse_write)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug start .debug (pmxcfs.c:103:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug .debug (cfs-plug.c:52:cfs_plug_base_lookup_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: cfs_plug_base_lookup_plug name = .debug new path = (null) (cfs-plug.c:59:cfs_plug_base_lookup_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: find_plug end .debug = 0x55979610c210 ((null)) (pmxcfs.c:110:find_plug)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] debug: enter cfs_plug_func_write (cfs-plug-func.c:158:cfs_plug_func_write)
Sep 04 16:02:08 node01-sxb-pve01 pmxcfs[2146]: [main] notice: disable debug mode (pmxcfs.c:664:write_debug_setting_cb)
Sep 04 16:02:19 node01-sxb-pve01 sshd[240123]: Invalid user henrique from 95.167.212.50 port 13321
Sep 04 16:02:19 node01-sxb-pve01 sshd[240123]: input_userauth_request: invalid user henrique [preauth]
Sep 04 16:02:19 node01-sxb-pve01 sshd[240123]: Connection closed by 95.167.212.50 port 13321 [preauth]
Sep 04 16:02:27 node01-sxb-pve01 sshd[240066]: Connection closed by 220.130.11.4 port 53256 [preauth]

Content of etc/pve

Code: 
drwxr-xr-x   2 root www-data     0 Jan  1  1970 .
drwxr-xr-x 117 root root     12288 Sep  4 05:25 ..
-rw-r-----   1 root www-data   451 Aug 17  2016 authkey.pub
-rw-r-----   1 root www-data  1592 Dec 11  2017 ceph.conf
-r--r-----   1 root www-data  8804 Jan  1  1970 .clusterlog
-rw-r-----   1 root www-data   577 Dec 22  2017 corosync.conf
-rw-r-----   1 root www-data   580 Jun 27  2017 corosync.conf.BACK
-rw-r-----   1 root www-data   580 Dec 22  2017 corosync.conf.new
-rw-r-----   1 root www-data   545 Jun 22  2017 corosync.conf.ORIG
-rw-r-----   1 root www-data   126 Sep  4 03:23 datacenter.cfg
-rw-r-----   1 root www-data     2 Jan  1  1970 .debug
-rw-r-----   1 root www-data   116 Jun 22  2017 domains.cfg
drwxr-xr-x   2 root www-data     0 Jul 12  2017 firewall
drwxr-xr-x   2 root www-data     0 Aug 17  2016 ha
lrwxr-xr-x   1 root www-data     0 Jan  1  1970 local -> nodes/node01-sxb-pve01
lrwxr-xr-x   1 root www-data     0 Jan  1  1970 lxc -> nodes/node01-sxb-pve01/lxc
-r--r-----   1 root www-data   350 Jan  1  1970 .members
drwxr-xr-x   2 root www-data     0 Aug 17  2016 nodes
lrwxr-xr-x   1 root www-data     0 Jan  1  1970 openvz -> nodes/node01-sxb-pve01/openvz
drwx------   2 root www-data     0 Aug 17  2016 priv
-rw-r-----   1 root www-data  6145 Jun 23  2017 pve-root-ca.pem
-rw-r-----   1 root www-data  1679 Aug 17  2016 pve-www.key
lrwxr-xr-x   1 root www-data     0 Jan  1  1970 qemu-server -> nodes/node01-sxb-pve01/qemu-server
-r--r-----   1 root www-data  7579 Jan  1  1970 .rrd
-rw-r-----   1 root www-data   808 Jul 28 16:20 storage.cfg
-rw-r-----   1 root www-data  1160 Sep  4 00:00 user.cfg
-r--r-----   1 root www-data   506 Jan  1  1970 .version
-r--r-----   1 root www-data  3475 Jan  1  1970 .vmlist
-rw-r-----   1 root www-data   768 Jul 15 13:09 vzdump.cron

I will need you to give me more hints, because like I said no configs were altered and this causes me quite some working discrepancies.

Best,
Cosmin
UHL Services
 
Try running as root
Code: 
/usr/bin/ssh -vvv -o 'BatchMode=yes' -o 'HostKeyAlias=node01-sxb-pve01' root@172.18.1.1
from node2.
This has to work without password input.

If it doesn't work - try taking the authorized_keys file from a node and reproducing the connection issue with another host.
 
Hi Stoiko,

This last command worked just well:

Code: 
root@node02-sxb-pve01:~# /usr/bin/ssh -vvv -o 'BatchMode=yes' -o 'HostKeyAlias=node01-sxb-pve01' root@172.18.1.1
OpenSSH_7.4p1 Debian-10+deb9u4, OpenSSL 1.0.2l  25 May 2017
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "172.18.1.1" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to 172.18.1.1 [172.18.1.1] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4p1 Debian-10+deb9u4
debug1: match: OpenSSH_7.4p1 Debian-10+deb9u4 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to 172.18.1.1:22 as 'root'
debug1: using hostkeyalias: node01-sxb-pve01
debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /root/.ssh/known_hosts:8
debug3: load_hostkeys: loaded 1 keys from node01-sxb-pve01
debug3: hostkeys_foreach: reading file "/etc/ssh/ssh_known_hosts"
debug3: record_hostkey: found key type RSA in file /etc/ssh/ssh_known_hosts:8
debug3: load_hostkeys: loaded 1 keys from node01-sxb-pve01
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-ed25519-cert-v01@openssh.com,ssh-ed25519
debug2: ciphers ctos: blowfish-cbc,aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc
debug2: ciphers stoc: blowfish-cbc,aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes128-ctr MAC: umac-64-etm@openssh.com compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: umac-64-etm@openssh.com compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:tgY+lqxka/pH5oTZijsoHi695F+B3b9QCBwG2lOkT3E
debug1: using hostkeyalias: node01-sxb-pve01
debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /root/.ssh/known_hosts:8
debug3: load_hostkeys: loaded 1 keys from node01-sxb-pve01
debug3: hostkeys_foreach: reading file "/etc/ssh/ssh_known_hosts"
debug3: record_hostkey: found key type RSA in file /etc/ssh/ssh_known_hosts:8
debug3: load_hostkeys: loaded 1 keys from node01-sxb-pve01
debug1: Host 'node01-sxb-pve01' is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:8
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 4294967296 blocks
debug2: key: /root/.ssh/id_rsa (0x55dff7774620)
debug2: key: /root/.ssh/id_dsa ((nil))
debug2: key: /root/.ssh/id_ecdsa ((nil))
debug2: key: /root/.ssh/id_ed25519 ((nil))
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey
debug3: authmethod_lookup publickey
debug3: remaining preferred: ,gssapi-with-mic,publickey
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /root/.ssh/id_rsa
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: pkalg ssh-rsa blen 279
debug2: input_userauth_pk_ok: fp SHA256:/zJYSbpqgkCmiUN+2KD3ar9/jvR9vtVK0D5GyOLh7do
debug3: sign_and_send_pubkey: RSA SHA256:/zJYSbpqgkCmiUN+2KD3ar9/jvR9vtVK0D5GyOLh7do
debug3: send packet: type 50
debug3: receive packet: type 52
debug1: Authentication succeeded (publickey).
Authenticated to 172.18.1.1 ([172.18.1.1]:22).
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting no-more-sessions@openssh.com
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: network
debug3: receive packet: type 80
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug3: receive packet: type 91
debug2: callback start
debug2: fd 3 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x10
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug3: send packet: type 98
debug1: Sending environment.
debug1: Sending env LANG = en_GB.UTF-8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env HUSHLOGIN
debug3: Ignored env USER
debug3: Ignored env PWD
debug3: Ignored env HOME
debug3: Ignored env MAIL
debug3: Ignored env SHELL
debug3: Ignored env TERM
debug3: Ignored env SHLVL
debug3: Ignored env LOGNAME
debug3: Ignored env PATH
debug3: Ignored env _
debug2: channel 0: request shell confirm 1
debug3: send packet: type 98
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
Linux node01-sxb-pve01 4.15.18-2-pve #1 SMP PVE 4.15.18-20 (Thu, 16 Aug 2018 11:06:35 +0200) x86_64
Linux node01-sxb-pve01 4.15.18-2-pve #1 SMP PVE 4.15.18-20 (Thu, 16 Aug 2018 11:06:35 +0200) x86_64 GNU/Linux

server    : 717284
ip        :
hostname  : node01-sxb-pve01

Last login: Tue Sep  4 1

What is causing it not to work for outside keys?
 
I found the issue!

Thank you for your time and guidance, it will be good to add a note also to documentation on similar issues, people should check the authorised_keys to maintain a proper format. Somehow the format was altered upon a edit, and this caused some keys to be missused.
 
UHL-Services said:
I found the issue!

Thank you for your time and guidance, it will be good to add a note also to documentation on similar issues, people should check the authorised_keys to maintain a proper format. Somehow the format was altered upon a edit, and this caused some keys to be missused.
Click to expand...
Hi,

Sorry reviving this old thread but I'm with exactly same problem and I didn't understand how you solve.

I have 3 nodes: pve01, pve02, pve03

pve01 connect fine with ssh to pve02 and pve03
pve03 connect fine with ssh to pve01 and pve02

But pve02 connect only with password to pve01 and pve03, and if i try to see console from some VM I receive the error: Failed to connect to server on noVNC screen

logs show: Connection closed by 10.51.10.229 port 47670 [preauth]

The file authorized_keys is the same on the 3 nodes.

Any tips?
 
Roger_cwb said:
Any tips?
Click to expand...
without reproducing your situation here - one of the first things I'd try is running:
Code: 
pvecm updatecerts --force

Roger_cwb said:
Sorry reviving this old thread but I'm with exactly same problem and I didn't understand how you solve.
Click to expand...
please create a new thread instead (especially after more than 2 years - the software changes quite a bit, some things might work differently - and more people might see your thread.)
 
  • Like
Reactions: Roger_cwb
Stoiko Ivanov said:
without reproducing your situation here - one of the first things I'd try is running:
Code: 
pvecm updatecerts --force

please create a new thread instead (especially after more than 2 years - the software changes quite a bit, some things might work differently - and more people might see your thread.)
Click to expand...
THANK YOU!!!!

The command:
Code: 
pvecm updatecerts --force

Solved the problem.

I will create a new thread the next time I'm sorry, I was desperate.
 
  • Like
Reactions: Stoiko Ivanov
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back