OpenSSH dropping RSA; Proxmox failing 'ssh'.

[egberts]

Given that OpenSSH is dropping RSA from its default lineup of KeX algorithms, I proceeded to do the same for Proxmox here.

Unfortunately, the removal of the RSA-related `ssh_host_rsa_key`/`ssh_host_rsa_key.pub` file resulted in some sporadic web-based popup "Error" message:

```
can't open '/etc/ssh/ssh_host_rsa_key.pub' - No such file or directory (500)
```
It is not clear to me who is reporting this but I do see this message when visiting

* View Host (https://example.test:8006/#v1:0:=node/tank:4:5::::6:6:16 )
* Repositiories (https://example.test:8006/#v1:0:=node/tank:4:=aptrepositories::::::16 )
* missed one other and cannot recall what the link was.

 

[egberts]

Doing a little peek at sshd by doing a show process command:

/usr/sbin/sshd -D -oCiphers=aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc -oMACs=hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512 -oGSSAPIKexAlgorithms=gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1- -oKexAlgorithms=curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1 -oHostKeyAlgorithms=ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-256,rsa-sha2-256-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-512-cert-v01@openssh.com,ssh-rsa,ssh-rsa-cert-v01@openssh.com -oPubkeyAcceptedKeyTypes=ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-256,rsa-sha2-256-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-512-cert-v01@openssh.com,ssh-rsa,ssh-rsa-cert-v01@openssh.com -oCASignatureAlgorithms=ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-256,rsa-sha2-512,ssh-rsa


OK, this is a bit much that needs turning off as I move toward the OpenSSH v9.0p1 while complying with various SSH hardening guidelines, of which the most thorny is “PermitRootLogin off”. so, I started creating and substituting “admin” account for root.

I also have this whole other set of OpenSSH settings (all of them hardened) and they all reside in /etc/ssh/sshd_config.d/ subdirectory:

rw-r----- 1 root sshd 1447 Apr 11 09:51 100-daemon-logging.conf
-rw-r----- 1 root sshd 2619 Apr 11 10:11 111-daemon-network.conf
-rw-r----- 1 root sshd 960 Apr 11 09:51 112-daemon-pidfile.conf
-rw-r----- 1 root sshd 1402 Apr 11 09:51 113-daemon-maxstartups.conf
-rw-r----- 1 root sshd 1201 Apr 11 09:51 114-daemon-ip-addr-families.conf
-rw-r----- 1 root sshd 1119 Apr 11 09:51 115-daemon-tcpkeepalive.conf
-rw-r----- 1 root sshd 720 Apr 11 09:51 119-daemon-rdomain.conf
-rw-r----- 1 root sshd 130 Apr 11 09:51 200-ip-socket-created
-rw-r----- 1 root sshd 551 Apr 11 09:51 201-daemon-login-grace.conf
-rw-r----- 1 root sshd 558 Apr 11 09:51 220-daemon-subversion.conf
-rw-r----- 1 root sshd 538 Apr 11 09:51 223-daemon-banner-debian.conf
-rw-r----- 1 root sshd 867 Apr 11 09:51 260-login-auth-keyagent.conf
-rw-r----- 1 root sshd 2832 Apr 11 09:51 300-protocol-kex-algos.conf
-rw-r----- 1 root sshd 1550 Apr 11 09:51 302-protocol-ciphers.conf
-rw-r----- 1 root sshd 1909 Apr 11 09:51 304-protocol-macs.conf
-rw-r----- 1 root sshd 634 Apr 11 09:51 310-protocol-compression.conf
-rw-r----- 1 root sshd 1031 Apr 11 09:51 324-protocol-rekey.conf
-rw-r----- 1 root sshd 1155 Apr 11 09:51 336-protocol-hostkey.conf
-rw-r----- 1 root sshd 2251 Apr 11 09:51 337-protocol-host-algos.conf
-rw-r----- 1 root sshd 667 Apr 11 09:51 338-protocol-host-certs.conf
-rw-r----- 1 root sshd 1149 Apr 11 09:51 339-protocol-sec-key-provider.conf
-rw-r----- 1 root sshd 617 Apr 11 09:51 400-protocol-dns.conf
-rw-r----- 1 root sshd 5007 Apr 11 09:51 410-protocol-auth-methods.conf
-rw-r----- 1 root sshd 1191 Apr 11 09:51 412-protocol-auth-max-tries.conf
-rw-r----- 1 root sshd 969 Apr 11 09:51 422-protocol-password-empty-permit.conf
-rw-r----- 1 root sshd 823 Apr 11 09:51 424-protocol-auth-type-password.conf
-rw-r----- 1 root sshd 841 Apr 11 09:51 426-protocol-key-fingerprint.conf
-rw-r----- 1 root sshd 873 Apr 11 09:51 430-auth-pubkey.conf
-rw-r----- 1 root sshd 1432 Apr 11 09:51 431-auth-pubkey-options.conf
-rw-r----- 1 root sshd 1543 Apr 11 09:51 432-auth-pubkey-types.conf
-rw-r----- 1 root sshd 2114 Apr 11 09:51 433-auth-pubkey-principals-file.conf
-rw-r----- 1 root sshd 1147 Apr 11 09:51 434-auth-pubkey-cas-algos.conf
-rw-r----- 1 root sshd 993 Apr 11 09:51 435-auth-pubkey-revoked.conf
-rw-r----- 1 root sshd 1117 Apr 11 09:51 436-auth-pubkey-auth-file.conf
-rw-r----- 1 root sshd 1140 Apr 11 09:51 436-auth-pubkey-cmds-key-username.conf
-rw-r----- 1 root sshd 2252 Apr 11 09:51 437-auth-pubkey-login-auth-cmds-key.conf
-rw-r----- 1 root sshd 1157 Apr 11 09:51 438-auth-pubkey-ca-keys.conf
-rw-r----- 1 root sshd 1172 Apr 11 09:51 439-auth-pubkey-permit-user-environment.conf
-rw-r----- 1 root sshd 767 Apr 11 09:51 440-auth-challenge.conf
-rw-r----- 1 root sshd 888 Apr 11 13:48 440-login-strict-modes.conf
-rw-r----- 1 root sshd 794 Apr 11 09:51 442-auth-hostbased.conf
-rw-r----- 1 root sshd 3776 Apr 11 09:51 443-auth-hostbased-opts.conf
-rw-r----- 1 root sshd 1030 Apr 11 09:51 450-auth-krb5-gss-api.conf
-rw-r----- 1 root sshd 1062 Apr 11 09:51 460-auth-kbd.conf
-rw-r----- 1 root sshd 35 Apr 11 09:51 500-session.conf
-rw-r----- 1 root sshd 1840 Apr 11 09:51 501-session-client-alive.conf
-rw-r----- 1 root sshd 785 Apr 11 09:51 507-kex-permit-open.conf
-rw-r----- 1 root sshd 1461 Apr 11 09:51 510-auth2-pam.conf
-rw-r----- 1 root sshd 1402 Apr 11 09:51 510-login-pam.conf
-rw-r----- 1 root sshd 1953 Apr 11 09:51 512-auth2-permit-root-login.conf
-rw-r----- 1 root sshd 1937 Apr 11 09:51 515-auth2-permit-listen.conf
-rw-r----- 1 root sshd 909 Apr 11 09:51 516-auth2-use-dns.conf
-rw-r----- 1 root sshd 1756 Apr 11 09:51 520-login-chroot-dir.conf
-rw-r----- 1 root sshd 841 Apr 11 09:51 530-login-users-deny.conf
-rw-r----- 1 root sshd 872 Apr 11 09:51 540-login-users-allow.conf
-rw-r----- 1 root sshd 625 Apr 11 09:51 550-login-groups-deny.conf
-rw-r----- 1 root sshd 1453 Apr 11 09:51 570-login-groups-allow.conf
-rw-r----- 1 root sshd 1651 Apr 11 09:51 580-login-auth-cmds-princ.conf
-rw-r----- 1 root sshd 1739 Apr 11 09:51 590-login-auth-cmds-key.conf
-rw-r----- 1 root sshd 451 Apr 11 09:51 592-login-user-rc-permit.conf
-rw-r----- 1 root sshd 573 Apr 11 09:51 610-login-auth-keyagent.conf
-rw-r----- 1 root sshd 1164 Apr 11 09:51 700-tunnel-permit.conf
-rw-r----- 1 root sshd 1775 Apr 11 09:51 750-allow-tcp-forwarding.conf
-rw-r----- 1 root sshd 2969 Apr 11 09:51 811-pty-x11-support.conf
-rw-r----- 1 root sshd 864 Apr 11 09:51 849-sess-chan-forwarding.conf
-rw-r----- 1 root sshd 2065 Apr 11 09:51 850-sess-chan-subsystem.conf
-rw-r----- 1 root sshd 1060 Apr 11 09:51 851-sess-chan-allow-agent-fwd.conf
-rw-r----- 1 root sshd 508 Apr 11 09:51 852-pty-max-sessions.conf
-rw-r----- 1 root sshd 1896 Apr 11 09:51 852-sess-chan-pty-set-env.conf
-rw-r----- 1 root sshd 150 Apr 11 09:51 853-pty-allowed.conf
-rw-r----- 1 root sshd 555 Apr 11 09:51 853-sess-chan-pty-allowed.conf
-rw-r----- 1 root sshd 433 Apr 11 09:51 854-pty-lastlogin.conf
-rw-r----- 1 root sshd 1098 Apr 11 09:51 854-sess-chan-pty-lastlogin.conf
-rw-r----- 1 root sshd 451 Apr 11 09:51 855-pty-banner.conf
-rw-r----- 1 root sshd 544 Apr 11 09:51 855-pty-printmotd.conf
-rw-r----- 1 root sshd 669 Apr 11 09:51 855-sess-chan-pty-banner.conf
-rw-r----- 1 root sshd 1108 Apr 11 09:51 856-pty-forced-command.conf
-rw-r----- 1 root sshd 817 Apr 11 09:51 856-sess-chan-pty-printmotd.conf
-rw-r----- 1 root sshd 1148 Apr 11 09:51 857-pty-shell-env.conf
-rw-r----- 1 root sshd 1846 Apr 11 09:51 857-sess-chan-pty-forced-cmd.conf
-rw-r----- 1 root sshd 2355 Apr 11 09:51 858-sess-chan-pty-shell-env.conf
-rw-r----- 1 root sshd 821 Apr 11 09:51 859-pty-ip-qos.conf
-rw-r----- 1 root sshd 1019 Apr 11 09:51 859-sess-chan-pty-max.conf
-rw-r----- 1 root sshd 1467 Apr 11 09:51 860-sess-chan-pty-ip-qos.conf
-rw-r----- 1 root sshd 1786 Apr 11 09:51 900-pipe.conf
-rw-r----- 1 root sshd 624 Apr 11 09:51 902-expose-auth-info.conf
-rw-r----- 1 root sshd 1705 Apr 11 09:51 910-ctrl-gateway-ports.conf
-rw-r----- 1 root sshd 517 Apr 11 09:51 912-stream-local-bind-unlink.conf
-rw-r----- 1 root sshd 2540 Apr 11 09:51 999-match-template.conf
-rw-r----- 1 root sshd 1216 Apr 11 09:51 allow-tcp-forwarding.conf
-rw-r----- 1 root sshd 1468 Apr 11 09:51 client-alive.conf
-rw-r----- 1 root sshd 371 Apr 11 09:51 expose-auth-info.conf
-rw-r----- 1 root sshd 364 Apr 11 09:51 forwarding-disabled.conf
-rw-r----- 1 root sshd 1021 Apr 11 09:51 gateway-ports.conf
-rw-r----- 1 root sshd 527 Apr 11 09:51 match-sftp-users.sh
-rw-r----- 1 root sshd 1044 Apr 11 09:51 permit-root-login.conf
-rw-r----- 1 root sshd 575 Apr 11 09:51 permit-user-environment.conf
-rw-r----- 1 root sshd 712 Apr 11 09:51 per-source-net-block-size.conf

Unfortunately, the settings are combatative between what is passed at command line options and what is in the config files. Worse over, SOME CLI have precedence over config file, while OTHER settings in config files have priority over CLI options.

also, the choices of hmac, KeX, public key algorithms all needs changing as well. it is all in https://github.com/egberts/easy-admin/tree/main/490-net-ssh



Perhaps, it is time to consider using the split-file config mode for OpenSSH.

 

[egberts]

it should be possible to draft a drop-in sshd_config (into the sshd_config.d subdirectory in sshd split-config mode) just for the for this PVE cluster usage (and also for some web-based retrieved by-ssh status APIs) .

this is what I am doing.

 

[scurrier]

Thanks for your interesting posts. They pertain to me. I would like to enable SSH key-only authentication with an admin user like you describe, but from other forum posts, it seems like this may interfere with the Proxmox engine.

this is what I am doing.

Would you be able to elaborate on how you got SSH key-only authentication with an admin user working?

 

[fabian]

you misunderstood something I think - openssh didn't remove support for RSA keys (and has no plans for that either) - it dropped support for one SHA1-based signature scheme called 'ssh-rsa' (this has no relation to the key type 'ssh-rsa', except that both use the RSA public crypto algorithm). the reason for that was the weakness of SHA1, not any weakness of RSA.

you can check what your ssh client and server support and agree on algorithm and cipher-wise by adding -v to the ssh invocation:

ssh -v ..
...
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
...
debug1: Server host key: ssh-ed25519 SHA256:XXXXX
...
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com>
...

 

[binaryanomaly]

I have a similar issue. I have regenerated the ssh keys and only created a new ed25519 key.

Now the UI complains: can't open '/etc/ssh/ssh_host_rsa_key.pub' - No such file or directory (500)

Don't need and don't want it.

Is this somewhere hardcoded in the UI or why am I seeing this?
The rsa key is absolutely not required.

 

[fabian]

yes, it is needed. you can configure sshd to not use it if you really don't want it, although I don't see any practical reason for that..

 

[binaryanomaly]

yes, it is needed. you can configure sshd to not use it if you really don't want it, although I don't see any practical reason for that..

Where would I have to do this? I see no explicit reference in the ssh config files.

SSH works fine it seems only pve is complaining about it

Aug 17 19:33:48 bm-deb-proxmox pvecm[3195]: can't open '/etc/ssh/ssh_host_rsa_key.pub' - No such file or directory

 

[fabian]

Where would I have to do this? I see no explicit reference in the ssh config files.

HostKey and/or HostKeyAlgorithms

SSH works fine it seems only pve is complaining about it

Aug 17 19:33:48 bm-deb-proxmox pvecm[3195]: can't open '/etc/ssh/ssh_host_rsa_key.pub' - No such file or directory

yes, it's not required by SSH, but by PVE

 

[binaryanomaly]

Thanks.

Explicitly setting

HostKey /etc/ssh/ssh_host_ed25519_key

in /etc/ssh/sshd_config seems to have no effect.

Looks a bit like hardcoded UI? legacy to me - as I have not recognized any effect besides that error message in the UI?