PVE host ssh timeout issues

[crazyelectron-io]

I have a fresh install of PVE 9.1.1 on two nodes for testing purpose and have very annoying and frequent issues with SSH timeouts from my local machine (MacOS 26.2). It works fine for a little while and then it stops working until the node is reboot or I wait a long time. I use Ansible 2.20 to configure the nodes and have already optimized SSH connection reuse parameters in ansible.cfg like this:

[defaults]
timeout = 30
forks = 5
host_key_checking = False
[ssh_connection]
# Keep the session alive and reuse a single TCP connection per host
ssh_args = -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -o TCPKeepAlive=yes -o ControlMaster=auto -o ControlPersist=10m -o ConnectTimeout=15
pipelining = True

After a few runs of my Ansible playbook one or both hosts don't respond to connection attempts to port 22 anymore. The network is fine because I can ping the hosts and access port 8006 at that moment without any issue. There is no logging on the hosts as shown with journalctl when the port becomes unresponsive. The ssh -vvvvv command shows:

debug1: OpenSSH_10.0p2, LibreSSL 3.3.6
debug3: Running on Darwin 25.2.0 Darwin Kernel Version 25.2.0: Tue Nov 18 21:09:40 PST 2025; root:xnu-12377.61.12~1/RELEASE_ARM64_T6000 arm64
debug3: Started with: ssh -vvvvv root@10.100.0.11
debug1: Reading configuration data /Users/ron/.ssh/config
debug1: /Users/ron/.ssh/config line 4: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 22: Including file /etc/ssh/ssh_config.d/100-macos.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/100-macos.conf
debug1: /etc/ssh/ssh_config.d/100-macos.conf line 1: Applying options for *
debug3: /etc/ssh/ssh_config.d/100-macos.conf line 3: Including file /etc/ssh/crypto.conf depth 1
debug1: Reading configuration data /etc/ssh/crypto.conf
debug3: kex names ok: [ecdh-sha2-nistp256]
debug2: resolve_canonicalize: hostname 10.100.0.11 is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/ME/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/ME/.ssh/known_hosts2'
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug3: channel_clear_timeouts: clearing
debug3: ssh_connect_direct: entering
debug1: Connecting to 10.100.0.11 [10.100.0.11] port 22.
debug3: set_sock_tos: set socket 3 IP_TOS 0x48
debug1: connect to address 10.100.0.11 port 22: Operation timed out
ssh: connect to host 10.100.0.11 port 22: Operation timed out

I've also disabled the firewall and increased the OpenSSH parameters:

MaxStartups 100:50:500
LoginGraceTime 30
MaxSessions 200
PerSourceMaxStartups 100
UsePAM no
UseDNS no

It seems to delay the inevitable, but I still run into SSH connection timeouts.
Any suggestions for further troubleshooting?

 

[Gilou]

Maybe add a network packet dump.. that feels a bit weird..