Unable to open Shell : Connection failed (Error 500: Timeout while waiting for port '5900' to get ready!)

[dimitriz]

I haven't had any issues with this previously and it seems like this started after applying last updates. (Especially since I was able to apply them previously without any problems.)
Now any time I try to open shell to either of my 2 nodes this comes up.

Mar 12 08:58:50 node1 pvedaemon[7683]: starting termproxy UPID:node1:00001E03:03E090B9:604B739A:vncshell::root@pam:
Mar 12 08:58:50 node1 pvedaemon[19880]: <root@pam> starting task UPID:node1:00001E03:03E090B9:604B739A:vncshell::root@pam:
Mar 12 08:58:50 node1 pvedaemon[7683]: command '/usr/bin/termproxy 5900 --path /nodes/node1 --perm Sys.Console -- /bin/login -f root' failed: exit code 1
Mar 12 08:58:50 node1 pvedaemon[19880]: <root@pam> end task UPID:node1:00001E03:03E090B9:604B739A:vncshell::root@pam: command '/usr/bin/termproxy 5900 --path /nodes/node1 --perm Sys.Console -- /bin/login -f root' failed: exit code 1

The only change I made to either of the nodes was opening port 80 because SSL certificate renewal wasn't going through. Nothing else.
It worked fine since I set up the nodes in December.

Thanks!

 

[Moayad]

Could you try to renew the certificate again? it could be from the certificate because it works here well

 

[Moayad]

also please post the output of pveversion -v

 

[dimitriz]

Moayad,

Here is the certificate Order: (It's identical with both nodes)

Task viewer: SRV - Order Certificate

OutputStatus

Stop
Loading ACME account details
Placing ACME order
Order URL: https://acme-v02.api.letsencrypt.org/acme/order/10XXXXXXXXXXXXX

Getting authorization details from 'https://acme-v02.api.letsencrypt.org/acme/authz-v3/11XXXXXXX'
node1.dcon.solutions is already validated!

All domains validated!

Creating CSR
Checking order status
Order is ready, finalizing order
valid!

Downloading certificate
Setting pveproxy certificate and key
Restarting pveproxy
TASK OK


root@node1:~# pveversion -v
proxmox-ve: 6.3-1 (running kernel: 5.4.103-1-pve)
pve-manager: 6.3-6 (running version: 6.3-6/2184247e)
pve-kernel-5.4: 6.3-7
pve-kernel-helper: 6.3-7
pve-kernel-5.4.103-1-pve: 5.4.103-1
pve-kernel-5.4.101-1-pve: 5.4.101-1
pve-kernel-5.4.98-1-pve: 5.4.98-1
pve-kernel-5.4.78-2-pve: 5.4.78-2
pve-kernel-5.4.78-1-pve: 5.4.78-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.0-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.0.7
libproxmox-backup-qemu0: 1.0.3-1
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.3-5
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.1-1
libpve-storage-perl: 6.3-7
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.10-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.4-6
pve-cluster: 6.2-1
pve-container: 3.3-4
pve-docs: 6.3-1
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.2-2
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.2.0-3
pve-xtermjs: 4.7.0-3
pve-zsync: 2.0-4
qemu-server: 6.3-8
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.3-pve2


Thank you.

 

[dimitriz]

The only way I can get shell is if I click Shell option on the upper right corner and select noVNC.
Also by connecting with Putty of course.

Initially when I set it up I am thinking I got the certificate before setting up firewall. 3 month later because it couldn't auto renew my certificate unless I opened port 80.

 

[Ilya_Kuznetsov]

The only way I can get shell is if I click Shell option on the upper right corner and select noVNC.
Also by connecting with Putty of course.

Initially when I set it up I am thinking I got the certificate before setting up firewall. 3 month later because it couldn't auto renew my certificate unless I opened port 80.

Hello. write an entry in DNS: 127.0.0.1 localhost. localdomain localhost

 

[dimitriz]

Hello. write an entry in DNS: 127.0.0.1 localhost. localdomain localhost

Wow, man! I waited for 6m for you to write this
Works!
Awesomeness.

 

[cyber7]

Wow, man! I waited for 6m for you to write this
Works!
Awesomeness.

THANK YOU, THANK YOU, THANK YOU!!!

 

[SteveCliff]

Hello. write an entry in DNS: 127.0.0.1 localhost. localdomain localhost

Works a treat - thank you !!!

 

[GyulaMasa]

Hello everybody!

Can everybody explain the "Hello. write an entry in DNS: 127.0.0.1 localhost. localdomain localhost" in a bit more details... I am about 8 hours new to Proxmox, but not a newbie to linux in general.
I made the mistake to want to change the node name from "pve" to something else. I changed /etc/hosts and /etc/hostname with the desired name, but that messed up everything big time.. The new node name appeared, but all the VMs were orphaned, and I could not start them.
So, I restarted the PC and logged in to the local terminal, and undid both the changes.
I restarted again, and now the name is back to "pve", everything else is working, only the pve node shell does this failure. If I do what was mentioned above ("The only way I can get shell is if I click Shell option on the upper right corner and select noVNC.") that works for me too.
Thank you very much!
UPDATE: I tried to log in from another machine, but the symptoms are the same.

 

[r51f]

Hello everybody!

Can everybody explain the "Hello. write an entry in DNS: 127.0.0.1 localhost. localdomain localhost" in a bit more details... I am about 8 hours new to Proxmox, but not a newbie to linux in general.
I made the mistake to want to change the node name from "pve" to something else. I changed /etc/hosts and /etc/hostname with the desired name, but that messed up everything big time.. The new node name appeared, but all the VMs were orphaned, and I could not start them.
So, I restarted the PC and logged in to the local terminal, and undid both the changes.
I restarted again, and now the name is back to "pve", everything else is working, only the pve node shell does this failure. If I do what was mentioned above ("The only way I can get shell is if I click Shell option on the upper right corner and select noVNC.") that works for me too.
Thank you very much!
UPDATE: I tried to log in from another machine, but the symptoms are the same.

Node -> System -> Hosts
In text field delete string with 127.0.0.1 and paste 127.0.0.1 localhost.localdomain localhost.
Worked for me.
*Edited mistake

 

[DAKnn]

wrong 127.0.0.1 localhost. localdomain localhost.
correctly 127.0.0.1 localhost.localdomain localhost