[SOLVED] noVNC and xterm are not working since subdomain changes

[pl11x]

::::: It has been SOLVED :::::​

Until few days ago, I was using https://domain.ltd:8006 to login into the web UI but I've changed it to https://sub.domain.ltd and it seems something has been broken because I cannot use noVNC or xterm console outside LAN (there is no problem in LAN, it works properly). Tasks log outputs:

"timed out waiting for client
TASK ERROR: command '/usr/bin/termproxy 5900 --path /nodes/p33ps --perm Sys.Console -- /bin/login -f root' failed: exit code 4"

"TASK ERROR: Failed to run vncproxy."

"'/bin/nc6 -l -p 5903 -w 10 -e '/usr/sbin/qm vncproxy 201 2>/dev/null'' failed: exit code 1"

I tried everything I found surfing on Google but nothing seems to work so, please, could someone take a look and shedding some light on the issue...?

Thanks!

 

[pl11x]

I opened 8006 just to test if xterm/VNC works outside LAN, and it worked like a charm through https://domain.ltd:8006, so it seems that something happens with subdomain setup but I cannot find anything

 

[dcsapak]

can you post the output of your 'pveversion -v' ?

 

[pl11x]

can you post the output of your 'pveversion -v' ?

Yes, of course. Here you have:

proxmox-ve: 5.2-2 (running kernel: 4.4.19-1-pve)
pve-manager: 5.2-1 (running version: 5.2-1/0fcd7879)
pve-kernel-4.15: 5.2-2
pve-kernel-4.15.17-2-pve: 4.15.17-10
pve-kernel-4.4.128-1-pve: 4.4.128-111
pve-kernel-4.4.35-2-pve: 4.4.35-79
pve-kernel-4.4.35-1-pve: 4.4.35-77
pve-kernel-4.4.19-1-pve: 4.4.19-66
pve-kernel-4.4.10-1-pve: 4.4.10-54
pve-kernel-4.4.6-1-pve: 4.4.6-48
corosync: 2.4.2-pve5
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-apiclient-perl: 2.0-4
libpve-common-perl: 5.0-31
libpve-guest-common-perl: 2.0-16
libpve-http-server-perl: 2.0-8
libpve-storage-perl: 5.0-23
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 3.0.0-3
lxcfs: 3.0.0-1
novnc-pve: 0.6-4
proxmox-widget-toolkit: 1.0-18
pve-cluster: 5.0-27
pve-container: 2.0-23
pve-docs: 5.2-4
pve-firewall: 3.0-9
pve-firmware: 2.0-4
pve-ha-manager: 2.0-5
pve-i18n: 1.0-5
pve-libspice-server1: 0.12.8-3
pve-qemu-kvm: 2.11.1-5
pve-xtermjs: 1.0-5
qemu-server: 5.0-26
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.9-pve1~bpo9

 

[pl11x]

can you post the output of your 'pveversion -v' ?

Hi,

Is the output above useful?

Thanks!

 

[r.jochum]

You run on a very old Kernel, seems you didn't reboot for a long long time, he?

You realy need a reboot of that box, do it in the next maintenance window.

What should help for the moment without reboot (while the reboot is still very good!):

systemctl restart pveproxy
systemctl restart pvedaemon

As root please run:

/usr/bin/termproxy 5900 --path /nodes/p33ps --perm Sys.Console -- /bin/login -f root

/nodes/p33ps is a dumb path, not sure its either "old" or just wrong. (I'm new @ proxmox please forgive me).

 

[pl11x]

You run on a very old Kernel, seems you didn't reboot for a long long time, he?

You realy need a reboot of that box, do it in the next maintenance window.

What should help for the moment without reboot (while the reboot is still very good!):

systemctl restart pveproxy
systemctl restart pvedaemon

As root please run:

/usr/bin/termproxy 5900 --path /nodes/p33ps --perm Sys.Console -- /bin/login -f root

/nodes/p33ps is a dumb path, not sure its either "old" or just wrong. (I'm new @ proxmox please forgive me).

Hey,

Thanks for reply. It was rebooted when I updated it to Proxmox v5 (not quite long ago). I'll update kernel now!

Regarding the code above (/nodes/p33ps is correct ):

/usr/bin/termproxy 5900 --path /nodes/p33ps --perm Sys.Console -- /bin/login -f root

The output is:

timed out waiting for client

Seems very strange...

 

[r.jochum]

timed out waiting for client is good, try that command and nc to port 5900.

Then you should get Auth error which is fine.

 

[pl11x]

timed out waiting for client is good, try that command and nc to port 5900.

Then you should get Auth error which is fine.

Hmm, what do you mean exactly? Could you explain a bit more?

 

[r.jochum]

Timeout means no connection in x seconds, which is good.

You could try to start that command (termproxy) and start nc in another terminal to it.

 

[r.jochum]

After thinking again about your problem, is it maybe a problem with the client pointing to the wrong url (when you open xterm.js for example?) I don't think that's wrong but that could cause a timeout.

 

[r.jochum]

'/bin/nc -l -p 5903 -w 10 -c '/usr/sbin/qm vncproxy 100' <-- works fine for me outputs the standard VNC header when i telnet to it.

please see the different "-c" from "-e" else nc means its a binary path.

Will ask Dominik tomorrow what he thinks.

 

[pl11x]

Timeout means no connection in x seconds, which is good.

You could try to start that command (termproxy) and start nc in another terminal to it.

If I exec termproxy in terminal1 and then nc -l -p 5900 in terminal2, the results are:

term1: timed out waiting for client
term2: nc: bind to source 0.0.0.0 5900 failed: Address already in use

However, if term2 first and then term1, it returns:

term1: failed to open socket: Address already in use
term2: nothing, it keeps listening

After thinking again about your problem, is it maybe a problem with the client pointing to the wrong url (when you open xterm.js for example?) I don't think that's wrong but that could cause a timeout.

It could be a possibility, because I am doing an inverse proxy pointing 8006 to 443, but I am totally lost on how to fix this.

 

[pl11x]

'/bin/nc -l -p 5903 -w 10 -c '/usr/sbin/qm vncproxy 100' <-- works fine for me outputs the standard VNC header when i telnet to it.

please see the different "-c" from "-e" else nc means its a binary path.

Will ask Dominik tomorrow what he thinks.

nc: connection timed out is the output in my case

 

[r.jochum]

@pl11x give more Infos about your environment, especialy how you connect to ya pve box. After rereading your first Post you say everything works as expected from LAN but not from WAN wheres the difference?

 

[pl11x]

@r.jochum the difference between both is:

LAN -> you connect through https://localip:8006
WAN -> you connect from https://domain.ltd:8006

In my case, to avoid to open the port, I created a subdomain https://pve.domain.ltd, and here is the problem. I can navigate and interact with Proxmox web UI without problems except with console/xterm functionality. If I open the port and enter to https://pve.domain.ltd:8006, noVNC/xterm works successfully.

Regarding environment info, Proxmox is running on a HP Gen8 homeserver as main OS, at home. Feel free to ask for specific information, I don't know exactly what more to say

 

[dcsapak]

so you have some kind of reverse proxy inbetween? if yes, it should be configured to also proxy websocket connections (this is not default)

 

[pl11x]

so you have some kind of reverse proxy inbetween? if yes, it should be configured to also proxy websocket connections (this is not default)

Omg that was all! I've configured it and everything works quite well now!

Thank you so much both!

 

[roboboard]

Omg that was all! I've configured it and everything works quite well now!

Thank you so much both!

This is possible to explain, I have the same problem.
Thank you.

 

[pl11x]

Hi @roboboard,

What I simply did is to configure Websocket connections in my reverse proxy service (nginx, Apache, HAProxy, Traefik, whatever). I've no access to the old config as I entirely changed my lab architecture, sorry!