Lost console access to container after cluster reboot running PVE 6.0

[Rosario Contarino]

Container is still up and running, accessible via IP but console is blank, nothing shown up.
Keyboard within console works, pressed keys show but that's it, no prompt, no nothing.

Any idea?

Thank you
Rosario

 

[Chris]

HI,
have you tried to clear the browser cache? This sometimes solves issues.

 

[Rosario Contarino]

HI,
have you tried to clear the browser cache? This sometimes solves issues.

Yes, I tried also with different browsers (Firefox, IE, Edge, Chrome), different platforms (MacOS, Win 10 En., Fedora) and different Java versions.

To be clear, it is only the console for the container. The console for the VM works fine.

 

[Chris]

Can you enter the container with a `pct enter CTID`? What about `pct console CTID`? Is it just one container or all of them?

 

[Rosario Contarino]

Can you enter the container with a `pct enter CTID`? Is it just one container or all of them?

What's that? Can you please elaborate?

I have only created one container. I can create more and then test if this might help to debug.

 

[Chris]

What's that? Can you please elaborate?

I have only created one container. I can create more and then test if this might help to debug.

You can get a shell/console in the container by running these commands on the command line of your PVE host.

 

[Rosario Contarino]

You can get a shell/console in the container by running these commands on the command line of your PVE host.

Hi Chris, 'pct enter CTID' worked just fine.
Still the console through the WEB doesn't.
Any log I can extract to let you identify the issue?

Best regards
Rosario

 

[Chris]

Here are some suggestions on what to try next:
* Are you using noVNC or xterm.js? Please try with both.
* Do you get any error in the task log when starting the console?
* What is the output of `journalctl -u pvedaemon`?
* What is the output of `journalctl -u pveproxy`?
* Please try to restart pveproxy and pvedaemon by running: `systemctl restart pvedaemon.service pveproxy.service`. Does this affect the issue?
* Are there any ad-blockers, browser plugins, etc. which might interfere (although you stated you tried with different systems/browsers)?
* Try with a new/different container. Is the issue still there?
Hope this will help to close in on the problem.

 

[Rosario Contarino]

Here are some suggestions on what to try next:
* Are you using noVNC or xterm.js? Please try with both.

Done. Same issue

* Do you get any error in the task log when starting the console?

No errors.

* What is the output of `journalctl -u pvedaemon`?

Aug 16 09:29:44 pve-01 pvedaemon[5058]: <root@pam> starting task UPIDve-01:00002178:00006330:5D565B68:vzstart:101:root@pam:

Aug 16 09:30:32 pve-01 pvedaemon[5058]: <root@pam> end task UPIDve-01:00002178:00006330:5D565B68:vzstart:101:root@pam: OK

Aug 16 10:23:49 pve-01 pvedaemon[5057]: unable to open file '/var/lib/pve-manager/apl-info/releases.turnkeylinux.org' - No such file or directory

Aug 16 10:23:49 pve-01 pvedaemon[5057]: <root@pam> starting task UPIDve-01:000103CC:00055693:5D566815:download::root@pam:

Aug 16 10:26:06 pve-01 pvedaemon[5057]: <root@pam> end task UPIDve-01:000103CC:00055693:5D566815:download::root@pam: OK​

* What is the output of `journalctl -u pveproxy`?

Aug 16 09:25:56 pve-01 systemd[1]: Started PVE API Proxy Server.

Aug 16 09:30:15 pve-01 pveproxy[5963]: proxy detected vanished client connection

Aug 16 10:23:20 pve-01 pveproxy[5962]: unable to open file '/var/lib/pve-manager/apl-info/releases.turnkeylinux.org' - No such file or directory

Aug 16 11:08:35 pve-01 pveproxy[5964]: worker exit​

* Please try to restart pveproxy and pvedaemon by running: `systemctl restart pvedaemon.service pveproxy.service`. Does this affect the issue?

From $> gournalctl -u pvedaemon

Aug 16 11:35:46 pve-01 pvedaemon[5047]: server stopped

Aug 16 11:35:46 pve-01 pvedaemon[149575]: got inotify poll request in wrong process - disabling inotify

Aug 16 11:35:47 pve-01 systemd[1]: pvedaemon.service: Succeeded.

Aug 16 11:35:47 pve-01 systemd[1]: Stopped PVE API Daemon.

Aug 16 11:35:47 pve-01 systemd[1]: Starting PVE API Daemon...

Aug 16 11:35:47 pve-01 pvedaemon[149639]: starting server

Aug 16 11:35:47 pve-01 pvedaemon[149639]: starting 3 worker(s)

Aug 16 11:35:47 pve-01 pvedaemon[149639]: worker 149640 started

Aug 16 11:35:47 pve-01 pvedaemon[149639]: worker 149641 started

Aug 16 11:35:47 pve-01 pvedaemon[149639]: worker 149642 started

Aug 16 11:35:47 pve-01 systemd[1]: Started PVE API Daemon.
​

* Are there any ad-blockers, browser plugins, etc. which might interfere (although you stated you tried with different systems/browsers)?

Nope.

* Try with a new/different container. Is the issue still there?

Just created another container. This new one seems to be working but to do a comparative test I should shut down the cluster and restart the cluster. As soon as I will do it I will edit this post and update.

Hope this will help to close in on the problem.

 

[jim.bond.9862]

I have similar issue but I have no console at all. Not for CT not the host.

I run proxmox 5.4.x, not sure what the exact version right now.
All is up and running but when I go to web interface I get no consoles at all. Black screens all around.
I can ssh into any machines. Host and containers just fine. Sometimes if I start a noVNC pop-up window I get console but within the browser it just a black screen with blinking cursor.

 

[Chris]

What happens if you simply press enter within the empty console window? Do you then get a prompt?

 

[jim.bond.9862]

No. The cursor just jumps to next line as if I am typing in text editor.
When I try a pop-up noVNC it sometimes works or at times duplicates the black screen in the pop-up.
Ssh from remote machine always works fine.

 

[Chris]

Quick input from a colleague suggests that this could be related to the firewall. Please check if the firewall is enabled and if so if the problem persists with the CT firewall disabled.

 

[pixelbaker]

Experiencing the same issue on ProxMox 6.0-11. Blank console screen when trying to use NoVNC with any container in the cluster (2 nodes). All VM and node consoles are working correctly, services running inside containers working well, SSH into host then pct to connect to container works fine. Hitting enter just line feeds in the console. Disabling firewall for the container causes no change. Only fix so far is to restart the container.