[SOLVED] Web GUI not working

ChokoBG

New Member
Mar 19, 2023
4
1
1
Hello All,
My Proxmox VE Web Interface stopped working all of a sudden.
I have made the following investigation:

1. Output from # ip -a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
link/ether 18:66:da:22:21:2d brd ff:ff:ff:ff:ff:ff
3: wlp3s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether b8:8a:60:52:b6:b5 brd ff:ff:ff:ff:ff:ff
4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 18:66:da:22:21:2d brd ff:ff:ff:ff:ff:ff
inet 192.168.0.193/24 scope global vmbr0
valid_lft forever preferred_lft forever
inet6 fe80::1a66:daff:fe22:212d/64 scope link
valid_lft forever preferred_lft forever
2. Output from # ss -antlp | grep 8006
LISTEN 4 4096 *:8006 *:* users:(("pveproxy worker",pid=53317,fd=6),("pveproxy worker",pid=53316,fd=6),("pveproxy worker",pid=53315,fd=6),("pveproxy",pid=990,fd=6))
3. Output from # cat /etc/network/interfaces

iface vmbr0 inet static
address 192.168.0.193/24
gateway 192.168.0.1
bridge-ports enp2s0
bridge-stp off
bridge-fd 0
4. Output from # systemctl status pveproxy.service

pveproxy.service - PVE API Proxy Server
Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2023-04-11 20:15:15 EEST; 2h 17min ago
Process: 987 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=111)
Process: 988 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
Main PID: 990 (pveproxy)
Tasks: 4 (limit: 9303)
Memory: 203.5M
CPU: 4min 16.939s
CGroup: /system.slice/pveproxy.service
├─ 990 pveproxy
├─54691 pveproxy worker
├─54692 pveproxy worker
└─54693 pveproxy worker
Apr 11 22:32:13 proxmox-srv1 pveproxy[990]: worker 54654 finished
Apr 11 22:32:13 proxmox-srv1 pveproxy[990]: starting 1 worker(s)
Apr 11 22:32:13 proxmox-srv1 pveproxy[990]: worker 54692 started
Apr 11 22:32:13 proxmox-srv1 pveproxy[54655]: worker exit
Apr 11 22:32:13 proxmox-srv1 pveproxy[54691]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServe>
Apr 11 22:32:13 proxmox-srv1 pveproxy[54692]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServe>
Apr 11 22:32:13 proxmox-srv1 pveproxy[990]: worker 54655 finished
Apr 11 22:32:13 proxmox-srv1 pveproxy[990]: starting 1 worker(s)
Apr 11 22:32:13 proxmox-srv1 pveproxy[990]: worker 54693 started
Apr 11 22:32:13 proxmox-srv1 pveproxy[54693]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServe>
5. Output from # systemctl status pvedaemon.service

pvedaemon.service - PVE API Daemon
Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2023-04-11 20:15:13 EEST; 2h 20min ago
Process: 974 ExecStart=/usr/bin/pvedaemon start (code=exited, status=0/SUCCESS)
Main PID: 981 (pvedaemon)
Tasks: 4 (limit: 9303)
Memory: 148.0M
CPU: 1.786s
CGroup: /system.slice/pvedaemon.service
├─981 pvedaemon
├─982 pvedaemon worker
├─983 pvedaemon worker
└─984 pvedaemon worker
Apr 11 20:15:12 proxmox-srv1 systemd[1]: Starting PVE API Daemon...
Apr 11 20:15:13 proxmox-srv1 pvedaemon[981]: starting server
Apr 11 20:15:13 proxmox-srv1 pvedaemon[981]: starting 3 worker(s)
Apr 11 20:15:13 proxmox-srv1 pvedaemon[981]: worker 982 started
Apr 11 20:15:13 proxmox-srv1 pvedaemon[981]: worker 983 started
Apr 11 20:15:13 proxmox-srv1 pvedaemon[981]: worker 984 started
Apr 11 20:15:13 proxmox-srv1 systemd[1]: Started PVE API Daemon.
6. Output from # cat /etc/hosts

127.0.0.1 localhost.localdomain localhost
192.168.0.193 proxmox-server.bulnet proxmox-server
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
7. Output from # journalctl -f
Apr 11 23:04:44 proxmox-srv1 pveproxy[67707]: worker exit
Apr 11 23:04:44 proxmox-srv1 pveproxy[67708]: worker exit
Apr 11 23:04:44 proxmox-srv1 pveproxy[990]: worker 67707 finished
Apr 11 23:04:44 proxmox-srv1 pveproxy[990]: worker 67708 finished
Apr 11 23:04:44 proxmox-srv1 pveproxy[990]: starting 2 worker(s)
Apr 11 23:04:44 proxmox-srv1 pveproxy[990]: worker 67741 started
Apr 11 23:04:44 proxmox-srv1 pveproxy[990]: worker 67742 started
Apr 11 23:04:44 proxmox-srv1 pveproxy[67709]: worker exit
Apr 11 23:04:44 proxmox-srv1 pveproxy[67741]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1996.
Apr 11 23:04:44 proxmox-srv1 pveproxy[67742]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1996.
Apr 11 23:04:44 proxmox-srv1 pveproxy[990]: worker 67709 finished
Apr 11 23:04:44 proxmox-srv1 pveproxy[990]: starting 1 worker(s)
Apr 11 23:04:44 proxmox-srv1 pveproxy[990]: worker 67743 started
Apr 11 23:04:44 proxmox-srv1 pveproxy[67743]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1996.


Here is the Output from the line 1996 as per journalctl # cat -n AnyEvent.pm

1989 }
1990 # workaround until anyevent supports disabling TLS 1.3 directly
1991 if (exists($self->{ssl}->{tlsv1_3}) && !$self->{ssl}->{tlsv1_3}) {
1992 $tls_ctx_flags |= &Net::SSLeay::OP_NO_TLSv1_3;
1993 }
1994
1995
1996 $self->{tls_ctx} = AnyEvent::TLS->new(%{$self->{ssl}});
1997 Net::SSLeay::CTX_set_options($self->{tls_ctx}->{ctx}, $tls_ctx_flags);
1998 if (defined($ciphersuites)) {
1999 warn "Failed to set TLS 1.3 ciphersuites '$ciphersuites'\n"
2000 if !Net::SSLeay::CTX_set_ciphersuites($self->{tls_ctx}->{ctx}, $ciphersuites);
2001 }
2002 }
2003

I saw that there is problem with the ssl-key, but I havent changed anything. And I have checked the directory from where it is searching the key from, but there is no such directory on my machine.

root@proxmox-srv1:/etc/pve# ls -all
total 8
drwxr-xr-x 2 root root 4096 Mar 22 00:48 .
drwxr-xr-x 92 root root 4096 Apr 11 20:14 ..
And I've checked here, because I didn't know where and what to search for.
root@proxmox-srv1:/usr/share/perl5/PVE/APIServer# ls -all
total 80
drwxr-xr-x 3 root root 4096 Apr 11 02:25 .
drwxr-xr-x 21 root root 4096 Apr 11 02:25 ..
-rw-r--r-- 1 root root 59353 Mar 16 17:57 AnyEvent.pm
drwxr-xr-x 2 root root 4096 Apr 11 02:25 Formatter
-rw-r--r-- 1 root root 2150 Mar 16 17:57 Formatter.pm
-rw-r--r-- 1 root root 2667 Mar 16 17:57 Utils.pm

I have SSH Access to the server, I've rebooted the server and restarted the Services without change of any kind.
I don't know where is the problem, can you please help me with this issue?
 

Attachments

  • Screenshot 2023-04-11 230000.png
    Screenshot 2023-04-11 230000.png
    17 KB · Views: 50
standard context of these files:
Code:
cat /etc/hostname
pve7demo1

Code:
cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
172.16.101.174 pve7demo1.localnet pve7demo1

# The following lines are desirable for IPv6 capable hosts

::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts



Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
standard context of these files:
Code:
cat /etc/hostname
pve7demo1

Code:
cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
172.16.101.174 pve7demo1.localnet pve7demo1

# The following lines are desirable for IPv6 capable hosts

::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts



Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
mine is a standard fresh install and looks basically the same, no changes the IP used is reflected on the router administrator, but cannot access the web GUI or even ping the IP address. ping from the shell itself is working but cannot ping other IP on the same network.
 
turn out somehow my Router is the culprit, after moving the cable other router it work, I can reach the web gui.
 
Thank you for all of your help and ideas!

I'm just now starting on Proxmox and still have no clue what I'm doing here after 35+ years in PC hardware. Not much experience on servers but this is the goal! Happy to see the Proxmox community is so kind and strong.

I am in the middle of dealing with a T-mobile 5G access point that won't allow to have its router DHCP off so my Nighthawk could manage it and I don't know how to turn it off. Watched a couple videos on some tricking it to deactivate DHCP but I'm not a code/programing guy (another goal of mine on using the virtual environment) and you must hack into it by injecting the configuration through some C++ code, which I can follow the instructions but no clue on how to upload such to the router in case.

Mine, I did tested the /etc/network/interfaces files to achieve some DHCP going and that was the culprit. Since the misbehaved T-mobile router got into it after I deactivated my Nighthawk for testing, it changed the ip address on Proxmox from 132 to 175, something I didn't ask at all. Using your instructions here, and during ping command I noticed the change. Once I tested with ip 175 it worked, then I went back into the hosts to check the hostnames I'm using and was able to access web UI al well when using the pve:8006 host. So much to learn, so little time!!! But the mistakes are the best teachers and will push us into reaching out to each other. When humans gather together to help one another, we are amazingly creative and positive.

Well, just a great thank you to you all who invest your time and knowledge on helping others!
Have a great weekend!
Citizen5