[SOLVED] Web GUI not working, unknown issues after power outage

tallguyjenks

New Member
Apr 4, 2022
9
0
1
www.bryanjenks.dev
Im at my wits end. I've scoured the forum for solutions and no commands are helping and i have no idea what in the hell is wrong with the system and my knowledge of linux internals is not up to this task.

Ran
Bash:
systemctl status pve-cluster pveproxy pvedaemon

1.png
2.png

Code:
pveversion -v

3.png4.png

I've tried:

Code:
apt update
apt install --reinstall pve-manager proxmox-widget-toolkit
apt upgrade
apt full-upgrade
apt install pve-kernal-helper
systemctl restart pve-cluster.service

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Code:
apt update

6.png

Code:
apt install --reinstall pve-manager proxmox-widget-toolkit

5.png

Code:
apt full-upgrade

7.png8.png9.png
10.png
 
I've also tried pieces of suggestions from these posts:

And still i have no web GUI, my services WERE running despite the GUI earlier and now i don't even have that, and all this happened after a sudden blackout where my UPS didn't take over for some reason.
 
and what does "journalctl -xe" say?
you can also try: pvecm updatecerts --force

you also appear to have some disk corruption and may need to run fsck on your drives.


Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
Code:
journalctl -xe

output is almost 1200 lines, but the red that keeps popping up is:

Code:
/etc/pve/local/pve-ss1.key: failed to load local private key (key_file or key) at /usr/share/per 15/PVE/APIServer/AnyEvent.pm line 1943.

15.png

Code:
pvecm updatecerts --force

yields:

14.png

i tried running fsck on a drive and I'm not sure what I'm doing with that command or how to repair the disk :(

16.png
 
working on it and making progress

running fsck on the disks seemed to work, i went a different route through the menus and they made more sense and actually wrote the changes so that worked

next i noticed that the problem of the SSL keys error seemed to be that the /etc/pve directory was not mounting and then that led to discovery that the /var/lib/pve-cluster/config.db was malformed and so i renamed with a .bak added to it and ran `pmxcfs -l` and that worked then ran `systemctl restart pve-cluster` and then checked `journalctl -xe` and the output looks a bit more promising:

17.png
 
ran `pvecm updatecerts` and it worked this time!
then ran `systemctl restart pve-cluster` but now the error is just

Code:
[main] crit: unable to acquire pmxcfs lock: Resource temporarily unavailable
 
reboot has fixed it! i have web GUI again

have to repair the /var/lib/pve-cluster/config.db file though as it looks like my node is now a blank slate and even though i have backups of the LXC's and VM's I'm not sure if my notes and other settings are accessible, anyone know anything to help me there?
 
looks like all i lost is a little configuration stuff like some storage settings and notes on the node object, restoring items from backups pretty much fixes the issue for me, calling this solved
 
@tallguyjenks hey, in yours firsts screenshots, when your're upgrading, you've a read-only SDB2 referenced. Reboot has re-mounted all in RW.

But your problem come of this read-only state. Check why your FS has gone read-only mode ;)