Can't access proxmox GUI

joyjoe

New Member
Dec 16, 2022
14
0
1
Hello everyone, I can't access my Proxmox GUI anymore since a power outage. Unfortunately I do not have a monitor attached to it but I can successfully SSH into it. Could please someone help me troubleshoot the problem?

Thank you.
 
Hi,
please check the status of systemctl status pveproxy.service pvedaemon.service, any errors in the journal? journalctl -b -r gives you a paged output of the journal since boot in reverse.
 
pveproxy.service and pvedaemon.service are active and running. Tons of errors in the journal:
mar 27 14:33:39 hypervisor tailscaled[3856]: [RATELIMIT] format("control: trying bootstrapDNS(%q, %q) for %q ...")
mar 27 14:33:39 hypervisor tailscaled[3856]: control: trying bootstrapDNS("derp23c.tailscale.com", "") for "controlplane.tailscale.com" ...
mar 27 14:33:39 hypervisor tailscaled[3856]: [RATELIMIT] format("control: bootstrapDNS(%q, %q) for %q error: %v")
mar 27 14:33:39 hypervisor tailscaled[3856]: control: bootstrapDNS("derp11c.tailscale.com", ":861") for "controlplane.tailscale.com" error: Get "https://derp11c.tailscale.com/bootstrap-dns?q=controlplane.tailscale.com": dial tcp []:443: connect: netw>
mar 27 14:33:39 hypervisor tailscaled[3856]: control: trying bootstrapDNS("derp11c.tailscale.com", ":861") for "controlplane.tailscale.com" ...
mar 27 14:33:39 hypervisor tailscaled[3856]: control: bootstrapDNS("derp4h.tailscale.com", "") for "controlplane.tailscale.com" error: Get "https://derp4h.tailscale.com/bootstrap-dns?q=controlplane.tailscale.com": context deadline exceeded
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-storage/hypervisor/VMs: /var/lib/rrdcached/db/pve2-storage/hypervisor/VMs: illegal attempt to update using time 1679920418 when last update time is 1689812379 (minimum one seco>
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-storage/hypervisor/VMs: -1
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-storage/hypervisor/Storage: /var/lib/rrdcached/db/pve2-storage/hypervisor/Storage: illegal attempt to update using time 1679920418 when last update time is 1689812379 (minimum >
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-storage/hypervisor/Storage: -1
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-storage/hypervisor/local: /var/lib/rrdcached/db/pve2-storage/hypervisor/local: illegal attempt to update using time 1679920418 when last update time is 1689812379 (minimum one >
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-storage/hypervisor/local: -1
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-storage/hypervisor/Backup: /var/lib/rrdcached/db/pve2-storage/hypervisor/Backup: illegal attempt to update using time 1679920418 when last update time is 1689812379 (minimum on>
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-storage/hypervisor/Backup: -1
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/108: /var/lib/rrdcached/db/pve2-vm/108: illegal attempt to update using time 1679920418 when last update time is 1689812378 (minimum one second step)
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/108: -1
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/102: /var/lib/rrdcached/db/pve2-vm/102: illegal attempt to update using time 1679920418 when last update time is 1689812378 (minimum one second step)
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/102: -1
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/100: /var/lib/rrdcached/db/pve2-vm/100: illegal attempt to update using time 1679920418 when last update time is 1689812378 (minimum one second step)
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/100: -1
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/110: /var/lib/rrdcached/db/pve2-vm/110: illegal attempt to update using time 1679920418 when last update time is 1689812378 (minimum one second step)
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/110: -1
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/101: /var/lib/rrdcached/db/pve2-vm/101: illegal attempt to update using time 1679920418 when last update time is 1689812378 (minimum one second step)
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/101: -1
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/109: /var/lib/rrdcached/db/pve2-vm/109: illegal attempt to update using time 1679920418 when last update time is 1689812388 (minimum one second step)
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/109: -1
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/107: /var/lib/rrdcached/db/pve2-vm/107: illegal attempt to update using time 1679920418 when last update time is 1689812378 (minimum one second step)
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/107: -1
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/105: /var/lib/rrdcached/db/pve2-vm/105: illegal attempt to update using time 1679920418 when last update time is 1689812388 (minimum one second step)
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/105: -1
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-vm/106: /var/lib/rrdcached/db/pve2-vm/106: illegal attempt to update using time 1679920418 when last update time is 1689812378 (minimum one second step)
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/106: -1
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-node/hypervisor: /var/lib/rrdcached/db/pve2-node/hypervisor: illegal attempt to update using time 1679920418 when last update time is 1689812388 (minimum one second step)
mar 27 14:33:38 hypervisor pmxcfs[3770]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-node/hypervisor: -1
mar 27 14:33:36 hypervisor tailscaled[3856]: control: trying bootstrapDNS("derp4h.tailscale.com", "") for "controlplane.tailscale.com" ...
mar 27 14:33:36 hypervisor tailscaled[3856]: control: bootstrapDNS("derp18c.tailscale.com", "") for "controlplane.tailscale.com" error: Get "https://derp18c.tailscale.com/bootstrap-dns?q=controlplane.tailscale.com": dial tcp []:443: connect: ne>
mar 27 14:33:36 hypervisor tailscaled[3856]: control: trying bootstrapDNS("derp18c.tailscale.com", "") for "controlplane.tailscale.com" ...
mar 27 14:33:36 hypervisor tailscaled[3856]: control: bootstrapDNS("derp8f.tailscale.com", "") for "controlplane.tailscale.com" error: Get "https://derp8f.tailscale.com/bootstrap-dns?q=controlplane.tailscale.com": dial tcp :443: connect: no route to ho>
mar 27 14:33:36 hypervisor tailscaled[3856]: control: trying bootstrapDNS("derp8f.tailscale.com", "") for "controlplane.tailscale.com" ...
mar 27 14:33:36 hypervisor tailscaled[3856]: control: bootstrapDNS("derp9e.tailscale.com", "") for "controlplane.tailscale.com" error: Get "https://derp9e.tailscale.com/bootstrap-dns?q=controlplane.tailscale.com": dial tcp []:443: connect: ne>
mar 27 14:33:36 hypervisor tailscaled[3856]: control: trying bootstrapDNS("derp9e.tailscale.com", "") for "controlplane.tailscale.com" ...
mar 27 14:33:36 hypervisor tailscaled[3856]: control: bootstrapDNS("derp8d.tailscale.com", "") for "controlplane.tailscale.com" error: Get "https://derp8d.tailscale.com/bootstrap-dns?q=controlplane.tailscale.com": context deadline exceeded
mar 27 14:33:36 hypervisor tailscaled[3856]: [RATELIMIT] format("control: bootstrapDNS(%q, %q) for %q error: %v") (6 dropped)
mar 27 14:33:33 hypervisor tailscaled[3856]: control: trying bootstrapDNS("derp8d.tailscale.com", ) for "controlplane.tailscale.com" ...
mar 27 14:33:33 hypervisor tailscaled[3856]: [RATELIMIT] format("control: trying bootstrapDNS(%q, %q) for %q ...") (6 dropped)
 
Last edited:
All my VMs are down, including the ones that should autostart, I suppose that if it was a tailscale issue for GUI access, the VMs should autostart without any problem, since it's not a related problem. Could I be right?
 
this may or may not be connected at all. There is not enough information to draw any conclusions, except that we know that GUI is in fact responding on "localhost".
You can also try the same command, from PVE ssh session, with your LAN IP: curl -sk https://[192.168.x.x]:8006|grep title

The second portion of error messages refers to pmxcfs having trouble updating statistics database due to time mismatch. You may address it by using one of the methods here: https://forum.proxmox.com/threads/rrdc-and-rrd-update-errors.76219/#post-345702

Examine you boot journal log in detail. Remove the chatter you already understand with grep: journalctl -b -r |grep -vi tailscale


Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
pveproxy.service and pvedaemon.service are active and running. Tons of errors in the journal:
the illegal attempt to update using time 1679920418 when last update time is 1689812379 errors might indicate that you have a time sync issue. Is chrony up and running systemctl status chrony.service. What exact error do you get when connecting to the webui via browser?
curl -sk https://[192.168.x.x]:8006|grep title
Please check the output of this command as suggested by @bbgeek17 but use `-v` instead of `-s` and without the pipe to grep to see and check the TLS handshake output.
 
SOLVED IT! I was very suspicious about that time difference, so I dug deeper. It was just the f****ing BIOS that reset, changing the time AND defaulting to IOMMU off -.-"

Thank you everyone for helping me!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!