[SOLVED] Web GUI does not display graphs and vm names after an update from 6.1-1 to 6.1-2

[fester]

Hello,

After an auto update this night, I have a problem with my web GUI that does not display anymore names of vm and their related graphs.

I did:

apt update apt dist-upgrade

service rrdcached stop rrdcached -P FLUSHALL service rrdcached start

systemctl pvedaemon restart systemctl pvedaemon status

● pvedaemon.service - PVE API Daemon
Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2020-04-03 14:36:16 CEST; 5s ago
Process: 26806 ExecStart=/usr/bin/pvedaemon start (code=exited, status=0/SUCCESS)
Main PID: 26832 (pvedaemon)
Tasks: 4 (limit: 4915)
Memory: 127.1M
CGroup: /system.slice/pvedaemon.service
├─26832 pvedaemon
├─26833 pvedaemon worker
├─26834 pvedaemon worker
└─26835 pvedaemon worker

Apr 03 14:36:13 brix systemd[1]: Starting PVE API Daemon...
Apr 03 14:36:16 brix pvedaemon[26832]: starting server
Apr 03 14:36:16 brix pvedaemon[26832]: starting 3 worker(s)
Apr 03 14:36:16 brix pvedaemon[26832]: worker 26833 started
Apr 03 14:36:16 brix pvedaemon[26832]: worker 26834 started
Apr 03 14:36:16 brix pvedaemon[26832]: worker 26835 started
Apr 03 14:36:16 brix systemd[1]: Started PVE API Daemon.

systemctl pveproxy restart systemctl pveproxy status

● pveproxy.service - PVE API Proxy Server
Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2020-04-03 14:42:12 CEST; 10s ago
Process: 27527 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
Process: 27539 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
Main PID: 27540 (pveproxy)
Tasks: 4 (limit: 4915)
Memory: 140.7M
CGroup: /system.slice/pveproxy.service
├─27540 pveproxy
├─27541 pveproxy worker
├─27542 pveproxy worker
└─27543 pveproxy worker

Apr 03 14:42:08 brix systemd[1]: Starting PVE API Proxy Server...
Apr 03 14:42:12 brix pveproxy[27540]: starting server
Apr 03 14:42:12 brix pveproxy[27540]: starting 3 worker(s)
Apr 03 14:42:12 brix pveproxy[27540]: worker 27541 started
Apr 03 14:42:12 brix pveproxy[27540]: worker 27542 started
Apr 03 14:42:12 brix pveproxy[27540]: worker 27543 started
Apr 03 14:42:12 brix systemd[1]: Started PVE API Proxy Server.

However, it seems to be related to pvestatd as a restart restores vm's names, but not graphs.
But after a while, names disappears again (and status of pvedaemon, pveproxy & pvestatd stay OK)...

systemctl pvestatd restart systemctl pvestatd status

● pvestatd.service - PVE Status Daemon
Loaded: loaded (/lib/systemd/system/pvestatd.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2020-04-03 14:47:41 CEST; 5min ago
Process: 28282 ExecStart=/usr/bin/pvestatd start (code=exited, status=0/SUCCESS)
Main PID: 28292 (pvestatd)
Tasks: 3 (limit: 4915)
Memory: 123.2M
CGroup: /system.slice/pvestatd.service
├─28292 pvestatd
├─28336 /bin/mount -t nfs 192.168.1.15:/export/proxmox /mnt/pve/NAS
└─28337 /sbin/mount.nfs 192.168.1.15:/export/proxmox /mnt/pve/NAS -o rw

Apr 03 14:47:39 brix systemd[1]: Starting PVE Status Daemon...
Apr 03 14:47:41 brix pvestatd[28292]: starting server
Apr 03 14:47:41 brix systemd[1]: Started PVE Status Daemon.

I also tried to empty cache of Firefox and changed browser, and reboots, no change.
An idea? Thanks for your help!

My configuration:

proxmox-ve: 6.1-2 (running kernel: 5.3.18-3-pve)
pve-manager: 6.1-8 (running version: 6.1-8/806edfe1)
pve-kernel-helper: 6.1-7
pve-kernel-5.3: 6.1-6
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.18-2-pve: 5.3.18-2
pve-kernel-5.3.18-1-pve: 5.3.18-1
pve-kernel-5.3.10-1-pve: 5.3.10-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.3-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.15-pve1
libpve-access-control: 6.0-6
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.0-17
libpve-guest-common-perl: 3.0-5
libpve-http-server-perl: 3.0-5
libpve-storage-perl: 6.1-5
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 3.2.1-1
lxcfs: 4.0.1-pve1
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.1-3
pve-cluster: 6.1-4
pve-container: 3.0-23
pve-docs: 6.1-6
pve-edk2-firmware: 2.20200229-1
pve-firewall: 4.0-10
pve-firmware: 3.0-6
pve-ha-manager: 3.0-9
pve-i18n: 2.0-4
pve-qemu-kvm: 4.1.1-4
pve-xtermjs: 4.3.0-1
qemu-server: 6.1-7
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.3-pve1

 

[Stoiko Ivanov]

├─28292 pvestatd
          ├─28336 /bin/mount -t nfs 192.168.1.15:/export/proxmox /mnt/pve/NAS
          └─28337 /sbin/mount.nfs 192.168.1.15:/export/proxmox /mnt/pve/NAS -o rw

just on a hunch - could it be that your storage named NAS is unreachable or that the nfs-mount hangs?

- check the complete journal since boot (`journalctl -b`)

hanging nfs mounts can cause such problems

I hope this helps!

 

[fester]

just on a hunch - could it be that your storage named NAS is unreachable or that the nfs-mount hangs?

- check the complete journal since boot (`journalctl -b`)

hanging nfs mounts can cause such problems

Exactly!!! I found that in the journal:

Apr 03 11:17:28 brix kernel: nfs: server 192.168.1.15 not responding, still trying

NFS sharing of my NAS (Openmediavault) indeed turned to be inoperant as I could not mount manually NAS in pve.
I did yesterday some reorganization on my NAS and probably something went wrong.

So I have inactivated/reactivated NFS sharing in OMV, and then I could do in pve:
mount -t nfs 192.168.1.15:/export/proxmox /mnt/pve/NAS

And vm names & graphs came back immediately

I have rebooted pve to be sure and everything is OK. Nothing to do with 6.1-1 to 6.1-2 update, just a coincidence...

Thanks so much Stoiko for your help!

 

[Stoiko Ivanov]

Glad you found the issue!

please mark the thread as 'SOLVED' - this can help other users in similar situations
Thanks!