Graphs quit working after NFS mistake

[frankb00]

Hi all, I have ProxMox version 5.3-6 running as a single node server. There is a NFS storage it mounts for Disk Image and Containers. The other day on the file server I quit exporting this directory while troubleshooting a different share.
Once I fixed my mistake, I came back to find the ProxMox server, all the containers needed to be stopped and started, they couldn't read/write to their file system, big surprise their image file resides on the NFS share, so expected this.
So restarted the containers and the VM's. However I noticed some of my graphs quit incrementing. So after googling and trying different things, like restarting pvestatd, rrdcached, trashing the /var/lib/rrdcached directory etc.. I now have NO graphs period.
To be specific I have graphs on the ProxMox GUI with no data on them, and the date ends on 2019/02/11.
I'd love to fix this, but am out of ideas of what to try.
Thanks for reading, any thoughts are appreciated.

 

[dcsapak]

are there any messages related to rrd etc. in the syslog?

 

[dcsapak]

also, did you restart the host after that? it may be that the containers cgroup directories did not get cleaned and our code now looks in the wrong place

 

[frankb00]

Hi Dominik,

This is the only stuff in syslog when grepping for rrd.

root@nexus:/var/log# grep -i rrd syslog
Mar 14 07:11:43 nexus rrdcached[2431]: flushing old values
Mar 14 07:11:43 nexus rrdcached[2431]: rotating journals
Mar 14 07:11:43 nexus rrdcached[2431]: started new journal /var/lib/rrdcached/journal/rrd.journal.1552561903.797630
Mar 14 07:11:43 nexus rrdcached[2431]: removing old journal /var/lib/rrdcached/journal/rrd.journal.1552554703.797628
Mar 14 08:11:43 nexus rrdcached[2431]: flushing old values
Mar 14 08:11:43 nexus rrdcached[2431]: rotating journals
Mar 14 08:11:43 nexus rrdcached[2431]: started new journal /var/lib/rrdcached/journal/rrd.journal.1552565503.797651
Mar 14 08:11:43 nexus rrdcached[2431]: removing old journal /var/lib/rrdcached/journal/rrd.journal.1552558303.797656

And yes I have restarted the main server a couple times now after all this happened. Right now it is currently up and running from a fresh reboot as of 20 hours ago.

UPDATE:
I see there are some upgrades when doing an "apt upgrade", I'm going to go ahead and run through that and reboot and see what happens..

UPDATE:
Even after doing the "apt upgrade" and rebooting, still no graph updates.