rrd update failed: time in past, proxmox backup server installed as VM on TrueNAS Scale

[sukerman]

** UPDATE - Solution found **

root@backup:/var/lib/proxmox-backup/rrdb# ls -l
total 188
drwxr-xr-x 6 backup backup 4096 Jan 22 15:55 datastore
drwxr-xr-x 2 backup backup 4096 Jan 24 2024 host
-rw-r--r-- 1 backup backup 177595 Jan 24 13:17 rrd.journal

I removed rrd.journal and rebooted, it did not help, so I went nuclear and deleted everything inside rrdb dir and now its working again.
Obvs. you will lose all you old graphs if you do this.

****************

Original Problem:

I have pbs installed in a VM on truenas, graphs etc were all working to begin with.

My log is full of this :

Jan 24 12:55:21 backup proxmox-backup-[506]: backup proxmox-backup-proxy[506]: rrd update failed: time in past (1706097321.386906 < 1708563725.8764591)

1706097321.386906 is GMT: Wednesday, 24 January 2024 11:55:21.386

1708563725.8764591 is GMT: Thursday, 22 February 2024 1:02:05.876

This seems to be because at one point my HW clock was set to February, its not anymore, but I can't seem to reset this data.

rrdcached is not installed, there is no data in /var/lib/rrdcached/db.

I have installed it but no data is created so theres nothing to delete.

I have installed qemu-guest-agent.

Where is this bad data being stored and how can I reset it?

Thanks,

Jack

 

[lucius_the]

Thanks !
Was having the same issue on PBS 3.2-7.

Did:

rm -fR /var/lib/proxmox-backup/rrdb/*

and rebooted the VM. That fixed it. Thanks again

 

[gildra]

I also have PBS as VM in TrueNAS Core, and this did not solve the issue permanently for me. After a reboot of the VM the issue reappeared.

The solution for me was to edit the "proxmox-backup-proxy.service" and add following:

[Unit]
After=time-sync.target
Wants=time-sync.target

[Service]
ExecStartPre=/bin/sleep 10

This delays the start of the service to make sure the time service is started and chrony is synched. This solved the issue so far and it has not reappeared. It must be said that this solution is not very elegant and is probably does not solve the root cause of the issue.

I would be interested in better solutions.

 

[herndl]

In my case it is running on a QNAP NAS and it started after the update I did after the initial installation.
The time was 2hours ahead, so I changed to the fitting timezone and did a restart. After the restart everything was ok and I set the timezone back to my timezone. Since that it seems to be fine again.
Is this the solution?