rrd update failed: time in past

I

informatique-itm

New Member
Jul 13, 2022
3
0
1
Hello,
as we had a disk full problem on our backup-server, i checked the directories and noticed that the syslog and daemon.log were becoming extremely big because of following messages:

Aug 4 14:12:19 bak-server proxmox-backup-proxy[575]: rrd update failed: time in past (1659615139.1970675 < 1651659550067.6885)
Aug 4 14:12:29 bak-server proxmox-backup-proxy[575]: rrd update failed: time in past (1659615149.19885 < 1651659550067.6885)
Aug 4 14:12:29 bak-server proxmox-backup-proxy[575]: write rrd data back to disk
Aug 4 14:12:29 bak-server proxmox-backup-proxy[575]: starting rrd data sync
Aug 4 14:12:29 bak-server proxmox-backup-proxy[575]: rrd journal successfully committed (18 files in 0.038 seconds)
Aug 4 14:12:39 bak-server proxmox-backup-proxy[575]: rrd update failed: time in past (1659615159.1999662 < 1651659550067.6885)

This keeps going on every 10 secs until a backup is performed; then i get the backup messages in syslog
and when the backup is done, the "time in past"-errors are coming again.

The server time is set correctly; against what time is the check performed that leads to these errors ?

Regards,
Gaston
 
Last edited:
Do you use systemd-timesyncd or chrony?
If you use systemd-timesyncd I'd recommend installing chrony with apt install chrony.

It provides a complete NTP implementation, while systemd-timesyncd just provides SNTP.
 
I have same problem with proxmox backup server. Chrony is installed, but frequently same error persists. What else can we try?
 
Make sure that chrony runs periodically so that the time is always kept up-to-date.

Is PBS running as VM or bare metal?
If it runs bare metal, do you have the latest BIOS installed? This can in theory have an effect on clock handling, which may lead to more clock drift which chrony then has to correct.
 
Have to come back to this.
Issue was just fixed temporary.

It is installed in VM. qemu agent is installed. Running in Qnap NAS - Virtualisation station.
Tried chrony and systemd-timesyncd.

Directly after syncing "rrd update failed : time in past"
Any ideas?
 
Is the QNAP software and firmware up-to-date?
Can you select a different clock source perhaps for the VM?
 
Firmware is up-to-date. PBS fully patched.
Clock Source from host is disabled for vm (tried both)
Log:
Code: 
Nov 14 11:05:26 CG-BAK-01 chronyd[603]: Selected source 162.159.200.1 (2.debian.pool.ntp.org)
Nov 14 11:05:33 CG-BAK-01 proxmox-backup-proxy[657]: rrd update failed: time in past (1668420333.7907386 < 1668423853.0766351)
 
Same issue here with PBS 3.0-1. I'm running it inside a VM on PVE 8. PVE itself is using ZFS.

Bash: 
.......
Jul 21 13:29:58 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689938998.6352339 < 1682079801682079700)
Jul 21 13:30:08 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939008.6357973 < 1682079801682079700)
Jul 21 13:30:18 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939018.6367445 < 1682079801682079700)
Jul 21 13:30:28 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939028.637629 < 1682079801682079700)
Jul 21 13:30:38 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939038.6387258 < 1682079801682079700)
Jul 21 13:30:48 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939048.6394417 < 1682079801682079700)
Jul 21 13:30:58 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939058.641828 < 1682079801682079700)
Jul 21 13:31:08 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939068.6422887 < 1682079801682079700)
Jul 21 13:31:18 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939078.6438355 < 1682079801682079700)
Jul 21 13:31:28 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939088.6451786 < 1682079801682079700)
Jul 21 13:31:38 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939098.647142 < 1682079801682079700)
Jul 21 13:31:48 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939108.6485932 < 1682079801682079700)
Jul 21 13:31:58 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939118.6495142 < 1682079801682079700)
Jul 21 13:32:08 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939128.6506143 < 1682079801682079700)
Jul 21 13:32:18 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939138.651547 < 1682079801682079700)
Jul 21 13:32:28 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939148.652849 < 1682079801682079700)
Jul 21 13:32:38 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939158.654734 < 1682079801682079700)
Jul 21 13:32:48 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939168.6559236 < 1682079801682079700)
Jul 21 13:32:58 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939178.6586094 < 1682079801682079700)
Jul 21 13:33:08 pbs1 proxmox-backup-[590]: pbs1 proxmox-backup-proxy[590]: rrd update failed: time in past (1689939188.6596887 < 1682079801682079700)
......

Converting the Unix epoch time here shows something like 3 months behind.

It seems rrdcached service isn't even installed:

Bash: 
root@pbs1:~# systemctl status rrdcached.service
Unit rrdcached.service could not be found.

I installed it but nothing's changed:

Bash: 
root@pbs1:~# systemctl status rrdcached.service
● rrdcached.service - LSB: start or stop rrdcached
     Loaded: loaded (/etc/init.d/rrdcached; generated)
     Active: active (running) since Fri 2023-07-21 13:51:04 CEST; 21s ago
       Docs: man:systemd-sysv-generator(8)
    Process: 9735 ExecStart=/etc/init.d/rrdcached start (code=exited, status=0/SUCCESS)
      Tasks: 7 (limit: 9440)
     Memory: 3.2M
        CPU: 14ms
     CGroup: /system.slice/rrdcached.service
             └─9742 /usr/bin/rrdcached -B -b /var/lib/rrdcached/db/ -j /var/lib/rrdcached/journal/ -p /var/run/rrdcached.pid -l unix:/var/run/rrdcached.sock

Jul 21 13:51:04 pbs1 systemd[1]: Starting rrdcached.service - LSB: start or stop rrdcached...
Jul 21 13:51:04 pbs1 rrdcached[9735]: rrdcached started.
Jul 21 13:51:04 pbs1 systemd[1]: Started rrdcached.service - LSB: start or stop rrdcached.
 
Last edited:
You must log in or register to reply here.
Top Bottom