RRDC and RRD update errors

May 11, 2019
90
4
8
I had to replace my NUC, used the same RAM and SSD, proxmox booted up ok and VMs seem fine too, but I am seeing errors/warnings as below.
My node is "pve" but in the log points to "pve2-storage" (??)

I tried:
"hwclock --systohc"
and
systemctl stop rrdcached.service
rrdcached -P FLUSHALL
systemctl start rrdcached.service
but so far unsuccessfully.

Any help appreciated!
Thx

Sep 20 08:55:19 pve pmxcfs[1108]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-storage/pve/VMs: -1
Sep 20 08:55:19 pve pmxcfs[1108]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-storage/pve/VMs: /var/lib/rrdcached/db/pve2-storage/pve/VMs: illegal attempt to update using time 1600617319 when last update time is 1600639262 (minimum one second step)
Sep 20 08:55:19 pve pmxcfs[1108]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-storage/pve/local-lvm: -1
Sep 20 08:55:19 pve pmxcfs[1108]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-storage/pve/local-lvm: /var/lib/rrdcached/db/pve2-storage/pve/local-lvm: illegal attempt to update using time 1600617319 when last update time is 1600639262 (minimum one second step)
 
May 11, 2019
90
4
8
Warnings do seem to be gone without anything !!!
Never seen anything similar ...

Keeping this post for benefits to other users.
 

Stoiko Ivanov

Proxmox Staff Member
Staff member
May 2, 2018
4,400
551
118
These error messages usually happen if the system clock was off - if it was in the future at some point the rrd is written with this timestamp - once your clock gets synchronized to the current time the rrd refuses to updated (since the timestamp is now older than the one from when the clock was ahead) - once your system reaches the future timestamp rrds get updated normally again - and this error vanishes :)

I hope this explains it.
 
  • Like
Reactions: Yuri Weinstein
These error messages usually happen if the system clock was off - if it was in the future at some point the rrd is written with this timestamp - once your clock gets synchronized to the current time the rrd refuses to updated (since the timestamp is now older than the one from when the clock was ahead) - once your system reaches the future timestamp rrds get updated normally again - and this error vanishes :)

I hope this explains it.
i continuously have this issue, there is no fix in sight, the clock is in sync continuously, even for VM which are stopped
 
Last edited:

Stoiko Ivanov

Proxmox Staff Member
Staff member
May 2, 2018
4,400
551
118
i continuously have this issue, there is no fix in sight, the clock is in sync continuously, even for VM which are stopped
could you post a few loglines with the error-messages?
 
could you post a few loglines with the error-messages?
Oct 22 18:03:01 pmx pmxcfs[386322]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/138: -1
Oct 22 18:03:01 pmx pmxcfs[386322]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/134: -1
Oct 22 18:03:01 pmx pmxcfs[386322]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/137: -1
Oct 22 18:03:01 pmx pmxcfs[386322]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/136: -1
Oct 22 18:03:01 pmx pmxcfs[386322]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/135: -1

because i noticed people mention this, this is also a ZFS based system
 
Last edited:

Stoiko Ivanov

Proxmox Staff Member
Staff member
May 2, 2018
4,400
551
118
Oct 22 18:03:01 pmx pmxcfs[386322]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/135: -1
is a different error - (no mention of the timestamps )...

depending on how important the historical data is - I would probably move the cache directory to a safe place and restart the services (that way recording should start fresh):
Code:
cd /var/lib/rrdcached/
systemctl stop rrdcached
mv rrdcached rrdcached.bck
systemctl start rrdcached
systemctl restart pve-cluster

I hope this helps!
 

pcready.cl

Member
Apr 16, 2017
6
0
6
33
is a different error - (no mention of the timestamps )...

depending on how important the historical data is - I would probably move the cache directory to a safe place and restart the services (that way recording should start fresh):
Code:
cd /var/lib/rrdcached/
systemctl stop rrdcached
mv rrdcached rrdcached.bck
systemctl start rrdcached
systemctl restart pve-cluster

I hope this helps!

There is an error in the commands, the folder path does not exist, please use these in Proxmox v6.2-15:

Bash:
cd /var/lib/
systemctl stop rrdcached
mv rrdcached rrdcached.bck
systemctl start rrdcached
systemctl restart pve-cluster
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!