Reducing rrdcached writes

emox

New Member
Dec 4, 2019
4
1
3
I didn't find any information about this, so I thought I'd post in case other were interested.

Some users would prefer to have reduced disk writes on their proxmox installation.
One source of writes is the very useful rrd data.

One way to continue to obtain the info, but reduce the disk writes can be directly configured by RRDcached.

1. Increase the write and flush timers

Increasing the write timer is easy
in /etc/defaults/rrdcached set the following:
# Optional override write interval (-w arg) , in seconds.
WRITE_TIMEOUT=3600


The flush timer is a bit trickier as there is no predefined variable for it, so I added this
FLUSH_TIMEOUT=7200
and added a line to /etc/init.d/rrdcached

RRDCACHED_OPTIONS="\
${BASE_OPTIONS} \
${NETWORK_OPTIONS} \
${WRITE_TIMEOUT:+-w ${WRITE_TIMEOUT}} \
${FLUSH_TIMEOUT:+-f ${FLUSH_TIMEOUT}} \
${WRITE_JITTER:+-z ${WRITE_JITTER}} \
${WRITE_THREADS:+-t ${WRITE_THREADS}} \
${BASE_PATH:+-b ${BASE_PATH}} \
${JOURNAL_PATH:+-j ${JOURNAL_PATH}} \
${DAEMON_GROUP:+-G ${DAEMON_GROUP}} \
${DAEMON_USER:+-U ${DAEMON_USER}} \
-p ${PIDFILE} \
${SOCKFILE:+${SOCKGROUP:+-s ${SOCKGROUP}} ${SOCKMODE:+-m ${SOCKMODE}} -l unix:${SOCKFILE}} \
"

2. Disable journaling
All the above is not so useful if rrdcached is still writing journal updates.
Note - removing journaling means the chance of recovering the data in case of a crash is lower. Best to read about what journaling does if you care about the data

in /etc/defaults/rrdcached set the following:
# Where journal files are placed. If left unset, journaling will
# be disabled.
#JOURNAL_PATH=/var/lib/rrdcached/journal/

Restart the rrdcached service:

systemctl restart rrdcached.service

And hopefully the writes will be reduced for you!

Note - if you have the proxmox webgui page open and you are viewing the rrd stats, i suspect somehow the system instructs the rrdcached to flush, so I would close the webpage if you are interested in viewing the effects with something like iotop.

Hope this helps - and if there are other adverse effects to this, please do let me know.
 

emox

New Member
Dec 4, 2019
4
1
3
As an aside, I was also successful in reducing the writes by using a persistent ramdisk. But since then I discovered that rrdcached has these built in settings so I figured those would be best to try first.
 
Jan 6, 2021
93
18
8
46
I have a similar issue, the writes appear every 2 seconds but very small. Are they necessary to be written so quickly?

I have a new proxmox setup and two new Samsung PRO drives running zfs mirror pool for the operating system. (no vms on these drives)
I've been online for about 2 weeks

my drives already show a 1% wear

zpool iostat 2 shows 1mb written every few seconds. Looks like its pmxcfs is writing but I am not certain.

Thanks for any suggestions?

PS. I tried the above cache changes and no change..

I also installed proxmox on a test machine and reproduced the writes. So this is confirms the effect the question now is it normal and can it be reduced. Thanks

zpool_iostat_2.png
 
Last edited:

vkeven

Active Member
Dec 22, 2009
32
2
26
Same problem , is there someone from proxmox which could explain whats is write every 2 seconds ? Thanks !
 

n1nj4888

Member
Jan 13, 2019
162
14
23
42
As an aside, I was also successful in reducing the writes by using a persistent ramdisk. But since then I discovered that rrdcached has these built in settings so I figured those would be best to try first.
Is it a simple configuration item to set the rrdcached writes to ram disk rather than disk? This approach could leave the WRITE_INTERVAL and FLUSH_INTERVAL as standard but write to ram instead of wearing out SSDs? I don’t fully understand why Proxmox writes so much to disk if it could easily use RAM disk?
 

ultima

New Member
Apr 9, 2021
6
1
1
71
I didn't find any information about this, so I thought I'd post in case other were interested.

Some users would prefer to have reduced disk writes on their proxmox installation.
One source of writes is the very useful rrd data.

One way to continue to obtain the info, but reduce the disk writes can be directly configured by RRDcached.

1. Increase the write and flush timers

Increasing the write timer is easy
in /etc/defaults/rrdcached set the following:
# Optional override write interval (-w arg) , in seconds.
WRITE_TIMEOUT=3600


The flush timer is a bit trickier as there is no predefined variable for it, so I added this
FLUSH_TIMEOUT=7200
and added a line to /etc/init.d/rrdcached

RRDCACHED_OPTIONS="\
${BASE_OPTIONS} \
${NETWORK_OPTIONS} \
${WRITE_TIMEOUT:+-w ${WRITE_TIMEOUT}} \
${FLUSH_TIMEOUT:+-f ${FLUSH_TIMEOUT}} \
${WRITE_JITTER:+-z ${WRITE_JITTER}} \
${WRITE_THREADS:+-t ${WRITE_THREADS}} \
${BASE_PATH:+-b ${BASE_PATH}} \
${JOURNAL_PATH:+-j ${JOURNAL_PATH}} \
${DAEMON_GROUP:+-G ${DAEMON_GROUP}} \
${DAEMON_USER:+-U ${DAEMON_USER}} \
-p ${PIDFILE} \
${SOCKFILE:+${SOCKGROUP:+-s ${SOCKGROUP}} ${SOCKMODE:+-m ${SOCKMODE}} -l unix:${SOCKFILE}} \
"

2. Disable journaling
All the above is not so useful if rrdcached is still writing journal updates.
Note - removing journaling means the chance of recovering the data in case of a crash is lower. Best to read about what journaling does if you care about the data

in /etc/defaults/rrdcached set the following:
# Where journal files are placed. If left unset, journaling will
# be disabled.
#JOURNAL_PATH=/var/lib/rrdcached/journal/

Restart the rrdcached service:

systemctl restart rrdcached.service

And hopefully the writes will be reduced for you!

Note - if you have the proxmox webgui page open and you are viewing the rrd stats, i suspect somehow the system instructs the rrdcached to flush, so I would close the webpage if you are interested in viewing the effects with something like iotop.

Hope this helps - and if there are other adverse effects to this, please do let me know.
/etc/defaults/rrdcached does not currently exist and Google shows only this document. Is this a typo? Can you please provide the correct filename?
 

leesteken

Famous Member
May 31, 2020
1,744
355
88
/etc/defaults/rrdcached does not currently exist and Google shows only this document. Is this a typo? Can you please provide the correct filename?
Try /etc/default/rrdcached without the s.
EDIT: Fixed typo for future readers
 
Last edited:
  • Like
Reactions: ultima

ultima

New Member
Apr 9, 2021
6
1
1
71
Try /etc/default/rddcached without the s.
Nope; doesn't exist either. However, you put me on the right track. The correct path is
/etc/default/rrdcached
Thanks!

EDIT: replaced 'rdd' with 'rrd' (typo)
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!