Optimizing pxmcfs Process in Proxmox: Reducing Hard Drive Wear with RAM-Based Storage [Question]

isasmendi

New Member
Dec 3, 2022
7
1
1
Hi everyone,

I have a question regarding the pxmcfs process in Proxmox. I've noticed that it writes to the hard drive constantly, which has me concerned about potential long-term damage to the disk. After looking at the pve-cluster service code (available at here), I discovered that most of the writes are to the /var/lib/pve-cluster folder, specifically in the config.db-wal file.

To address this, I created a service that runs before pve-cluster, which does a bind mount of /dev/shm/pve-cluster-ram to /var/lib/pve-cluster. This results in files that change permanently being written to memory instead of the hard drive.

I should mention that I use Proxmox as a standalone node, and I do not have a cluster configured.

However, I have some questions about this approach. Firstly, what data could I lose in the event of a server restart where the config.db and config.db-wal files are lost? Secondly, would it be risky for my system if I don't make any system modifications like creating containers or changing the firewall? Lastly, how would backups be affected if I create a backup of a container/VM and the system shuts down (with config.db and config.db-wal in memory)? Could the backup list become out of sync with the actual data on my disk?

I would greatly appreciate any advice or suggestions on how to handle this situation. Thank you in advance for your help.

Best regards, Agustin I
 
However, I have some questions about this approach. Firstly, what data could I lose in the event of a server restart where the config.db and config.db-wal files are lost?
Your PVE configuration.

Secondly, would it be risky for my system if I don't make any system modifications like creating containers or changing the firewall?
IF you don't change anything, it MAY work

Lastly, how would backups be affected if I create a backup of a container/VM and the system shuts down (with config.db and config.db-wal in memory)? Could the backup list become out of sync with the actual data on my disk?
No, backups are not stored in your /etc/pve directory. A backup is just a compressed file (vma or tar) on your backup storage.

To address this, I created a service that runs before pve-cluster, which does a bind mount of /dev/shm/pve-cluster-ram to /var/lib/pve-cluster. This results in files that change permanently being written to memory instead of the hard drive.
Writing to a harddisk is what it was made for. I don't see that it'll harm it. For consumer SSDs, this IS another matter and the main culprit for early loss of device. Therefore, buy enterprise SSDs, which don't have the huge write amplification that will result in early wear.
 
  • Like
Reactions: isasmendi
Hello all

Thanks @LnxBil for your response. It helped me to solve the problem.

If you've made it here, it's because you're looking for a way to eliminate I/O operations from the pmxcfs process.
I've created a repository where you'll find a service that 'forces' pmxcfs to use RAM instead of HDD/SDD. You will find the installation instructions and how it works in the repository itself.

Link to repo

If you're going to use it, I recommend that you create a virtual machine, install Proxmox, and perform the necessary tests on that VM. Once you're satisfied, you can install it on your real Proxmox installation.

I'm currently using the service in my home lab and it works perfectly.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!