NVMe wearout slowly going up

kolt

New Member
Oct 25, 2023
7
0
1
Basically, the writes on my only raid 1 on Proxmox is creeping up about a couple 100GBs a day with no explanation. HA and other services people usually recommend to turn off to prevent disk wear are off. I check daily (and weekly) maximums of disk IO on all VMs and it doesn't come close to account the writes. Not sure where it's coming from but wearout on both drives are at 13-14% and was at like 10% a couple of days ago, and same before then. I am using Crucial NVMe drives.
 
don't use consumer ssd. Proxmox is doing small sync writes all the times (for cluster, for stats,..).

if you don't have a ssd with power loss protection / supercapacitor, a small 4k write will be amplified to the size of your ssd nand cell (can be multiple MB). (with a supercapacitor protection, the small write are writen in a buffer safely, then flushing to a big write to the ssd nand ssd)
 
You don't say if you use ZFS or LVM mirrors but with ZFS the effect @spirit mentioned is more exaggerated because ZFS is copy-on-write. That means that a small write may cause a large amount of already-written data to be copied. This is good for data integrity but bad for wear.

With consumer hardware you're better off using mdraid + lvm, but you have to install via Debian to get that. I'm at 10% wear-out after 4 years on Samsung 870 EVO.
 
  • Like
Reactions: kolt and Dark26
You don't say if you use ZFS or LVM mirrors but with ZFS the effect @spirit mentioned is more exaggerated because ZFS is copy-on-write. That means that a small write may cause a large amount of already-written data to be copied. This is good for data integrity but bad for wear.

With consumer hardware you're better off using mdraid + lvm, but you have to install via Debian to get that. I'm at 10% wear-out after 4 years on Samsung 870 EVO.

I see people using Samsung NVMes without issues. I am on ZFS. Is your setup ZFS with Samsung? Might consider replacing both crucials with Samsung nvmes
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!