Proxmox and non-enterprise SSDs

[David Carmichael]

Hope everyone's day is going well, just a quick question. I've been trying to find out about how Proxmox handles SSDs. There are various posts from years ago mentioning that Proxmox kills SSDs with the amount of read and write cycles on SSDs. Is this still the case?

I run a homelab Proxmox server so nothing is mission critical - I am however trying to learn as much as I can. I can't find a conclusive answer on how to effectively manage SSD storage.

I have bought two brand new SSDs for my homelab setup. I am a little worried that they wont last long.

Does anyone know where I could possibly find any good documentation regarding this?

 

[tom]

Proxmox does not "kill" SSDs.

SSDs has limited write cycles, so you need to make sure that your SSD are suitable to your use case (write workload).

First, you should look on the write endurance of your SSD model.

 

[tschanness]

On the other hand: PVE will write a lot of data even in a homelab and may kill your SSD faster. This is still true.
Using a HDD for the PVE installation and consumer SSDs as Datastores is my "solution" to that... minor inconvinence.

 

[David Carmichael]

Thanks for your response Tom. - I bought a really cheap drive that only has 181 TDW, again it is not mission critical. How much data would you expect Proxmox to write each day? - Is there a way to monitor this within Proxmox? ( maybe check the wear on SSDs or drives in general )

Thanks again

 

[David Carmichael]

On the other hand: PVE will write a lot of data even in a homelab and may kill your SSD faster. This is still true.
Using a HDD for the PVE installation and consumer SSDs as Datastores is my "solution" to that... minor inconvinence.

OK - I may need to change my main installation media then, I have installed PVE on a 120GB cheapo SSD :/

 

[BlueLineSwinger]

Used Intel enterprise SATA SSDs are currently going for relatively cheap on eBay. ~$100-120 for 2.5" 480GB units, for example.

Most of the listings given here have ended, but it should give you some ideas of what to expect:
https://forums.servethehome.com/index.php?threads/everyones-next-boot-drive-s3500-480gb-120.19514/

 

[David Carmichael]

Thanks for all your responses, I really appreciate your help. I will have a look at those Intel SSD drives, I was thinking about just buying an enterprise drive. Maybe a WD Gold or something.

Thanks again.

 

[David Carmichael]

Used Intel enterprise SATA SSDs are currently going for relatively cheap on eBay. ~$100-120 for 2.5" 480GB units, for example.

Most of the listings given here have ended, but it should give you some ideas of what to expect:
https://forums.servethehome.com/index.php?threads/everyones-next-boot-drive-s3500-480gb-120.19514/

Those SSD drives are only rated for 140 TBW - 0_0 ; Must have a better control chip inside to be classed as enterprise surely? They need to make this stuff easier to figure out honestly.

 

[Tahsin]

One way to get more life out of the SSD drives is to over provision them. I have pretty cheap SSD with 256GB but actual partition size is 100GB with rest as unallocated. That will help with wear leveling.

 

[David Carmichael]

One way to get more life out of the SSD drives is to over provision them. I have pretty cheap SSD with 256GB but actual partition size is 100GB with rest as unallocated. That will help with wear leveling.

That is a pretty smart idea ... So if I use only 60GB of my 120s they will last longer? - such a good idea.

 

[Tahsin]

It will extend the life of the SSD. That is one of the reasons why enterprise drives are weird sizes. They come over provisioned from factory which is not user changeable.

 

[frenchfries]

if you don't have a cluster there are some services you can turn off which continuously write to the disk, thereby extending life significantly.
pve-ha-crm and pve-ha-lrm specifically are the ones to target

 

[BlueLineSwinger]

Those SSD drives are only rated for 140 TBW - 0_0 ; Must have a better control chip inside to be classed as enterprise surely? They need to make this stuff easier to figure out honestly.

If the manufacturer of your SSD is to be believed (I tend to trust Intel and the specs on on enterprise-grade gear in general more) it would seem consumer-grade units have caught up to where enterprise was ~5 years ago.

275 TBW is for the 480 GB model, 140 TBW for the 240 GB model. Over five years that's about a third of the drive's capacity (~150 GB for the 480 GB model) rewritten each day. That should be plenty for a home lab and even many business/production deployments. In a situation where write endurance is paramount then bumping up to the S3600 or S3700 would be warranted.

Enterprise drives have other useful features, such as power loss protection and typically much better performance under high write IOPS.

One way to get more life out of the SSD drives is to over provision them. I have pretty cheap SSD with 256GB but actual partition size is 100GB with rest as unallocated. That will help with wear leveling.

Over-provisioning can be helpful for garbage collection and some other performance aspects, but does nothing for extending write endurance. The SSD's controller, which of course provides for wear leveling, still knows what the actual capacity is and will do the job regardless of the partitioning scheme used.

 

[David Carmichael]

if you don't have a cluster there are some services you can turn off which continuously write to the disk, thereby extending life significantly.
pve-ha-crm and pve-ha-lrm specifically are the ones to target

I don't have a cluster, so I will look into that.

If the manufacturer of your SSD is to be believed (I tend to trust Intel and the specs on on enterprise-grade gear in general more) it would seem consumer-grade units have caught up to where enterprise was ~5 years ago.

275 TBW is for the 480 GB model, 140 TBW for the 240 GB model. Over five years that's about a third of the drive's capacity (~150 GB for the 480 GB model) rewritten each day. That should be plenty for a home lab and even many business/production deployments. In a situation where write endurance is paramount then bumping up to the S3600 or S3700 would be warranted.

Enterprise drives have other useful features, such as power loss protection and typically much better performance under high write IOPS.

Is there any need for the base system to be on an SSD in the first place? - I mean if you have VMs on the SSD media, would I really need an SSD for the PVE install?

I have been thinking if there is really any need for the PVE installed to be on an SSD? - I have my VMs on SSDs isn't that all that really matters?

 

[wimp9849]

I can confirm. Installing on SSD will kill it if you not stop and disable the 2 pve-ha services. The solution is installing over on a mirror of little mechanical disks. It's a pity because in this way you waste 2 disk bays.

It could be a good way to write the cluster status on ram on every node.

 

[scyche]

So can i safety stop and disable pve-ha-crm and pve-ha-lrm services on Proxmox 4.2 without cluster ?

 

[Rhinox]

So can i safety stop and disable pve-ha-crm and pve-ha-lrm services on Proxmox 4.2 without cluster ?

Yes, you can. I already discussed this some time ago...