Recommended SSDs/M.2s for a non critical mini server system

[Dunuin]

1) Is 800GB enough for proxmox OS and ISO files? (VMs data will be stored separately on U.2 high endurance SSDs)

For PVE 16-32GB should be fine. Then keep in mind that a ZFS pools will become slower after filling it more than 80% and at 90% it will become even slower. So with 2x 800GB in a mirror you get roughly 608GB for ISOs, templates, VMs, LXCs and backups.

Is 5,000 MB/s read and 1,400 MB/s write speed is enough for proxmox OS drive? Does proxmox OS even benefit from very high speed SSD or regular SATA SSD of 550 MB/s is more than enough for proxmox OS drive?

You don't need NVMe for PVE. Even slow HDDs would work. Only VMs/LXCs would really benefit from that.

 

[rigel.local]

For PVE 16-32GB should be fine. Then keep in mind that a ZFS pools will become slower after filling it more than 80% and at 90% it will become even slower. So with 2x 800GB in a mirror you get roughly 608GB for ISOs, templates, VMs, LXCs and backups.

You don't need NVMe for PVE. Even slow HDDs would work. Only VMs/LXCs would really benefit from that.

Thank you so much!!

I found your old thread here https://forum.proxmox.com/threads/ashift-for-intel-s3700-s3710.74902/

And I think for proxmox PVE OS install, I will try to follow your route with Intel Intel DC S3700/S3710, found some cheap options on eBay, need to make sure they deliver to me. 10 drive writes per day for 5 years sounds too sweet to resist.

 

[Dunuin]

And I think for proxmox PVE OS install, I will try to follow your route with Intel Intel DC S3700/S3710, found some cheap options on eBay, need to make sure they deliver to me. 10 drive writes per day for 5 years sounds too sweet to resist.

Jup, but these are basically overkill for a PVE system disk. But really nice if you want to run some guests on top of ZFS without fearing the wear.
There are also some really cheap, small and durable consumer optanes that would make a perfect M.2 boot disk:
You used to get the 32GB version with 182.5TB TBW new for under 30€. The question there was more if you want to waste two M.2 slots just for boot disks.

 

[Patrick79]

Hi everyone,

i have a question regarding the Micron M.2 options:

The Micron 7450 MAX has a DWPD of 3; the Micron 7400/7450 PRO have a DWPD of 1. Unfortunately the 7450 MAX is only with a maximum capacity of 800GB. Would a PRO version be sufficient for a home lab with the PVE and 10 - 20 vms / lxcs with with light to medium traffic? Or would SSD wear out be a problem? I plan to use two disks as a mirror

Thank you

 

[Dunuin]

Does your server fit 22110 M.2? Because only the 480GB/960GB PROs are 2280 M.2 (which should be very similar to the 400/800GB MAXs, both probably using 512GB/1024GB of TLC NAND just that the PROs are using 32GB/64GB as protected spare area and the MAXs 112GB/224GB protected area for spare NAND cells and SLC cache).

If they are sufficient really depends on your workload. Especially small random sync writes will wear them a lot, so it really depends on how much data you write to DBs. Big async sequential writes are usually not really a problem, as the write amplification for these will be a fraction compared to small random sync writes.

 

[Patrick79]

Thank you for your answer. I have not yet decided on the hardware; i am currently undecided between a Lenovo M920x tiny, P350 SFF or a custom build. But thats a good point you make about the form factor...i did not notice that.

My workload will be a mix of home automation, development playground, networking stuff, Nas and download station. I guess its difficult to project, but to be on the safe side i should prefer the Max version i suppose. Perhabs someone in this forum has experience regarding both options

 

[leesteken]

I love the 7450 PRO 960 because it was cheap ($150) and sync/sec reached 17500 (pveperf, other SSDs were less than 800), probably because of the PLP. I don't run any heave I/O VMs but the speed is great and I don't have to worry about power outage or accidental reboots or too many sync writes.

 

[dasbooter]

Jup, but these are basically overkill for a PVE system disk. But really nice if you want to run some guests on top of ZFS without fearing the wear.
There are also some really cheap, small and durable consumer optanes that would make a perfect M.2 boot disk:
You used to get the 32GB version with 182.5TB TBW new for under 30€. The question there was more if you want to waste two M.2 slots just for boot disks.

I've read many of your threads where you weigh in on hardware and I appreciate your opinion and have taken it to heart.

I can see s3710's up on eBay. I'm in Canada. These are all used drives and I'm wondering how you are vetting them. This is my most important question.

 

[Dunuin]

I've read many of your threads where you weigh in on hardware and I appreciate your opinion and have taken it to heart.

I can see s3710's up on eBay. I'm in Canada. These are all used drives and I'm wondering how you are vetting them. This is my most important question.

I bought for my homelab the last years:
11x S3710 400GB
8x S3710 200GB
2x S3700 400GB
2x S3700 200GB
2x S3700 100GB

All got wear between 0-4% when I bought them, even when they already ran 24/7 for 6 years in someone's server. Off all those disks only one S3710 400GB failed after a year or so. I guess the controller failed, as all the SMART attributes looked absolutely fine (3% wear if I remember right) until that SSD wasn't recognized anymore.

When buying them I first ask the seller if he can use the free smartctl/CrystalDiskInfo to show me the SMART attributes. With that I can decide if that SSD is in good condition or not (especially have a look at the "233 Media_Wearout_Indicator" for the wear, "175 Power_Loss_Cap_Test" for the PLP and "9 Power_On_Hours" to know how long the SSD was in use. "5 Reallocated_Sector_Ct", "171 Program_Fail_Count", "172 Erase_Fail_Count", "184 End-to-End_Error", "187 Reported_Uncorrect" and "199 CRC_Error_Count" should be 0).

If the seller can't or doesn't want to provide me with SMART data I only buy it if it as a good deal and sold as tested and working. For example, some years ago I bought 2x S3700 100GB for combined 25€ incl shipping which is great as a boot mirror. And some months ago I bought 2x S3700 400GB for combined 50€ incl shipping. Even those SSDs, which I didn't know the condition, none of the SSDs got more than 4% wear.

Those are real workhorses. My homelab is writing over 1TB per day and I think none of the SSDs actually climbed above 4% wear yet

PS:
There are also 1.8" and SAS models, so look twice before buying them.
There is a datasheet going really deep into the details including what every SMART attribute means, which you don't often see: https://www.intel.com/content/dam/w.../product-specifications/ssd-dc-s3710-spec.pdf

And the old Intel tool for changing settings (you can switch between 512B/4K sectors, limit power consumption, encryption, protection mode so you can't secure_erase it by accident, ...), updating the firmware or doing a secure_erase (I would recommend doing both when you get them) isn't available anymore, as these are now sold as Solidigm instead of Intel. But the Solidigm Storage Tool works fine for that: https://www.solidigm.de/support-page/drivers-downloads/ka-00085.html

 

[dasbooter]

I bought for my homelab the last years:
10x S3710 400GB
8x S3710 200GB
2x S3700 400GB
2x S3700 200GB
2x S3700 100GB

All got wear between 0-4% when I bought them, even when they already ran 24/7 for 6 years in someone's server. Off all those disks only one S3710 400GB failed after a year or so. I guess the controller failed, as all the SMART attributes looked absolutely fine (3% wear if I remember right) until that SSD wasn't recognized anymore.

When buying them I first ask the seller if he can use the free smartctl/CrystalDiskInfo to show me the SMART attributes. With that I can decide if that SSD is in good condition or not (especially have a look at the "233 Media_Wearout_Indicator" for the wear, "175 Power_Loss_Cap_Test" for the PLP and "9 Power_On_Hours" to know how long the SSD was in use. "5 Reallocated_Sector_Ct", "171 Program_Fail_Count", "172 Erase_Fail_Count", "184 End-to-End_Error", "187 Reported_Uncorrect" and "199 CRC_Error_Count" should be 0).

If the seller can't or doesn't want to provide me with SMART data I only buy it if it as a good deal and sold as tested and working. For example, some years ago I bought 2x S3700 100GB for combined 25€ incl shipping which is great as a boot mirror. And some months ago I bought 2x S3700 400GB for combined 50€ incl shipping. Even those SSDs, which I didn't know the condition, none of the SSDs got more than 4% wear.

Those are real workhorses. My homelab is writing over 1TB per day and I think none of the SSDs actually climbed above 4% wear yet

PS:
There are also 1.8" and SAS models, so look twice before buying them.
There is a datasheet going really deep into the details including what every SMART attribute means, which you don't often see: https://www.intel.com/content/dam/w.../product-specifications/ssd-dc-s3710-spec.pdf

And the old Intel tool for changing settings (you can switch between 512B/4K sectors, limit power consumption, encryption, protection mode so you can't secure_erase it by accident, ...), updating the firmware or doing a secure_erase (I would recommend doing both when you get them) isn't available anymore, as these are now sold as Solidigm instead of Intel. But the Solidigm Storage Tool works fine for that: https://www.solidigm.de/support-page/drivers-downloads/ka-00085.html

That's quite a bit of buying experience. Any experience with rebranded versus non. Example Lenovo/HP/Dell

How are these
____www.ebay.ca/sch/i.html?_nkw=intel+S3710&_trksid=p2334524.m4084.l1313&_odkw=S3710&LH_PrefLoc=1___

Looks to be 12.5 cents per gigabyte

The US has a lot of good stuff but importing into Canada gets you gouged by the delivery companies. Might be a good test though as I have been thinking about sourcing an enterprise server from there. Not much around these parts

 

[Dunuin]

Can't tell much about the branded ones. All I bought are unbranded. But that's not because they are worse, unbranded 200GB/400GB models are 90% what you find on the market here. 800GB and 1200GB versions are super rare and I think the refurbishers selling branded ones were more expensive.

 

[dasbooter]

Can't tell much about the branded ones. All I bought are unbranded. But that's not because they are worse, unbranded 200GB/400GB models are 90% what you find on the market here. 800GB and 1200GB versions are super rare and I think the refurbishers selling branded ones were more expensive.

First of only 2 options that are actually in my country. Second option has just the regular Intel Swatch it's 800gb for 95$. I did some reading and apparently you could not warranty the rebranded ones as the warranty would be only honoured for the immediate purchaser which they would typically have a record of. I'm guessing these were all produced at a time that would put them out of warranty anyways.

 

[Dunuin]

Back then when my 400GB Intel non-branded one failed I googled for the warranty at is was only 4,5 years old. It is the same with Intel. Warranty loss once you resell it.