Need some advice on disk config

[Axl_Rose]

I'm looking to reconfigure my proxmox server and need some help with the best configuration. I have a about a dozen vm/containers. Running plex, torrents, gitea, docker, a couple websites, zoneminder and a couple other various things.
Server:

• Xeon E3-127
• 32GB ECC

For video storage I have a 8TB RAID 1 in ZFS, that's getting close to full so I need to add to that. But primarilary what should I do for the system. I've been reading conflicting posts about using SSDs for the host. I assume that would go for VMs as well. My initial idea was:

• Small SSD for Host
• 2 500GB SSDs in a RAID 1 for VMs
• 2 1TB 7200 RPM drives in a RAID 1 for centralized logging, influxdb, Zoneminder data
• 2 8TB Drives in Raid 1 for Video

I feel like this might be wasteful, and I question how long the SSDs will last with VMs. Would a RAIDZ1/2 with a bunch of spinning drives be a better choice for VMS and non video storage? IS 32GB(max for my CPU) enough for all this storage?

 

[Dominic]

Hi!

I question how long the SSDs will last with VMs

Depends on the type of SSD.

Would a RAIDZ1/2 with a bunch of spinning drives be a better choice for VMS and non video storage?

Is there any other reason than money to choose spinners? I think the speed of SSDs is a pretty good argument in favor of SSDs. The Proxmox VE Ceph Benchmark has a fio benchmark for different types of disks in the beginning. What might be a good compromise for you is using an SSD as cache for ZFS. See our documentation for some possibilities.

IS 32GB(max for my CPU) enough for all this storage?

For ZFS a rough calculation is 4GB plus 1GB RAM for each TB RAW disk space (docu).

 

[Axl_Rose]

I guess the reason I was leaning towards spinning disks is because they can handle more read/write cycles than an SSD and maybe they would last longer.

 

[Dominic]

There are SSDs where you can write to really much, like estimated 30 PB until failure. Just one example from WD and an explanation. However, the cost of this example series ranges from 500€ to 6000€ per disk.

This should just give an idea of what is possible. I suggest taking a closer look at what SSD options could be relevant for you and compare them to your HDD options.