System storage concept questions

jeschke

New Member
Oct 31, 2019
17
0
1
Hi all,

I finally have the time to setup my new proxmox-server, but as I am new to ZFS I need to understand how a system is setup best before I order the relevant storage devices!
Many posts in the forum discuss hardware requirements but I still have questions when it comes to redundant system setups with NVMe's:

My server:
ASUS P11C-M/4L, 32GB ECC-RAM, i3-8300

Use case:
- PiHole
- pfSense
- NC
- FreeNAS
- different containers for testing
- possibly webservers
- possibly mailserver
- ...

Planned storage setup:
PVE system: 2x small SATA-SSD's (ZFS mirror) - for example Samsung 860 EVO/Pro
VM's/CT's: 2x 1TB m.2 NVMe SSD's (ZFS mirror) - for example Samsung 970 EVO/Pro
Data/NAS: several HDD's (ZFS Raid) - probably WD red

Open questions:
1. Does the above setup (especially the NVMe's) make sense when it comes to performance and redundancy or are single devices with a proper backup-schedule a better solution?
2. What kind of SSD's/NVMe's are recommended, are Samsung EVO/Pro a good choice? I have read about SSD's being worn out fast when used as system storage device
2. What about SSD-caching, would it improve performance with my setup or are 32GB ECC enough for my use case?

Please be patient as I am new to proxmox and especially ZFS!

Thanks a lot in advance!!
Jeschke
 
Hi,

1. It is ok. But you need to reduce the write io if you want to keep a long life for your ssd/nvme. You could move some folders on hdd pool (like /tmp, /var/log)

2. as a general rule DataCenter ssd are more reliable and will live long

3. you need ssd caching only for hdd pool only. but you can start without it and then you will see if you really need or not.

Good luck / Bafta
 
Hi guletz,

thanks a lot for your reply!

I will probably go with the follwing (for a start):
- 2x Intel DC P4511 for OS and VM's (ZFS mirrored)
- 2x WD Red 10TB for data (ZFS mirrored)
...and later add a cache-SSD if necessary

Furthermore I will follow your advice and move some folders on hdd pool (like /tmp, /var/log) .

In the end I want a system that I can leave alone (hardware) for the next years without too much maintenance. I hope this setup does the trick :)

Cheers,
Christian
 
this will be deadly slow without a fast SSD cache device.

Hi Tom,

I am trying to find out about a suitable setup for days now, please advice on the following setup:
- 2x 1TB Intel DC P4511 for OS and VM's (ZFS mirrored)
- 2x 10TB WD Red for data (ZFS mirrored)
- 1x Intel DC S4500 for Cache, but what size?!

Thanks in advance!
Jeschke
 
2x WD Red 10TB for data (ZFS mirrored)

Hi

Be aware, you could see that yours hdd can be broken at the same time. For any hdd > 4 TB I would go with a raidz2. Recently I see this on a raidz2 with 4 x 8 TB when 2 hdds was decide to die at the same time ;)

Good luck / Bafta
 
outdated model, not really the fastest one. but maybe fast enough for your workload.

can you recommend any specific DC SSD?

what about these:
Intel® SSD D3-S4510 - 240GB
Intel® SSD D3-S4610 - 240GB
Intel® SSD DC S3110 - 256GB

And what about Samsung 983 instead of the Intel DC P4511? As they are a lot cheaper...
 
Last edited:
maybe you can also help me with the decision regarding a new case:

Can anyone give me a recommendation on one of the below? Any other hint for good a case is very welcome!
- Fractal Design Define R5 Black (8x 3,5")
- Nanoxia Deep Silence 4 (6x 3,5")
- SilverStone SST-CS350B (5x 3,5" HotSwap)

Requirements are:
- can fit a micro-ATX board
- as silent as possible
- can handle at least 5 HDD's

Cheers,
Jeschke
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!