New Setup/Hardware advice and ZFS questions

RCaldeira

Member
Jun 2, 2020
6
1
8
46
I'm an enthusiast and I'm just looking for some more detailed info in regards to proxmox and how this can actually help us. Any comments will be welcome and appreciated. Thanks in advance for your time.

FOR HOME USE
The main objective is to use VMs to run OS (Windows 10 pro) and the clients hardware to be a "less better system hardware" like a Minix but taking the advantage of the file share speeds, gpu passthru, ram and cores of the server. At the same time there is the need to run some small docker apps for home automation (running currently on a NAS with the 2x 2TB). Gaming is not one main objective but any comments would be appreciated as, in the end, this would help a lot in a final decision.

2 VMs (pfSense, Samba AD)
Docker (Nodered, Unifi Controller, MQTT, ZigBee2Mqtt)
Plex (encoding, 3 local concurrent streams max over ethernet)
File share between 3 Windows 10 Pro computers - Intent to use roaming profiles/main storage on the file server
Casual Video editing

Hardware (new parts identified below with *)

-4x4TB HD WD - Media/Storage. (*Planing to add another drive to implement RAIDZ2 to obtain a safe/reliable solution, keeping the same available space and improving slightly the performance. Question 1 - Does it worth getting a 5th drive or keep them in mirror only?)
-2x2TB HD WD - Question 2 - How can we integrate these drives? Should I keep these in a separate pool? Don't mind to keep them just for ISOs and local storage. The main concern is the future drive upgrade and performance. (NAS system will not be an option anymore as the intention is to selling it)
-1xSSD 128Gb extra available
-1x LBA - 10Gtek® Internal SAS/SATA RAID Controller PCI Express Host Bus Adapter for LSI 9211-8I, LSI SAS2008 Chip, 8-Port 6Gb/s. The plan is not to use any SATA ports from the motherboard.

*2x EVO 970 PLUS 500Gb to be used as L2ARC and LOG - Question 3 - Does it worth it in this system and what would be a reasonable Log size?
*Motherboard Asrock X399D8A-2T . 2 GPUs to passthru. Question 4 - Is non ECC a reasonable option with this setup? Either way, the main justification is the amount of PCIe x16 available in this motherboard. Question 5 - Would be possible to add more GPUs to this motherboard or 2 is the max?
*64GB Ram - NON ECC as there are several concerns if the ECC ram will actually work with the threadripper. Question 6 - Can anyone confirm if ECC is supported with this CPU?
*AMD Threadripper 2920X 3.5 GHz 12-Core Processor Question 7 - Any comments or sugestions?

So the current budget reaches aprox 1700 Euros. The file share speed expectations are to, at least, largely improve current gigabit/synology nas system (SMB3/CIFS). Question 8 - Would this be a good choice of hardware or is there a better overall solution for the proposed objectives?

Thanks in advance for your time and as I've mention, any feedback will be appreciated. I'm just an enthusiast looking and learning about what Proxmox can actually do to improve our experience.
 
Last edited:
Question 2 - How can we integrate these drives?

Do stripped mirror of your 4x4 TB and 2x2TB and you'll get net 10 TB of space and use all devices

Question 3 - Does it worth it in this system and what would be a reasonable Log size?

No, those are crappy devices for ZFS. ZIL/SLOG is only 5-10G, most of the time much less. Use very small maybe used enterprise SSD.
Another way is to get enterprise SSDs in a mirror (e.g. also 240GB) and use them for special allocation class and SLOG/ZIL to boost the performance of your ZFS system.
 
  • Like
Reactions: RCaldeira
Thank you LnxBill, appreciated!

Do stripped mirror of your 4x4 TB and 2x2TB and you'll get net 10 TB of space and use all devices

Do you mean 4Tb+4Tb+2Tb in mirror?

No, those are crappy devices for ZFS. ZIL/SLOG is only 5-10G, most of the time much less. Use very small maybe used enterprise SSD.
Another way is to get enterprise SSDs in a mirror (e.g. also 240GB) and use them for special allocation class and SLOG/ZIL to boost the performance of your ZFS system.

Any specific recommendation so I can dig on it a bit more? Optane would work?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!