How would you do it. ZFS layout question

Longun

New Member
May 13, 2020
2
0
1
45
Hi All,

First post from me, so hi. I've got a home lab that's been running a single node storage spaces direct lab, then a VSAN lab and now I've discovered proxmox and containers I'm a convert. Love it want to learn more and then keep it as my home production server for want of a better phrase.

Question I have is around how you'd layout the disks on the following spec.

Ryzen 7 3700
64 GB RAM (Have space for another 64 when funds allow)
10GbE Network

Disk specs

2x240GB SSD - These will be raid 1 ZFS for proxmox install so can be ignored.

2x1GB Samsung NVMe
4x1GB Samsung SSD
6X3TB 7200 SATA Drives

The SSD Drives are on the motherboard SATA ports and the HDDs are on a 6 port expansion card but it's in a PCI 1x slot as my Nic is taking up my only high speed PCI slot (Bad mobo choice on my part but I didn't purchase it with this in mind at the time )

I'm tempted to split them into 3 pools. NVMe using raid 1, SSD using raid 10 and HDD using either 5 or 6 (rebuild times not a major as I have solid backups).

Curious to know if there is a layout I've not considered that the more experiences community could recommend and how others would setup this hardware? I know adding more RAM will give good gains, I won't be in a position to do this for a month or two but consider the system will be running 128GB when it's finished.

Use case is multiple low use containers, VPN PiHole, Plex, grafana, DB, Minecraft servers, file server, torrent server and other small uses I'll no doubt find as I learn. The big use for me though is around 3TB of video files that'll I'll be wanting to edit directly from the storage on this server if I can. Currently running off a NAS but it's a bit slow so I find myself constantly copying data off the Nas editing and copying it back again which is getting to be a pain.

I want to try and get as close to the 10GbE limit as I can, but I fear only the NVMe drives would be capable of pushing that speed and even the SSDs might not be capable using ZFS on the controllers I'm using.

Sorry for the essay

Thoughts and opinions very much welcomed

Cheers, Joe.
 
Hi Joe,

I would try like this:

- 2x1GB Samsung NVMe as special devices class(metadata and maybe small blocks < 4k)
- one fast pool raid10 using 4x1GB Samsung SSD
- the slow pool using 6X3TB 7200 SATA Drives also in Raid10 (using special devices class/NVME)


Good luck /Bafta!
 
Thanks very much for this suggestion. Seems to be working well and performance of the slow pool is way better than I expected. I've not used a special device setup before, I always cringe when using an NVMe for something where I don't see its blistering speed but seems to really help the Slow pool.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!