Difficulty visualizing ZFS setup

[Marieau]

There is some confusion going on trying to visualize or design the setup I think will fit my needs. I am hoping that a more seasoned zfs and proxmox king can lend me hand.

The needs:

• Media server
• Backups as a service
• Private GPT
• Large docker suite

Machine:

• 12th Gen 12-core
• 96GB RAM
• A2000 ADA
• 4 x NVMe Gen 4 x 4 - Backplane
• 1 x NVMe Gen 3 x 8 - MoBo
• 6 x SATA 3.5 bays
• 10GbE

1. On the motherboard I intend to use a P1600X Optane 118GB storage device for Proxmox, Truenas and any docker container I want.
2. For the backplane I would like to use four 2TB NVMe drives.
3. For the 3.5 bays I intend to use 24TB drives - adding 1 driver each month/quarter

In this setup I would like to emphasis performance and storage. Important files/snapshots are done off-site so there is little appetite for investment in redundancy.

Can someone check this:
1. OS as its own vdev and allocate 64GB as a SLOG device?
2. NVMe drives will be setup as 2 x mirrored 2TB drives > 4TB striped mirror used for special device, ZIL, L2Arc, apps and GPT.
3. Spinning rust setup as Raidz1 in a single vdev > used for media and backup files

So it looks like this:


The constraint is Optane availability but I know that many of you run the SLOG in a mirror. This setup also misses a UPS.

 

[Pifouney]

zfs doesn't like electric failures.

Did you gonna use only enterprises disks hardware ? Zfs running bad with consumer disks.

 

[Marieau]

All consumer stuff

 

[Pifouney]

All consumer stuff

Be extremely attentive to that you wanna do. Consummer stuff + zfs isn't a really good idea, many post on this forum can confirm this.

Be optimist:
For zfs in your situation, if a small clients of your installation, you can discover ZFS. But you canno't compare performance stats with professionnal hardware. So don't worries if reals stats differs of that you see on forums.

Protect your electric access of you server: take a UPS that permit you to have a small window of action in power loss situation.

Be analyst:
If never used ZFS, you don't really know how it's cpu consummer.
In fact, ZFS take times on processor frequently, more than a standard FS. Check if this doesn't affect your server use.
ZFS not configured by default on Proxmox gonna use 50% of your max Memory. Take time to tune it if you don't want this situation.

 

[Marieau]

Hi Pifouney,

Thank you for your advice. It is not what I was looking for but I will keep it in mind.