ZFS: Need help with vdevs & pools across multiple HDDs with different capacity

[RyanMP1011]

Hi all,

I have the following disks installed in my NAS which I'm trying to set up for Media Server, Network Storage, and Backups:
20TB
12TB
4TB
3TB

Essentially, I'd like to partition the 3 largest disks to have a 2TB partition, which will mirror all 4 HDDs for backups.
Then I would use another 1TB for Linux Timeshift backups and PBS.

I would then use most of the remaining space on the two larger drives for NAS storage. I'll attach a picture of a spreadsheet I made (the units are not perfectly accurate, I mixed up TiB and TB in a few places).

Does this seem like I'm taking the correct approach? Or is this unnecessarily complicated? I'd also like to mention that I keep Cold Storage on some 500GB & 1TB Hard Drives.

I apologize for the lack of detail, I'm relatively new to ZFS. I understand that this configuration isn't optimal, but due to the ridiculous price of HDDs lately, I'm having to work with what I have. Thanks in advance,
Ryan

 

[UdoB]

That's... not the way ZFS usually works. It needs to have identically sized disks.

Of course this is Linux and "anything goes". But any "creative" idea to use all the space will possibly lead to trouble in the future. Manual partitioning and then creating multiple pools with these partitions will lead to multiple pools on the same physical spindle. I've never tried that, because... it feels really wrong. (And of course multiple pools will die be affected when (not: if) one spindle dies.)

If you want to use all four drives for ZFS in one single pool you can create a striped mirror (similar "Raid10") of the 20+12 and 4+3 TB drives. The result is a pool with a theoretical capacity of 12+3 = 15 TB --> probably ~14 TB --> 11 to 12 TB usable. You would need to create the pool on the command line as the Web GUI won't let you do such crazy things.

This is not a recommendation, just a possibility! The difference between both mirrored vdevs is larger than I would accept for my pools.

The only use case I see is a simple striped pool with zero redundancy - usable only for "scratch"-data without any value. I am fairly sure you do not want that one!

Sorry, I have no good advice :-(

 

[ness1602]

I would usually recommend something like mergefs or bcachefs in this case,certainly not zfs.

 

[RyanMP1011]

That's... not the way ZFS usually works. It needs to have identically sized disks.

Of course this is Linux and "anything goes". But any "creative" idea to use all the space will possibly lead to trouble in the future. Manual partitioning and then creating multiple pools with these partitions will lead to multiple pools on the same physical spindle. I've never tried that, because... it feels really wrong. (And of course multiple pools will die be affected when (not: if) one spindle dies.)

If you want to use all four drives for ZFS in one single pool you can create a striped mirror (similar "Raid10") of the 20+12 and 4+3 TB drives. The result is a pool with a theoretical capacity of 12+3 = 15 TB --> probably ~14 TB --> 11 to 12 TB usable. You would need to create the pool on the command line as the Web GUI won't let you do such crazy things.

This is not a recommendation, just a possibility! The difference between both mirrored vdevs is larger than I would accept for my pools.

The only use case I see is a simple striped pool with zero redundancy - usable only for "scratch"-data without any value. I am fairly sure you do not want that one!

Sorry, I have no good advice :-(

Thanks for the response! I'll need to rethink this setup of mine, maybe I just need to bite the bullet and get a couple more 12TB drives.

I would usually recommend something like mergefs or bcachefs in this case,certainly not zfs.

Thank you, I don't remember why I had settled on ZFS, but I'm gonna go back to the drawing board & look into the two filesystems you mentioned.

 

[leesteken]

I have a similar (but smaller size and solid-state) setup. Proxmox is installed (with ZFS on 16GB partitions) on all three drives (mirror), to make sure it runs even if two drives fail (although the motherboard BIOS might need to be adjusted to select a functioning boot drive). One drive is 1 (TB, but it could be any other unit), one is half that and the last (oldest) drive is a 1/4. The 1/2-space is used is 2-way mirror for VMs and the 1/4-space is a two way mirror for "mostly read-only" store such a ISOs, Linux distro torrents and other shared file storage. The 1/4 that remains on the remaining (fastest and enterprise async write caching) SSD is a "temp pool" used for unimportant/reproducible data and ephermeral VM clones.

EDIT: I'm a weirdo that uses a three-way mirror for PBS with 1 SSD and 2 HDDs. That gives me the assurance that my data is safe when one drives fails (as it still has a two-way redundancy) and improves (random) reads a lot (as modern ZFS sends operations to the fastest drive). It also gives me two different media and a remote system to fulfill the 3-2-1- backup principle.