Recommended ZFS setup for Mini-PC with 2-3 SSD's

[apprysler]

Hey there!

I just got a Dell OptiPlex 7060 Micro with 32GB of RAM and had a Samsung 980 Pro Nvme SSD lying around, where I installed Proxmox on with ZFS RAID0. To better use ZFS I would like to get another one or two (enterprise) SSD's (the Dell has space for an additional 2,5 SATA SSD but I think with an A+E Adapter I can connect another SATA or M.2 to the Wifi Card slot).

• Would it be advisable to get 1 SATA and 1 M.2 or 2 SATA?
• What would the optimal setup look like regarding on which SSD to install the OS on, the VM's, the data...
• RAIDZ1 vs. Mirroring two SSD's and keep the last single vs. ....

I'm planning to use the PC for Immich, Paperless Ngx primarily plus some other small services like Stirling-PDF and Vaultwarden.
I know that just plain Debian with these services as Docker Containers would totally suffice for my usecase, but I would like to use it as a learning experience aswell, hence ZFS and Proxmox

EDIT: I also have an external USB HDD connected which I plan to use for backups with PBS, so these are covered!

 

[d.oishi]

• Would it be advisable to get 1 SATA and 1 M.2 or 2 SATA?

Technically, it is possible to build a mirror or RAIDZ using a mix of SATA and M.2/NVMe SSDs. However, in a mixed setup, the performance characteristics are not uniform, and the overall behavior tends to be limited by the slower device.

• What would the optimal setup look like regarding on which SSD to install the OS on, the VM's, the data...
• RAIDZ1 vs. Mirroring two SSD's and keep the last single vs. ....

Since RAID0 provides no redundancy, I think it would be worth reconsidering the use of RAID0.

With your current hardware, I think the simplest option would be to add another M.2/NVMe SSD alongside the Samsung 980 Pro NVMe SSD and make them a mirror.

In that case, the OS and VM storage would be managed within a single ZFS pool.

Personally, I think that using a single ZFS pool for both the OS and VM storage makes better use of the available SSD capacity. On the other hand, if you want to prioritize expandability and ease of maintenance for the VM storage, separating them into different pools also makes sense.

So my guess is that, if you later decide you want to expand the VM storage, you could then use the remaining SATA slot(s) for that purpose.

 

[apprysler]

Technically, it is possible to build a mirror or RAIDZ using a mix of SATA and M.2/NVMe SSDs. However, in a mixed setup, the performance characteristics are not uniform, and the overall behavior tends to be limited by the slower device.



Since RAID0 provides no redundancy, I think it would be worth reconsidering the use of RAID0.

With your current hardware, I think the simplest option would be to add another M.2/NVMe SSD alongside the Samsung 980 Pro NVMe SSD and make them a mirror.

In that case, the OS and VM storage would be managed within a single ZFS pool.

Personally, I think that using a single ZFS pool for both the OS and VM storage makes better use of the available SSD capacity. On the other hand, if you want to prioritize expandability and ease of maintenance for the VM storage, separating them into different pools also makes sense.

So my guess is that, if you later decide you want to expand the VM storage, you could then use the remaining SATA slot(s) for that purpose.

Thank you for the answer!
All right so two M.2s for a pool. If I were to get another SATA, would I make another standalone pool with that and use that for the data for the Docker containers, like images for Immich and documents for paperless?
For a second M.2, any 1TB Enterprise SSD should do, right?

 

[d.oishi]

All right so two M.2s for a pool. If I were to get another SATA, would I make another standalone pool with that and use that for the data for the Docker containers, like images for Immich and documents for paperless?

If you want to separate the data storage for capacity or management reasons, I think that would be perfectly fine.

My understanding is that this hardware can only accommodate three SSDs in total, so in that case this separate data pool would not have any redundancy.

It sounds like you already have PBS backups available, so if that drive failed, the recovery path would basically be to restore from PBS.

For a second M.2, any 1TB Enterprise SSD should do, right?

Even among M.2/NVMe SSDs, performance can vary quite a bit depending on the model.

For ZFS, I think it is better if the SSDs used in a mirror or RAIDZ have similar performance characteristics. In the best case, using the same SSD model would be ideal.