Advice requested on storage setup.

[wawawawa]

Hi All

Environment - Homelab - not production.

Single host, Supermicro X10SDV-8C+ (8C, 16T), 64GB.
1x NVMe SSD - 256GB
1x Samsung SSD - 1TB
4x WD Red - 2TB (total 8TB raw)

(I also have a Synology NAS which I use for backup / archive and all media)

I use a variety of containers / KVM guests.

With 5.0 being released and now that I am dockerizing everything (via CoreOS / RancherOS) I wanted to ask for advice on how I should use the available storage.

I plan to reinstall from scratch. I've taken snapshots and backed everything up.

I was using LVM-thin and FreeNAS as a guest to present the 4x WD Red HDs via NFS.

How would you use this setup?

The aim is for flexibility, ability to snapshot / restore and also shared storage between VMs.

Would you use the 4x2TB as ZFS pool with some of the NVMe SSD as cache?

Thank you in advance for any suggestions.

 

[Magneto]

/following with interest I'm planing on the same configuration, though with different size hard drives. Have you done the installation yet? How's it going?

 

[wawawawa]

/following with interest I'm planing on the same configuration, though with different size hard drives. Have you done the installation yet? How's it going?

Hi there.... I installed Proxmox on the NVME and use most of it as lvm-thin for a bunch of small (10GB) CoreOS and RancherOS hosts.

Before this reinstall I used to have a FreeNAS VM with direct access to the disks but now I use native Proxmox instead. Bulk storage is the 4x2TB as a ZFS RAIDZ volume that I also share (and mount locally) via NFS to allow me to use it for backups as well as KVM storage for some of the other VMs I have. I use this also within all of my Docker instances as persistent storage (via NFS). I had some fun getting this working with RancherOS! (I had to create a Docker system-level Application as "--net host" to enable NFS mounting for other Docker apps on the same VM).

I set up a couple of small partitions on the 1TB SSD to use L2ARC and ZIL partitions for the ZFS pool.

I am really not using that much of the resources of all of the storage options I have. There's about 900GB free on my Samsung SSD that's currently not even partitioned. I will get around to it eventually. I have found the performance of the zpool to be definitely fast enough for what I need. Maybe I should move persistent storage to the SSD and have the zpool as bulk data storage.... Hmmmm

Hope this helps!

Thanks

 

[Magneto]

Which NVME do you have?