Home LAN cluster of low power nodes + distributed storage?

[Ahks]

Hi all,

I have been pointing myself toward some form of resilient distributed storage with container/vm replication (or HA). I have 5 or 6 small nodes of x86 hardware (odroid h2+'s and couple old desktop pc's), and a VM node on my gaming PC. All have between 8GB and 32GB of RAM, 4+ CPU cores, a disk for OS, and a separate 4TB drive attached. My initial thought was to build out my own LXD cluster and setup GlusterFS in an n+2 configuration so I could lose 1 or 2 of the nodes without having to restore from backup but I feel like my work so far would have been MUCH easier with the management tools in PVE.

The goals I have are
1) to be able to add/remove nodes and storage as needed without having to restore from backup,
2) be able to upgrade drives in a way that doesn't require a backup and restore, and
3) have enough storage throughput from the cluster to have 3 concurrent 1080p streams from plex/jellyfin/etc

This all seems fairly "simple", but I'm sure there are complications I don't see. Any advice from experience?

Cheers

 

[LnxBil]

I think that GlusterFS is a good starting point given your requirements. Another solution would be CEPH, but that will not be very fast but may be enough for you. 3x 1080p are normalle 120 mBit, so 15 MB/s. Could work, you have to try if it satisfies your needs (reading will not be the limiting factor here, but writing)