Noob needing help with storage configuration / maximizing my benefits of using virtualization

[Kebel87]

Hello everyone,

I've been using promox for 2-3 years, but in a very simple and straightforward way (1 node, 1 disk, 2-3 VMs (Home Assistant, UniFi controller and a DebianOS to tinker with).

Last summer I was given 2 "old" computer that I integrated in a cluster with my primary node. Now my "datacenter" (lol) looks like this :

Pve1 (main) :
CPU : i5 6600k - 4 cores
RAM : 32 gb
Disks : 1x 250gb SSD + 1x 1tb HDD

Pve2 :
CPU : i7 870 - 8 cores
RAM : 8 gb
Disks : 1x 120gb SSD + 1x 750gb HDD

Pve3 :
CPU : i5 6500 - 4 cores
RAM : 8 gb
Disks : 1x 250gb SSD


Pve2 and Pve3 doesn't run any VM, they're just integrated to the cluster.

Given the hardware I have in hand, what would my best configuration be (particularly storage, where I'm at loss trying to understand the difference between LVM, LVM-Thin, ZFS, dir, etc.)

My ultimate goal is to maximize the benefits of running those 3 nodes, while not buying more hardware (except SSD if needed).

What I'd like to achieve first is to be able to get failover (manual if needed) for my HomeAssistant VM and my UniFi controller. I don't need high availability, and I don't care if I lose some data in the event of a failure. I just want to be able to flip the VM to another host fairly easily and quickly. I understand ZFS might be what I need, but I'm not capable of creating a ZFS pool as my disks are in LVM (I think?)

Finally, if I need to format disks to rearrange all of my storage, how will I be able to restore everything after? Should I temporarily use a USB key or something?

I know I'm asking a lot and I might not have given all the necessary information, but please ask any other relevant info needed and I'm gladly provide it.

Thanks for your help and happy holidays

 

[Dunuin]

Given the hardware I have in hand, what would my best configuration be (particularly storage, where I'm at loss trying to understand the difference between LVM, LVM-Thin, ZFS, dir, etc.)

My ultimate goal is to maximize the benefits of running those 3 nodes, while not buying more hardware (except SSD if needed).

To make really use of the cluster you usually want a shared storage (ceph/NFS) or local storage with replication (ZFS) so you can easily migrate guests between the nodes. But with NFS you get that single point of failure where nothing will be working if that one node running the NFS server will fail.
For ceph you want multiple fast (10+Gbit) NICs and enterprise SSD which you probably don't got. ZFS wants enterprise SSDs too.
So not great hardware to run a cluster.

Your best bet probably would be to use local LVM-Thin storage per node but then it is like running 3 unclustered nodes without the benefits of the shared storage that would allow you to use HA or fast migration. Other option would be to use ZFS but then don't expect too much. Depening on the workload the SSDs will become very slow and might fail very soon.
And with ZFS you only got like 80GB of combined replicated storage for your VMs for all 3 nodes. So you probably want to buy some same sized bigger enterprise SSDs anyway if you want to make use of ZFS with replication.

Another option would be to not cluster them at all. Have two unclustered PVE nodes. One PVE node running 24/7 with all the VMs/LXC and the other one always power off with no VMs/LXCs on it. the third machine as PBS for backups. In case your 24/7 node would fail you could then power on the other PVE node and restore the backups from the third PBS machine.