Best Practice for 2-Node Storage

A

akjome

New Member
Apr 22, 2025
1
0
1
My boss is pushing for two node clusters rather than four node to "reduce blast radius". I have been testing with ProxMox over the last few weeks.
Note: I love Ceph, it was simple to implement on a four node test cluster, and HA/Snapshots etc. worked very well.

However... Because of the push for two node clusters, I cannot use Ceph.
So far, I have created a two node cluster with a quorum device.
Tested ZFS. I like the snapshot capabilities but I do not like needing to have storage replicated for "HA". Seems like a waste of space and the potential for large amounts of data loss I also do not like.

I have attempted to use StarWind:
https://www.starwindsoftware.com/re...-controller-virtual-machine-cvm-using-web-ui/

I am able to attach the volumes created in StarWind, but they get attached to ProxMox as LVMs, which does not support snapshots. I have tried to use zfs volumes in StarWind and performed a pool import on my PVE nodes, however those do not get added to ProxMox as a "ZFS Pool". That just adds another local ZFS option which does not support HA...

My question... What does everyone recommend for a two node cluster with a quorum device for storage? Keep in mind that we need to have: Snapshot capabilities, shared storage, Thin provisioning.

I have seen the zfs over iSCSI option brought up, but I am not able to find any great documentation on the setup.

If your response is "Do not use ProxMox in a production environment without Ceph storage/4 node clusters", please also include that in the chat because then you would be agreeing with me :)
 
akjome said:
"Do not use ProxMox in a production environment without Ceph storage/4 node clusters",
Click to expand...
Do not deploy production PVE installation with either 2 or 4 nodes. Either go with 3, or implement a QDevice for quorum.
As for storage, it will depend on your budget and availability requirements. At least the later seems to be very lax.

Good luck

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
ZFS on top of iSCSI is not recommended because ZFS is not a shared filesystem, so it will error out on imports unless you do some unsafe stuff. You can point your storages at an iSCSI device that supports snapshots on its end (eg TrueNAS Enterprise, Nexenta, the expensive proprietary stuff). Blockbridge sells Proxmox integrated storages and there’s a few other SDS (at this point we are talking price points in the area of a new machine just for licensing).

Not sure what is meant by blast radius, but you’ll need a quorum device (third cloud system or whatever) to make sure your cluster doesn’t hang the minute one node goes down. Someone doesn’t seem to understand HA.
 
Last edited:
You must log in or register to reply here.
Top Bottom