VMware Refugee - Replacement for vSAN on 2-Node Cluster

[Ghan]

Hey folks,

Due to the recent changes in VMware coming from Broadcom, it looks like I'm going to need to find a new solution for my home lab cluster. Today, I'm running 2 primary nodes with a smaller 3rd host as a witness for HA and vSAN storage. This allows for any host to fail and the cluster will keep running, VMs reboot on the other host, etc. It's also very convenient for updates and hardware swaps. Can this set up be done in Proxmox, and if so, which solutions in Proxmox's broad feature set would be best to use?

 

[aaron]

If you only want to use built in options, you could use ZFS for the local storage disks and then use the Replication feature. The downside, this is async replication. The shortest possible interval is 1 minute.
You can combine it with HA, but in the worst-case scenario, you might see some small data loss, depending on the interval time and when the last successful replication ran.

You will need a 3rd vote for the cluster. This can be a full node or just an external small Linux box if you use a QDevice: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_corosync_external_vote_support

 

[Ghan]

I'd really like to have HA storage as well, which seems to be lacking here. Corosync external vote doesn't work with clustered storage does it?

 

[Johannes S]

I'd really like to have HA storage as well, which seems to be lacking here. Corosync external vote doesn't work with clustered storage does it?

Corosync external vote doesn't depend on a specific storage solution. For Storage HA: ZFS replication sync schedule is 15 minutes by default but can be reduced to one minute. If this isn't enough for your usecase you could go with Ceph ( needs at least three nodes ) or a third-party solution. Some users here use Starwinds vsan together with Proxmox VE for small Clusters maybe this is an option for you?

 

[Ghan]

Corosync external vote doesn't depend on a specific storage solution. For Storage HA: ZFS replication sync schedule is 15 minutes by default but can be reduced to one minute. If this isn't enough for your usecase you could go with Ceph ( needs at least three nodes ) or a third-party solution. Some users here use Starwinds vsan together with Proxmox VE for small Clusters maybe this is an option for you?

I've heard of Starwinds before so I'll have to take a closer look, thanks!

With the hardware I have, I can do 2 node + witness so it sounds like Ceph won't be an option. Synchronous storage isn't a 100% hard requirement, but I do have that today with VMware and it is very convenient for maintenance and outages, so that's my goal.

 

[aaron]

it is very convenient for maintenance

With the Replication enabled, a (live) migration of a VM between the nodes will be quick as only the latest delta of the disk image needs to be transferred. Therefore, in a maintenance situation, it will be almost as fast as if the disk image would be on a shared/sync storage.