Would a four/five node ZFS storage replication with HA cluster work?

casparsmit

Renowned Member
Feb 24, 2015
41
2
73
Hi All,

I've setup many ZFS two node clusters in the past (technically 3 node using a small HA node) and CEPH clusters in the past but i'm thinking about the following idea:

4 main nodes and a small HA node

1. Create a cluster using all five nodes.
2. Setup ZFS storage replication between node 1 & 2
3. Setup ZFS storage replication between node 3 & 4
4. Create VM's on nodes 1&2 using HA groups to limit them to nodes 1&2
5. Create VM's on nodes 3&4 using HA groups to limit them to nodes 3&4
6. Live migration of a vm between node 1&2 should work instantly because of the replication between the two
7. Live migration of a vm between node 3&4 should work instantly because of the replication between the two

Would this setup work?
Would a live migration of a VM between node 1 & 3 for instance work in this case? (In the release history of PVE 6.0 it should be possible to live migrate using local storage e.g ZFS). Sure it would take much longer but that's not a problem, just wondering if this is feasible at all.

Kind regards,
Caspar
 
The live migration should work but I am not sure how the HA groups might interfere there.

You also have to be aware that in the case of a node failure where the VM will be started on the node where it is replicated to, you are likely to have some data loss since the last replication. If that is okay, this could be a working, but complicated setup.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!