Would a four/five node ZFS storage replication with HA cluster work?

[casparsmit]

Hi All,

I've setup many ZFS two node clusters in the past (technically 3 node using a small HA node) and CEPH clusters in the past but i'm thinking about the following idea:

4 main nodes and a small HA node

1. Create a cluster using all five nodes.
2. Setup ZFS storage replication between node 1 & 2
3. Setup ZFS storage replication between node 3 & 4
4. Create VM's on nodes 1&2 using HA groups to limit them to nodes 1&2
5. Create VM's on nodes 3&4 using HA groups to limit them to nodes 3&4
6. Live migration of a vm between node 1&2 should work instantly because of the replication between the two
7. Live migration of a vm between node 3&4 should work instantly because of the replication between the two

Would this setup work?
Would a live migration of a VM between node 1 & 3 for instance work in this case? (In the release history of PVE 6.0 it should be possible to live migrate using local storage e.g ZFS). Sure it would take much longer but that's not a problem, just wondering if this is feasible at all.

Kind regards,
Caspar

 

[aaron]

The live migration should work but I am not sure how the HA groups might interfere there.

You also have to be aware that in the case of a node failure where the VM will be started on the node where it is replicated to, you are likely to have some data loss since the last replication. If that is okay, this could be a working, but complicated setup.