Add HA to already replicating VMs

[mstoneman]

My question relates to activating HA in a 3 node cluster which uses local ZFS storages.

Each node has two ZFS pools of equal capacity: pve-a ( zfs-a, zfs-b ) pve-b ( zfs-b, zfs-c ) pve-c ( zfs-c, zfs-a )

pve-a (zfs-a) replicates to pve-b (zfs-a)
pve-b (zfs-b) replicates to pve-c (zfs-b)
pve-c (zfs-c) replicates to pve-a (zfs-c)

This round robin style replication allows any VM to be activated quickly but manually if any node goes down.

Id like to activate HA to automate the process but not sure the best way.

Does HA automatically know each VM is already replicated on the next node in the round-robin ?
Will HA try to make another copy of a VM that is already replicated ?
Does HA know not to try and put a VM from pve-a to pve-c (since the only valid destination zfs exists on pve-b).
Will HA be upset that pve-c doesn't have a copy of zfs-a ?

I'm new to this and not sure if this style of setup is unique, apologies if this has already been asked/answered but I couldn't seem to find the answers after researching.

I suspect I may have to create a HA group for each node which only allows VMs from one node to restore on the next. But still unclear if the already replicated machines will be utilized. Also I'm not sure about fencing and creating a watchdog to prevent race conditions after a node restore.

Any feedback is appreciated. Thank you!

 

[jsterr]

Does HA automatically know each VM is already replicated on the next node in the round-robin ?

I dont think HA knows anything about the storage your vms are using. It "just" makes sure that the vm is not running anywhere else and picks a node according to your ha-settings and ha-groups. So you can change a llittle bit by using restriction feature in groups.

Will HA try to make another copy of a VM that is already replicated ?

Ha has nothing to do with copy of vm. The HA stacks moves the qm-file to the fitting host and then starts the vm. If the vm-config is not fitting the local storage on the new node (because for your example, theres no replicated disk, it wont work and vm wont be able to start)

I dont think you can actually tell him that he should always use the next host in your ring.

Does HA know not to try and put a VM from pve-a to pve-c (since the only valid destination zfs exists on pve-b).
Will HA be upset that pve-c doesn't have a copy of zfs-a ?

You can make a HA group to restrict the vm only to be allowed to run on a and b. You should also only replicate VMs between two servers. When your talking bout replicating from b to c, do you mean that b also should replicate the vms (it got from a) to c? in the end? If yes, I dont think this will work. You can put VMs on a, that gets replicated to b. New and other vms on b, can be replicated to c. and new and other vms from c can be replicated to a. You need to make HA-Groups for each Host-Group:

A-B (Group1)
B-C (Group2)
C-A (Group3)

and put your vms in there. and restrict to the fitting nodes.

 

[mstoneman]

Ha has nothing to do with copy of vm. The HA stacks moves the qm-file to the fitting host and then starts the vm. If the vm-config is not fitting the local storage on the new node (because for your example, theres no replicated disk, it wont work and vm wont be able to start)

This makes more sense now. In my case HA should work as long as the HA group restrictions are setup so that the qm-file copies to the node that already has the replicated disk(s).

I was stuck on the idea that HA somehow needed to know details that the replicated vms were already on the destination node. Going to test it out, thank you very much !