[SOLVED] Confirming my understanding of PVE HA

[unassassinable]

Having read the docs (Yes I did!) I have setup HA according to how I think I understand it. Please confirm or fix my understanding!

I have 4 PVE hosts in a cluster (PM01, PM02, PM03, PM04).
PM01 & 02 have an additional NIC interface hosts 03 and 04 do not. Therefor I want to ensure my pfSense VM cluster (2 node cluster: PF01 & PF02) remain on PM01 & PM02, and preferably on separate hosts from each other.

In the web UI, I created 2 HA groups:

• ID: pfSense-PF01
• Restricted: Checked
• Node PM01, priority 2
• Node PM02, priority 1

and

• ID: pfSense-PF02
• Restricted: Checked
• Node PM01, priority 1
• Node PM02, priority 2

I then added 2 resources (one fore each pfSense VM) under the HA section:

• PF01
  • VM: 100
  • Group: pfSense-PF01
• PF02
  • VM: 101
  • Group: pfSense-PF02

My understanding is that

1. PVE will startup PF01 on PM01, and in the event PM01 goes down the cluster will migrate it to PM02.
2. PVE will startup PF02 on PM02, and in the event PM02 goes down the cluster will migrate it to PM01.

My questions:

1. Is this correct?
2. Does this ensure when the hosts are all coming online in the event of a total power failure (and UPS are dead) the cluster will startup these VMs on the expected hosts (PF01 will start on PM01, and PF02 will start on PM02)?
3. When PM01 comes back online, does HA auto migrate PF1 back to PM01?

 

[sb-jw]

I have 4 PVE hosts in a cluster (PM01, PM02, PM03, PM04).

An even number in the HA cluster is not recommended. You should always make sure that you have an odd number to ensure quorum at all times.

My understanding is that

1. PVE will startup PF01 on PM01, and in the event PM01 goes down the cluster will migrate it to PM02.
2. PVE will startup PF02 on PM02, and in the event PM02 goes down the cluster will migrate it to PM01.

Now that depends somewhat on your definition. Proxmox can currently only do HA and no FT, which means that the VM will be migrated, but the last state will be lost.

Furthermore, HA also requires that you have shared storage or at least use ZFS replication. However, you may also lose data, depending on how long ago the last sync was.

Is this correct?

At least I can't see any major error, for now I would say that's how it should work. But, you should definitely test this in real life - only then will you find out how your systems behave and what might happen.

Does this ensure when the hosts are all coming online in the event of a total power failure (and UPS are dead) the cluster will startup these VMs on the expected hosts (PF01 will start on PM01, and PF02 will start on PM02)?

The end result will be like this. However, I'm not sure whether the VM is first started and then migrated or first migrated and then started.

When PM01 comes back online, does HA auto migrate PF1 back to PM01?

I thought you read the documentary? ;-)
As long as you haven't checked the "nofailback" checkbox, the CRM will always try to run the VM on the node with the highest ID.

 

[unassassinable]

An even number in the HA cluster is not recommended.

We are currently building a 5th node, so that will help.

Proxmox can currently only do HA and no FT.... Furthermore, HA also requires that you have shared storage

Yep, HA only. We have SANs via iSCSI, so all good there.

The end result will be like this. However, I'm not sure whether the VM is first started and then migrated or first migrated and then started.

Gotcha, will test for that.

I thought you read the documentary? ;-) ... "nofailback"

I dun did! Nothing stuck out as I read over nofailback, so I totally missed it when reading. Thanks for pointing out this option!!