API (atomic) call to add resource to HA affinity rule

[rmerz]

Hello,

With Proxmox 8.x it was possible to add a VM, e.g. with ID "1023", to an HA group via HTTP POST to "cluster/ha/resources"
to the group "mygroup" with the following body:

{"sid":"vm:1023","group":"mygroup"}

With Proxmox 9.x this doesn't work because HA groups no longer exist, but rather HA Affinity Rules.

As far as I can see, you first need to register a VM as an HA resource.
This requires an HTTP POST to "cluster/ha/resources" with the following body as an example:

{"sid":"vm:1023"}

After that, you still need to add the VM to the corresponding HA Affinity Rule.
This requires an HTTP PUT to "cluster/ha/rules/mygroup" with the following body as an example:

{"resources":"vm:1023","type":"node-affinity"}

That works. The problem is that I have to provide all VMs in "resources" with this PUT.

So you would first need to determine via HTTP GET what's currently in "resources",
add VM 1023, and then do the HTTP PUT.

But that's not atomic:
What if someone has modified the resources of this Affinity Rule after the HTTP GET but before the HTTP PUT?

To clarify this further, here's a scenario:

System A wants to add VM 23 and System B wants to add VM 42.

System A makes an HTTP GET
System B makes an HTTP GET
System A makes an HTTP PUT and adds VM 23 to the list of "resources"
System B makes an HTTP PUT and adds VM 42 to the list of "resources"
-> VM 23 is now not in the Affinity Rule.

Additionally, Proxmox itself also modifies the Affinity Rule (e.g., when deleting a VM?).
So even if you had Systems A and B under control (e.g., both systems use their own Affinity Rule instead of sharing one),
you wouldn't find a clean solution with Proxmox itself.

-> With 2 calls "read and then write" it's never clean from my perspective and always leads to problems, as it's not atomic.

What is the API call to simply add a VM to an Affinity Rule?

What I've already tested is making an HTTP POST instead of an HTTP PUT.
But POST doesn't seem to exist ("status code: 501, reason phrase: Method 'POST /cluster/ha/rules/mygroup' not implemented")

I can't imagine that this capability doesn't exist.
Especially with larger or more complex installations, these are basic requirements that such operations must be atomic,
so I can't imagine that this call doesn't exist.

Thanks
Regards

 

[dakralex]

Hi!

With GET /cluster/ha/rules and GET /cluster/ha/rules/{ruleid} you receive a digest value (of the whole config). If you provide this digest as an argument to PUT /cluster/ha/rules/{ruleid}, then the API handler will refuse to make changes to the HA rules if the digest (i.e. the rules config) has changed from the digest you compare it with. To create new HA rules, there is POST /cluster/ha/rules.

It could be an interesting feature to allow HA resources - at least when those are created - to be added to existing HA rules.. The only important part is that consistency of the HA rules is still checked. If you're interested in such a feature, you can create a Bugzilla entry [0] so we can track it.

[0] https://bugzilla.proxmox.com/enter_bug.cgi?product=pve&component=HA