Race conditions for mass VM creation

[Marcin Kubica]

Heya

How do you go about race conditions when requesting to create VMs from different "callers"?

It seems Proxmox is unable to assign vmid by itself and it always needs to be supplied, and the best guess is to ask for next free vmid.

However, in anything larger than homelab in case of multiple calls being done at the same time, this results in returning of the same next free vmid to different callers.

Cheers!

 

[dietmar]

This is not really a problem for manual VM creation, because it is seldom enough and the user can simply select another VMID and continue.

For automated tasks, it is up to the caller to allocate ID.

 

[Marcin Kubica]

This is not really a problem for manual VM creation, because it is seldom enough and the user can simply select another VMID and continue.

Ok, but hat's a homelab use case.

For automated tasks, it is up to the caller to allocate ID.

Again, sadly it's a homelab use case making Proxmox not fit for a scalable enterprise solution. (and already available since the beginning of time in kvm)

 

[dcsapak]

why not simply randomly generating ids? with ~32bit it should be enough to not get clashes often, and even then a retry with a different one should be ok

 

[Marcin Kubica]

why not simply randomly generating ids? with ~32bit it should be enough to not get clashes often, and even then a retry with a different one should be ok

Sounds good. Gonna try it.

Why not let proxmox engine do this as an option?

 

[dcsapak]

the biggest problem with something like that is to adhere to the REST principles of the api, a single api call should be idempotent, so a POST call to a specifi url with specific parameters should not do something different if done twice

for example:
there would be a hypothetical api call '/new_vm' where you create a vm with a random id

now a client calls this with the correct parameters, but the network drops out while the server answers (but the vm is already created)
now the client gets a timeout/error and tries again -> there are now 2 vms and the client does not know about the first one

 

[Marcin Kubica]

the biggest problem with something like that is to adhere to the REST principles of the api, a single api call should be idempotent, so a POST call to a specifi url with specific parameters should not do something different if done twice

for example:
there would be a hypothetical api call '/new_vm' where you create a vm with a random id

now a client calls this with the correct parameters, but the network drops out while the server answers (but the vm is already created)
now the client gets a timeout/error and tries again -> there are now 2 vms and the client does not know about the first one

Well in this use case, client must provide a name for vm, so calls for new_vm with name "bla".
The connection gets lost and client doesn't know.
Client calls again to /new_vm name "bla", server would respond "bla" exists.

 

[dcsapak]

yes now the name became the id and it has to choose again...

 

[Marcin Kubica]

No, name stays the name, and assigning an id is to be managed by proxmox.
This relieves clients from the work a cluster should do in an enterprise environment.

 

[dcsapak]

No, name stays the name, and assigning an id is to be managed by proxmox.

yes but the client has to choose a name now, why not choose a name and id ?

 

[Marcin Kubica]

yes but the client has to choose a name now, why not choose a name and id ?

this goes back to the race conditions when mass requesting vm.
if you look how this has been done in virtualbox, they have perfectly worked around this issue by not dropping vm id management on the client