CPU cores overcommit

[Paspao]

Hello,

what does happen if we overcommit total of CPU cores assigned to VMs higher than number of cores available on host?

Does Proxmox gives any alert during assignment?

Is it the same behaviour for LXC and KVM?

In my case I have 40 cores on host and 30 LXC with 1 core. What happens if I assign 2 cores to them totalling 60?

From what I see in pct cpusets it assign the same core to multiple LXC so they share the power, is it right?

Documentation describes cpuunits and cpulimit but it would be useful to have more information on this.

Thank you
P.

 

[t.lamprecht]

what does happen if we overcommit total of CPU cores assigned to VMs higher than number of cores available on host?

So there are two different scenarios, the total core count of a single VM/CT cannot exceed the host available core count.
The other one can just fine, but if all VMs and/or CTs are using them you will naturally not get each virtual core to full performance, they will get scheduled to a core in alternation.
But if the VM/CT are not doing compute work, their virtual core won't produce (almost any) CPU load on the host system.

In my case I have 40 cores on host and 30 LXC with 1 core. What happens if I assign 2 cores to them totalling 60?

They can each schedule two task simultaneously, but the totaling task count running in parallel can never exceed the count of cores your host has.

Often, processes need to wait on IO (disk or network, for example) to finish, there over commitment may even get you a better total host resource usage without much "penalty".
But if all your VMs/CTs do a lot of heavy compute stuff you will generate some performance penalty, the scheduler will switch the "virtual cores" so that each gets a fair share.
This can additionally be influenced with cpuunits and -limits, as the documentation explains.

So, in conclusion, if this and how much this can/should be done depends on what your VMs/CTs do. I know setups where there are 20 times as many CT cores assigned than host cores are there, but each CT just serves an occasional website hit through static nginx websites. On the otherhand there are setups where there are even cores "left over", e.g., if one uses cpu intensive services on the host, like ceph OSDs/Monitors, or one just wants to guarantee that a VM always has the 100% of all it's assigned cores available all of the time.

 

[Paspao]

Thank you for the clear explanation.

I suggest you to add this explanation to documentation as it is really useful.

I have a Ceph Hyper-converged setup, How can I assign some cores for OSDs/Monitors and avoid VMs to use them?

Any updates to this? https://forum.proxmox.com/threads/how-to-pin-cpu-cores-to-ceph-osd.37907/

Thank you.
Best regards.
P.