New to Proxmox and confused about sockets, cores, and vCPU's

[kevindd992002]

So I understand what a socket, a physical core, and a thread is on the physical hardware. I have read a lot of discussions (in this forum and in other forums) but it really just makes me more confused, so I resorted to posting here.

So my main question here is why are there sockets and cores in a VM configuration? Why not vCPU's only? I understand that one vCPU is processed by one thread by the hypervisor, so what is the use of sockets and cores in the VM? I'm reading that sockets are there for NUMA. Is that the only reason?

For example, if my host has 8 available threads, would these configs have effectively the same performance outcome?

1 socket, 8 cores, 1 vCPU
1 socket, 1 core, 8 vCPU's
2 sockets, 4 cores, 1 vCPU
2 sockets, 1 core, 4 vCPU's

 

[Dunuin]

So my main question here is why are there sockets and cores in a VM configuration? Why not vCPU's only?

Because if you know about physical sockets you should know that the link between them is a limiting factor and for example CPU in socket A will be slow when accessing data from RAM connected to socket B.
Here it helps to enable NUMA und work with virtual sockets so the hypervisor could better utilize the physical hardware. You for example tell the VM to only use one socket so the full VM is accessing only fast RAM of the same single socket. Or you got a VM that needs to access two NICs and one NIC is connected to socket A and the other NIC is connected to socket B. Then it might be useful if the VM would run on both sockets so it got faster access to both NICs.

And then there is licensing where you pay per socket...

 

[kevindd992002]

Ok. So barring the sockets piece (socket=1), what is the difference between cores and vcpu's inside the VM? From what I read, sockets * cores is usually limited to the total number of hardware threads that your system has. And the vCPU field just limits what is being presented to the VM.

So if you have 1 socket, 4 cores, and 2 vCPUs, you're basically limiting the VM to just use 2 harware threads at all times? Or will it start at 2 hardeare threads and balloon to 4 as necessary?

 

[BobhWasatch]

VM's don't know about HW threads. (edit: I mean, it may know about them but it doesn't really matter).

Each vCore is virtualized and gets executed on whatever actual HW thread the hypervisor decides. It also should be noted that a second HW thread on a core only gives you maybe a 50% performance boost, probably less. So having 16 threads on 8 cores is really kind of like having 10 or 12 cores, depending on what the workload is.

So, if you set the VM to have two cores the hypervisor will execute those on 2 hardware threads which may or may not be on the same physical core depending on system load and other factors. To anticipate your next questions:

• Yes, you can oversubscribe your physical threads and nothing bad will happen. Except possibly bad performance depending on what the workload of all your VM's combined looks like. Cloud providers make their money on this kind of arbitrage.
• No, each VM can't have more vCores than the number of physical threads available. That wouldn't make much sense.
• Even having a VM with vCores == HW threads is a bad idea. You should not allocate more cores to a VM than it actually needs because it makes the scheduler's job harder and can result in worse performance. Plus the hypervisor itself needs some resources.
• Yes there is a way to lock VM's to particular cores but if you think you need to do that why are you virtualizing that workload to begin with?

Virtualization is not magic. If you have a workload that really needs the whole machine, then you should not virtualize it.

 

[kevindd992002]

Gotcha. What then is the use of the vCPU advanced option? What is its use case?

 

[fabian]

please read the whole CPU part of the docs, it explains it: https://pve.proxmox.com/pve-docs/chapter-qm.html#qm_cpu

 

[kevindd992002]

please read the whole CPU part of the docs, it explains it: https://pve.proxmox.com/pve-docs/chapter-qm.html#qm_cpu

I did read that before posting but have some clarifications. First, there's a typo where it says "vpus" here instead of "vcpus""

In Proxmox VE the maximal number of plugged CPUs is always cores * sockets.To start a VM with less than this total core count of CPUs you may use the vpus setting, it denotes how many vCPUs should be plugged in at VM start.

Second, what does "at VM start" mean" If I have a total core count of 4 and set the vCPUs to 2, the VM starts with 2 vCPUs and then what? What happens to the other 2?

 

[fabian]

the other are not plugged in, but if your guest OS supports it (and your VM config allows it) you can hotplug them at a later point (and potentially even unplug them again, but that is more experimental) - it says so in the paragraph above the one you quote

 

[kevindd992002]

the other are not plugged in, but if your guest OS supports it (and your VM config allows it) you can hotplug them at a later point (and potentially even unplug them again, but that is more experimental) - it says so in the paragraph above the one you quote

Right. I guess I just don't understand what its use case is. Why would you assign a higher total core and then just limit it with vCPUs. It's an option that, for me, will generally not be used but I'm sure other people have their own use cases.

 

[Dunuin]

Right. I guess I just don't understand what its use case is. Why would you assign a higher total core and then just limit it with vCPUs. It's an option that, for me, will generally not be used but I'm sure other people have their own use cases.

Think of productive use where you can't have any downtime. Without hot plugging you can only change the amount of vCPUs, RAM and so on when stopping the VM. When you are not allowed to stop that VM for months you can't increase the CPU power of a VM. So it might be useful to assign that VM more ressources than it actually needs, but then not use it, so you might be able to later increase it, in case you realize you still need more power.

 

[kevindd992002]

Think of productive use where you can't have any downtime. Without hot plugging you can only change the amount of vCPUs, RAM and so on when stopping the VM. When you are not allowed to stop that VM for months you can't increase the CPU power of a VM. So it might be useful to assign that VM more ressources than it actually needs, but then not use it, so you might be able to later increase it, in case you realize you still need more power.

Ok, that makes more sense now. Thanks for the example.