[SOLVED] CPU Hotplug in Windows Server 2016/win10/win2012r2 did not work correctly

[fireon]

Good evening

pve-manager/5.1-36/131401db (running kernel: 4.13.4-1-pve)

First Thanks! CPUhotplug in Linux works a long time perfectly. So we would like to use the nice feature too in windows. A time ago is used it on win2012r2 Server. But since last updates (Windows/Proxmox...) this feature will not work again. Not in Server 2012r2 not in Server 2016. In win10 i can see the new cores in the devicemanager, but sometimes not all. For example: i add 4 cores, only 3 were available in devicemanager. And in taskmanager i can't see the new cores.

At server 2016 and 2012 after i added new cores to the machine they reset always. I tested this on different hardware with differnt cputypes (qemu and host) and with different storagepools. Always the same. I installed the failure hosbusdriver too... and so on...

I've talked also to the pvesupport and at there labs this is working, but there hardware is different. We use here intel, and they use amd.

I always searched the documantation:

Modern operating systems introduced the capability to hot-plug and, to a certain extent, hot-unplug CPUs in a running systems. Virtualisation allows us to avoid a lot of the (physical) problems real hardware can cause in such scenarios. Still, this is a rather new and complicated feature, so its use should be restricted to cases where its absolutely needed. Most of the functionality can be replicated with other, well tested and less complicated, features, see Resource Limits.
In Proxmox VE the maximal number of plugged CPUs is always cores * sockets. To start a VM with less than this total core count of CPUs you may use the vpus setting, it denotes how many vCPUs should be plugged in at VM start.
Currently only this feature is only supported on Linux, a kernel newer than 3.10 is needed, a kernel newer than 4.7 is recommended.

I not knowed that this feature is not supported for windows yet.
The documentation talked also from the Resource Limits:

In addition to the number of virtual cores, you can configure how much resources a VM can get in relation to the host CPU time and also in relation to other VMs. With the cpulimit (Host CPU Time') option you can limit how much CPU time the whole VM can use on the host. It is a floating point value representing CPU time in percent, so `1.0 is equal to 100%, 2.5 to 250% and so on. If a single process would fully use one single core it would have 100% CPU Time usage. If a VM with four cores utilizes all its cores fully it would theoretically use 400%. In reality the usage may be even a bit higher as Qemu can have additional threads for VM peripherals besides the vCPU core ones. This setting can be useful if a VM should have multiple vCPUs, as it runs a few processes in parallel, but the VM as a whole should not be able to run all vCPUs at 100% at the same time. Using a specific example: lets say we have a VM which would profit from having 8 vCPUs, but at no time all of those 8 cores should run at full load - as this would make the server so overloaded that other VMs and CTs would get to less CPU. So, we set the cpulimit limit to 4.0 (=400%). If all cores do the same heavy work they would all get 50% of a real host cores CPU time. But, if only 4 would do work they could still get almost 100% of a real core each.

For me this sounds really nice, and i tested it on windows2016 Server with a cpubenchmarktool. And yes it works excatly as it described.

So before i poste configs and details... is it possible with special vmsettings to get cputhoplug working also with intelcpu's? How are your experiences with this? And what does the limits mean?

I think it is ok to use "Resource Limits Function". It looks like the same result, and i can reduce the limit too durning a vm is running.

Thanks a lot!

 

[fireon]

Resource Limits is the best! You can change the cores up and down durning a vm is running

 

[sienar]

the workaround mentioned above doesn't necessarily play well with windows kernel scheduler.

see this thread for how to fix Server 2016 to allow proper hotplugging with QEMU/Proxmox
https://forum.proxmox.com/threads/windows-2016-cpu-hot-plug-support.42302/

 

[Lioh]

Does Hot-Unplug also works for you like this? CPU Hotplug works fine with the suggested fixes, but unplug doesn't here.