Windows 2016 CPU Hot plug support.

Discussion in 'Proxmox VE: Installation and configuration' started by scott.ehas, Mar 19, 2018.

  1. scott.ehas

    scott.ehas New Member

    Joined:
    Oct 5, 2017
    Messages:
    6
    Likes Received:
    0
    Hello All,

    I am trying to get CPU Hotplugging support working on Windows 2016. The VM has the latest updates, drivers, and qemu guest agent. I have hotplugging enabled on the Proxmox 5.2 end with Numa & vCPU cores enabled. Do I need any special configurations on the guest OS side to get this working?

    Memory hotplugging worked out of the box.
     
  2. Alwin

    Alwin Proxmox Staff Member
    Staff Member

    Joined:
    Aug 1, 2017
    Messages:
    1,626
    Likes Received:
    140
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. sienar

    sienar Member
    Proxmox VE Subscriber

    Joined:
    Jul 19, 2017
    Messages:
    37
    Likes Received:
    5
    I too had issues with CPU hotplugging on Proxmox inside a Windows Server 2016 guest. Getting it working required fixing some things inside the guest. My digging actually found a bug reported to Redhat for QEMU that is really an ACPI bug in Server 2016 (and Win10). I'll refer to this link - https://bugzilla.redhat.com/show_bug.cgi?id=1377155. Essentially MS has non-standard compliant ACPI configuration out of the box, but can be fixed fairly easily to work with QEMU/Proxmox. Essentially because of the ACPI issue, Windows is not able to see any CPUs not there at boot, AND actually NO CPU's show up in the device manager. AND you get an error'd "HID Button over Interrupt Driver" device in the device manager. This process fixes CPU hotplugging and fixes the error'd device in device manager.

    I actually wrote a script to automate this process as well. Pretty simple bat script. To run the script you need two executables from Microsoft that are not included in Windows. You will need to get psexec.exe from sysinternals to run reg command in the system context to delete the bad regkey's. And you'll need to get devcon.exe from the Windows Driver Development Kit (WDK). To avoid installing the whole kit, which you likely don't need, follow this link and do the administrative install which just downloads all the kit components. When you get to the point of extracting the MSI, the correct one is Windows Driver Kit-x86_en-us.msi not the one mentioned in the link which actually doesn't exist as the post is pretty ancient.

    Run the script or do the steps manually and reboot the guest. CPU hotplugging (at least to add cores) will then work.


    Script is here:

    Code:
    @echo offecho
    echo "removing bad device"
    \\fileserver\shares\Software\_Scripts\fix_acpi_proxmox\devcon.exe remove "ACPI\VEN_ACPI&DEV_0010"
    pause
    echo "removing 1st regkey"
    \\fileserver\shares\Software\_Scripts\SysinternalsSuite\PsExec.exe /accepteula /s reg delete "HKLM\SYSTEM\DriverDatabase\DriverPackages\hidinterrupt.inf_amd64_d01b78dcb2395f49\Descriptors\ACPI\ACPI0010" /f
    pause
    echo "removing 2nd regkey"
    \\fileserver\shares\Software\_Scripts\SysinternalsSuite\PsExec.exe /accepteula /s reg delete "HKEY_LOCAL_MACHINE\SYSTEM\DriverDatabase\DeviceIds\ACPI\ACPI0010" /f
    pause
    echo rebooting
    shutdown /r /t 30
     
    #3 sienar, May 3, 2018
    Last edited: Oct 8, 2018
  4. Pourya Mehdinejad

    Proxmox VE Subscriber

    Joined:
    Nov 8, 2017
    Messages:
    38
    Likes Received:
    0
    Tried the suggested workaround, but it didn't work.
    still when I add more cores to CPU while the VM is running, it becomes red and will be pending till next restart.
     
  5. sienar

    sienar Member
    Proxmox VE Subscriber

    Joined:
    Jul 19, 2017
    Messages:
    37
    Likes Received:
    5
    Did you also go into the options on the VM and add CPU to the hotplug list?

    upload_2018-10-7_11-39-58.png
     
  6. Pourya Mehdinejad

    Proxmox VE Subscriber

    Joined:
    Nov 8, 2017
    Messages:
    38
    Likes Received:
    0
    Yes it was already selected.
     
  7. sienar

    sienar Member
    Proxmox VE Subscriber

    Joined:
    Jul 19, 2017
    Messages:
    37
    Likes Received:
    5
    And the Qemu agent is installed and running?
     
  8. sienar

    sienar Member
    Proxmox VE Subscriber

    Joined:
    Jul 19, 2017
    Messages:
    37
    Likes Received:
    5
    Also, did you have the error'd "HID Button over Interrupt Driver" device in the device manager and now it's gone?
     
  9. Pourya Mehdinejad

    Proxmox VE Subscriber

    Joined:
    Nov 8, 2017
    Messages:
    38
    Likes Received:
    0
    Dear Sienar
    Yes, of course, the agent is installed, enabled and running.
    Hot-plug functionality is working fine for RAM and HDD but not for CPU.
    The error was there and now it's gone.
     
  10. sienar

    sienar Member
    Proxmox VE Subscriber

    Joined:
    Jul 19, 2017
    Messages:
    37
    Likes Received:
    5
    Are you able to see CPU devices in Device Manager, one for each core the VM booted with? If yes, if you delete them and scan for hardware changes, do they come back?
     
  11. sienar

    sienar Member
    Proxmox VE Subscriber

    Joined:
    Jul 19, 2017
    Messages:
    37
    Likes Received:
    5
    Also, how do you have the CPUs configured for the VM? I ask because, there is correct way and wrong way to configure them to use hotplugging.

    upload_2018-10-8_13-3-19.png

    Here's an example of how to do it. 'Cores' needs to be set to the max number of CPU cores the VM will ever be able to use. This number does not directly impact the number of virtual cores the VM boots with, it is the maximum number of cores the VM will potentially have access to without being shutdown. VCPUs will be set to the actual number of cores the VM should have access to on boot. To hot plug more cores, that is the setting to be increased, up to the number listed in 'Cores'. The above example VM would start with 4 cores, and be able to hot plug an additional 4 virtual cores while running.

    edited for clarity
     
  12. Pourya Mehdinejad

    Proxmox VE Subscriber

    Joined:
    Nov 8, 2017
    Messages:
    38
    Likes Received:
    0
    I tied it, it worked but the moment I increased the vCPU cores, the VM got hunged and rebooted :(
    It seems Proxmox and Windows guests are not best buddies.
     
  13. Pourya Mehdinejad

    Proxmox VE Subscriber

    Joined:
    Nov 8, 2017
    Messages:
    38
    Likes Received:
    0
    can we have an update from Proxmox staff, is anyone planning to fix this issue?
     
  14. spirit

    spirit Well-Known Member
    Proxmox VE Subscriber

    Joined:
    Apr 2, 2010
    Messages:
    3,137
    Likes Received:
    101
    Win10/2016 cpu hotplug bug was an Microsoft bug and has been fixed in last Windows update. Win2012r2 works out if the box for me. (Plug & unplug )
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    #14 spirit, Oct 11, 2018
    Last edited: Oct 12, 2018
    sienar likes this.
  15. sienar

    sienar Member
    Proxmox VE Subscriber

    Joined:
    Jul 19, 2017
    Messages:
    37
    Likes Received:
    5
    It's really not a Proxmox problem. Even happened in VMWare - https://vinfrastructure.it/2018/05/windows-server-2016-reboot-after-hot-adding-cpu-in-vsphere-6-5/

    You seem to be configured correctly and know how to actually add vCPUs now, you weren't previously configured right or weren't doing it correct previously. Now make sure your Server 2016 install is fully up to date and try again.
     
  16. spirit

    spirit Well-Known Member
    Proxmox VE Subscriber

    Joined:
    Apr 2, 2010
    Messages:
    3,137
    Likes Received:
    101
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  17. Pourya Mehdinejad

    Proxmox VE Subscriber

    Joined:
    Nov 8, 2017
    Messages:
    38
    Likes Received:
    0
    I'm not trying to be the only Naysayer
    But it's still not working for none of my Windows VMs.
    I have tried to upgrade the latest windows update, Double checked QEMU, etc...
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice