Windows Guest - High CPU usage

[Hugues Depasse]

Hello,

We currently have a 3 nodes cluster and encounter a high CPU usage on all of our Windows Guest. Just opening Google Chrome or the task manager raise the CPU usage to 90% for 3-4 seconds. It is not an IDLE usage.

All guest are installed with latest VIRTIO guest drivers (0.1.141) and the following config :

acpi: 1
agent: 1
autostart: 1
boot: dcn
bootdisk: virtio0
cores: 2
cpu: host,flags=+pcid
cpuunits: 2000
hotplug: 1
ide0: Backup_Stor:iso/virtio-win-0.1.141.iso,media=cdrom,size=309208K
keyboard: fr-be
kvm: 1
localtime: 1
memory: 8192
name: Alka-VM03
net0: virtio=52:54:00:00:90:6f,bridge=vmbr0,firewall=1
net1: virtio=16:80:95:F5:E0:6B,bridge=vmbr10
numa: 1
onboot: 1
ostype: win8
reboot: 1
scsihw: virtio-scsi-pci
smbios1: uuid=5f5ebcc2-bf5b-4dc6-9549-f810befdd841
sockets: 2
virtio0: LVMRPN5TB_1cfgc884q15e:vm-208-disk-1,size=100G
virtio1: LVMRPN5TB_1cfgc884q15e:vm-208-disk-2,size=100G

We already tried to :

• Increase the CPU socket/cores and removed CPU limits, same issue
• Remove NUMA, same issue
• Added PCID flag, same issue
• Move the VM to another node, same issue
• Disable USB Tablet in GUI, same issue
• Issued the "bcdedit /set {default} useplatformclock true" command in the guest, same issue
• Disabled Ballooning, same issue

Thank you for your help

 

[Alwin]

Just opening Google Chrome or the task manager raise the CPU usage to 90% for 3-4 seconds. It is not an IDLE usage.

Is that not a expected behavior on a program start? What is the IDLE usage?

And top of my head for CPU usage it also depends on what host flags the OS supports, if it is up-to-date, how fast its disk storage / network is.

virtio0: LVMRPN5TB_1cfgc884q15e:vm-208-disk-1,size=100G virtio1: LVMRPN5TB_1cfgc884q15e:vm-208-disk-2,size=100G

One place where you could try if it makes a difference, is to set the disks to SCSI (also allows to pass through trim commands) and set the cache of the disks (eg. writeback).
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_virtual_machines_settings

 

[Hugues Depasse]

Is that not a expected behavior on a program start? What is the IDLE usage?

And top of my head for CPU usage it also depends on what host flags the OS supports, if it is up-to-date, how fast its disk storage / network is.


One place where you could try if it makes a difference, is to set the disks to SCSI (also allows to pass through trim commands) and set the cache of the disks (eg. writeback).
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_virtual_machines_settings

Hello,

Sorry for the delay but I had to make some testing again.

I've tried to set the disk to SCSI and writeback, the issue remains, I've uploaded a quick video to show what happens :

-> I tried to put all disk on local SSD drives, same issue, so this is not due to a slow network share (which is a private 10Gbits link) + tried the exact same VM on a freshly installed Proxmox node (not added to my cluster) and there's no issues.

The issue is appearing on all my cluster nodes and it appeared all sudden which is really strange...

My cluster nodes config are :

Node 1 : https://www.online.net/en/dedicated-server/dedibox-miniwopr
CPU(s)
40 x Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz (2 Sockets)

Kernel Version
Linux 4.13.13-4-pve #1 SMP PVE 4.13.13-35 (Mon, 8 Jan 2018 10:26:58 +0100)

PVE Manager Version
pve-manager/5.1-42/724a6cb3

Node 2 : https://www.online.net/en/dedicated-server/dedibox-ent
CPU(s)
32 x Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz (2 Sockets)

Kernel Version
Linux 4.13.13-4-pve #1 SMP PVE 4.13.13-35 (Mon, 8 Jan 2018 10:26:58 +0100)
PVE Manager Version
pve-manager/5.1-42/724a6cb3


Node 3 : https://www.online.net/en/dedicated-server/dedibox-xc
CPU(s)
8 x Intel(R) Atom(TM) CPU C2750 @ 2.40GHz (1 Socket)

Kernel Version
Linux 4.13.13-4-pve #1 SMP PVE 4.13.13-35 (Mon, 8 Jan 2018 10:26:58 +0100)

PVE Manager Version
pve-manager/5.1-42/724a6cb3

Shared Storage : ISCSI SAN connected trough the private 10Gbits network

Thank you for your precious help

 

[Alwin]

I've tried to set the disk to SCSI and writeback, the issue remains, I've uploaded a quick video to show what happens :

The video shows a normal behavior, program starts and needs some CPU time to load everything. If you have a performance monitoring, then we could compare and visualize the difference.

The issue is appearing on all my cluster nodes and it appeared all sudden which is really strange...

I assume, you had kernel update(s) in between? If so, then you know Meltdown and Spectre, those patches that try to mitigate them are a performance drop, especially visible inside a VM. The drop in performance can be reduced, by bringing your VM up-to-date and setting the PCID flag (your cpu should support it) on the processor of the VM.