High CPU use (18+%) by idle KVMs

[shubes]

Here is a chart that an associate of mine created which illustrates the scenarios he has tested.
Nice bit of work.
https://docs.google.com/spreadsheet/ccc?key=0AtGmTsyum5BqdGlBVlpJVm0tR2lwbTdDcnJnSEpSNHc&usp=sharing

 

[spirit]

Here is a chart that an associate of mine created which illustrates the scenarios he has tested.
Nice bit of work.
https://docs.google.com/spreadsheet/ccc?key=0AtGmTsyum5BqdGlBVlpJVm0tR2lwbTdDcnJnSEpSNHc&usp=sharing

Hi, I can't reproduce it currently on my test cluster,
can you try 2 things (separatly) in vm config file :

1)
remove "cpu: ..." if it exist

and add

"args: -cpu kvm64,+x2apic"



2)
keep "cpu:" if exist

and add

"args: -machine pc,kernel_irqchip=off"




it could be also interesting to check on the host

"cat /proc/interrupts"

to see if interrupts are going to only 1cpu, or are shared on all cpus

 

[shubes]

Hi, I can't reproduce it currently on my test cluster,
can you try 2 things (separatly) in vm config file :

1)
remove "cpu: ..." if it exist

and add

"args: -cpu kvm64,+x2apic"



2)
keep "cpu:" if exist

and add

"args: -machine pc,kernel_irqchip=off"




it could be also interesting to check on the host

"cat /proc/interrupts"

to see if interrupts are going to only 1cpu, or are shared on all cpus

Results aren't encouraging.

#1 - with CentOS5 VM (2.6.18 kernel), no noticable change (still ~30% cpu use when idle). With IPCop VM (2.6.32 kernel), it wouldn't boot. Went into a loop loading kernel.

#2 - with CentOS5 VM, symptoms were much worse. cpu use was 50-60%. With IPCop VM, also got worse, 11% instead of 9%.

Interrupts are shared on all (2) cpus. 2 cores, not 2 sockets (if that makes any difference).

Thanks for the help.

 

[shubes]

This appears to be getting worse.

I've built a PVE3.0 host, and a Win7 KVM which I migrated from bare iron uses 40% of 2 cpus when it's essentially idle.

I've removed some unused Win features on the guest, to no avail. Guest is configured with Virtio drivers for disk and network. I've set cpu=host. This is a 32-bit version of Win7 running on an AMD E350 cpu (2 cores).

Any guesses as to why KVM is eating so much cpu running a Win7 guest?

 

[spirit]

This appears to be getting worse.

I've built a PVE3.0 host, and a Win7 KVM which I migrated from bare iron uses 40% of 2 cpus when it's essentially idle.

I've removed some unused Win features on the guest, to no avail. Guest is configured with Virtio drivers for disk and network. I've set cpu=host. This is a 32-bit version of Win7 running on an AMD E350 cpu (2 cores).

Any guesses as to why KVM is eating so much cpu running a Win7 guest?

Can you try with win7 64bit to compare ? Also do you see the same behaviour with ide disk and e1000 network card ? (to be sure it don't come from virtio driver)

By the way, isn't AMD E350 a notebook cpu ?

 

[snowman66]

By the way, isn't AMD E350 a notebook cpu ?

It's mobile/notebook version: http://www.cpu-world.com/CPUs/Bobcat/AMD-E Series E-350 - EME350GBB22GT.html
Anyway i'm running 4 KVM+1 container on celeron 847 (pretty much the same spec. as amd e350) and cpu usage is about 3-4%.

This appears to be getting worse.

Did you try with faster storage (raid10, ssd, etc...)?

 

[cesarpk]

Hello everyone

I do not know if this question is related to the same problem.

My scenery:
- I have a Windows Server 2008 r2 in PVE 2.3 using 4 cores
- The server is a DELL 2900
- My VMID config:
boot: c
bootdisk: virtio0
cores: 4
cpu: host
ide2: none,media=cdrom,size=6604K
memory: 36864
name: Win2008R2
net0: virtio=72:60:0D:E5:29:65,bridge=vmbr0
net1: virtio=DE:B36:AB:B5:6B,bridge=vmbr1
ostype: win7
sockets: 1
virtio0: local:107/vm-107-disk-1.raw,size=350G
virtio1: local:107/vm-107-disk-2.raw,size=32G
virtio2: local:107/vm-107-disk-3.raw,size=150G
virtio3: local:107/vm-107-disk-4.qcow2,size=50G

The behavior:
When PVE GUI shows (Tag: Summary, Section: Status) the CPU usage between 8% to 10%,
and the task manager of Windows VM shows between 0% to 1%

My question:
Why no difference in the report of CPU usage?

 

[bessome]

I've noticed a much higher CPU use by some KVMs since upgrading from PVE2.1 to PVE2.2 (and PVE2.3). I suspect that the problem might be related to qemu (1.1.1 on PVE2.1, 1.2.0 on PVE2.2, and 1.4.0 on PVE2.3).

The same problem with 2.6 kernels end freebsd guests on 3.0 version.

 

[cesarpk]

The same problem with 2.6 kernels end freebsd guests on 3.0 version.

Hi bessome

Please see this link:
http://forum.proxmox.com/threads/14193-Higher-CPU-Load-on-WinXP-KVM-since-update-to-Proxmox-3

 

[bessome]

Hi bessome

Please see this link:
http://forum.proxmox.com/threads/14193-Higher-CPU-Load-on-WinXP-KVM-since-update-to-Proxmox-3

There is about windows VM. I tried to disable tables and doesn't help. On centos 5 i updated kernel to the latest 2.6.32 and now it is much better - with old kenrel it was 12% of CPU, with new only 7%.

 

[spirit]

do you use sqlserver in your windows guest ?

 

[spirit]

The same problem with 2.6 kernels end freebsd guests on 3.0 version.

a virtio nic driver is known to cause lot of interrupts with 2.6.32 linux kernel (like debian squeeze) since qemu 1.2 (pve 2.2), I recommend to use 3.X kernel.
Don't known for freebsd

 

[bessome]

a virtio nic driver is known to cause lot of interrupts with 2.6.32 linux kernel (like debian squeeze) since qemu 1.2 (pve 2.2), I recommend to use 3.X kernel.
Don't known for freebsd

i use e1000 nic