High Cpu Load on idle KVM host

Hi

Hope someone can assist me, We have just loaded Proxmox 4.2 details below. We are planning to upgrade from PVE 3.4 and create a 3 Server cluster, but we now seems to have some load issues, see below screenshot of a servers where the guest shows idle but GUI shows avg of 10% cpu usage.

upload_2016-7-14_13-48-57.png

# pveversion --verbose
proxmox-ve: 4.2-56 (running kernel: 4.4.13-1-pve)
pve-manager: 4.2-15 (running version: 4.2-15/6669ad2c)
pve-kernel-4.4.6-1-pve: 4.4.6-48
pve-kernel-4.4.13-1-pve: 4.4.13-56
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 1.0-1
pve-cluster: 4.0-42
qemu-server: 4.0-83
pve-firmware: 1.1-8
libpve-common-perl: 4.0-70
libpve-access-control: 4.0-16
libpve-storage-perl: 4.0-55
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.5-19
pve-container: 1.0-70
pve-firewall: 2.0-29
pve-ha-manager: 1.0-32
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u2
lxc-pve: 1.1.5-7
lxcfs: 2.0.0-pve2
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5.7-pve10~bpo80
 

spirit

Famous Member
Apr 2, 2010
4,296
286
103
www.odiso.com
Hi, this could be interrupts related.
Do you have a lots of network packets going to this vm ? (As I'm seeing cumulus as vm name)

Do you see same behaviour on proxmox 3.4 ?
 

spirit

Famous Member
Apr 2, 2010
4,296
286
103
www.odiso.com
They are also a new feature in kvm module :"halt_poll_ns"
https://lkml.org/lkml/2015/2/6/319

This improve latency in vm, but it's increase the cpu usage of idle vm by pooling continuously the vms.

you can try to add "halt_poll_ns=0" to kvm module options

/etc/modprobe.d/kvm.conf

options kvm halt_poll_ns_grow=0 halt_poll_ns_shrink=0 halt_poll_ns=0
 

spirit

Famous Member
Apr 2, 2010
4,296
286
103
www.odiso.com
Hi Spirit
Thanks you for the tip. I have done this and I think I have found my issue.

Seems it was related to standard linux bridge, I removed on lan port from the bridge and the CPU usage dropped.

View attachment 4037
So, you should check with tcpdump if you don't have broadcast or multicast flooding your brige (and vms on this bridge)
 

LnxBil

Famous Member
Feb 21, 2015
5,435
588
133
Germany
Yes, I thought about this too. Often you'll run into increased load if you do not split your cluster from your normal network. All the cluster multicast messages will be passed to each VM, therefore increased and unnecessary network load.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!