BUG: soft lockup - CPU#0 stuck for 67s! [kvm:3094]

[DeepB]

Hello,

i have a problem.

The KVM machines are unresponding, and also the host is unresponding, when I am locally logged in it says "BUG: soft lockup - CPU#0 stuck for 67s! [kvm:3094]".

pveversion:

pve-manager/3.1-4/f6816604 (running kernel: 2.6.32-23-pve)

what could that be?
Is there any more data I can supply?

best regards
Daniel

 

[DeepB]

output of pveversion -v:

# pveversion -v
proxmox-ve-2.6.32: 3.1-109 (running kernel: 2.6.32-23-pve)
pve-manager: 3.1-4 (running version: 3.1-4/f6816604)
pve-kernel-2.6.32-20-pve: 2.6.32-100
pve-kernel-2.6.32-22-pve: 2.6.32-107
pve-kernel-2.6.32-23-pve: 2.6.32-109
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-2
pve-cluster: 3.0-7
qemu-server: 3.1-1
pve-firmware: 1.0-23
libpve-common-perl: 3.0-6
libpve-access-control: 3.0-6
libpve-storage-perl: 3.0-10
pve-libspice-server1: 0.12.4-1
vncterm: 1.1-4
vzctl: 4.0-1pve3
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.4-17
ksm-control-daemon: 1.1-1
glusterfs-client: 3.4.0-2

additional information:
one vm is used as a router, so has two NICs. This seem to happen always when there is a lot of network traffic.
I switched the network card from virtio to intel, but it changes nothing.

When I am logged in locally on the server on the commandline I see
Bug: soft lockup - CPU#0 stuck for 67s! [kvm:3094]
this line is printed many times.
unfortunately the host freezes, and the only possible solution is the reset button.

Unfortunately this happens every 5 minutes or so, so the system is completely unusable.

Aby more information I can provide?

Best regards
Daniel

EDIT:
the only thing I found in the logs so far is:

Sep 11 14:35:52 home kernel: kvm: 2924: cpu0 unhandled rdmsr: 0xc0010112
Sep 11 14:35:52 home kernel: kvm: 2924: cpu0 unhandled rdmsr: 0xc0010001

 

[DeepB]

Sorry for spamming, but new information
kernel panic, please find information in attached pic.

Can anyone give me a tip what to do?

Best regards
Daniel

 

[dietmar]

Can anyone give me a tip what to do?

What kernel versions do you run inside the VMs? Try to use newer 3.X kernels (I am not sure if that helps, but it is worth a try).

 

[DeepB]

by chance I am running exactly the same kernel inside the vm.

Unfortunately I am limited in that, and cannot use a newer kernel....

best regards
Daniel

 

[dietmar]

Unfortunately I am limited in that, and cannot use a newer kernel....

If you use virtio network, try to use e1000 instead - does that help?