soft lockup - CPU

[fusion]

Hello,on the console of a machine kvm centos7

every x minutes out:

kernel:BUG: soft lockup - CPU#6 stuck for 22s! [lvestats-server:2606]
Message from syslogd@server2 at Jan 12 00:46:44 ...
kernel:BUG: soft lockup - CPU#1 stuck for 23s! [mysqld:166735]

Message from syslogd@server2 at Jan 12 00:46:44 ...
kernel:BUG: soft lockup - CPU#5 stuck for 23s! [mysqld:166731]

Message from syslogd@server2 at Jan 12 00:46:44 ...
kernel:BUG: soft lockup - CPU#3 stuck for 23s! [mysqld:166734]

Message from syslogd@server2 at Jan 12 00:46:44 ...
kernel:BUG: soft lockup - CPU#6 stuck for 23s! [mysqld:166721]

Message from syslogd@server2 at Jan 12 00:46:44 ...
kernel:BUG: soft lockup - CPU#7 stuck for 23s! [mysqld:166730]

Message from syslogd@server2 at Jan 12 00:46:44 ...
kernel:BUG: soft lockup - CPU#4 stuck for 23s! [mysqld:166733]

Message from syslogd@server2 at Jan 12 00:46:44 ...
kernel:BUG: soft lockup - CPU#2 stuck for 22s! [lvestats-server:2606]

What can happen?

 

[windinternet]

Are you running emulators within the machine by any chance? What is the output of uname -a in the KVM?

Also check if you have sufficient disk space left in the disk image (qcow?).

 

[fusion]

Linux server2 3.10.0-329.7.2.lve1.3.58.el7.x86_64 #1 SMP Tue Dec 8 10:15:36 EST 2015 x86_64 x86_64 x86_64 GNU/Linux
_________________

pve-manager/4.1-2/78c5f4a2 (running kernel: 4.2.6-1-pve)
_______________________
balloon: 2048
bootdisk: virtio0
cores: 4
ide2: local:iso/CentOS-7-x86_64-Minimal-1511.iso,media=cdrom
keyboard: es
memory: 8096
name: server2.
net0: virtio=02:00:00:86:83:60,bridge=vmbr0
numa: 0
onboot: 1
ostype: l26
parent: cloudlinux
protection: 1
smbios1: uuid=6cdc925f-51b0-4daa-b204-9f8ef545caf6
sockets: 1
vcpus: 4
virtio0: rpool2:vm-103-disk-1,cache=writeback,size=100G

 

[windinternet]

Can you show the usage for the rpool2? Something like

zfs list -r rpool2

 

[fusion]

NAME USED AVAIL REFER MOUNTPOINT
rpool2 84.5G 171G 96K /rpool2
rpool2/subvol-100-disk-1 3.34G 47.7G 2.32G /rpool2/subvol-100-disk-1
rpool2/vm-103-disk-1 37.7G 171G 34.0G -
rpool2/vm-103-state-cloudlinux 6.15G 171G 6.15G -
rpool2/vm-104-disk-1 37.3G 171G 37.3G -

 

[windinternet]

Ok, I think we can rule out lockups caused by no disk space

There is a lot of different stories about these lockups on the internet. There is a problem with KVM on KVM sometimes, in other cases it has to do with physical hardware problems. Or it can be buggy network drivers causing trouble or frequency scaling (maybe again causing problems with some drivers).

Some links:
http://unix.stackexchange.com/questions/70377/bug-soft-lockup-cpu-stuck-for-x-seconds
http://the-hydra.blogspot.nl/2009/06/how-to-reduce-cpu-soft-lock-up-in-kvm.html
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1413540

 

[manu]

Hi
Most of the times, this is an operating system simply telling you're loacking I/O throughput.
See http://pve.proxmox.com/pipermail/pve-user/2016-January/009727.html for some hints
( different message but probably same cause)

BTW are your sure your kvm guest is running Centos 7 ? What is your kernel version ?