VM with old kernel 2.6.26 hangs on PVE 8.1.4

Kurgan

Well-Known Member
Apr 27, 2018
33
5
48
54
I'm trying to debug this issue: I have an old Debian VM with kernel 2.6.26 (i386) than has been converted from physical to virtual on a PVE 7.x temporary installation. It worked flawlessly on that PVE until it was moved to a new machine with pve-manager/8.1.4/ec5affc9e41f1d79 (running kernel: 6.5.11-7-pve).

The VM uses 4 GB of RAM, standard vCPU configuration and an LSI 53C895A (again standard) emulated SCSI controller because the VM kernel does not support Virtio-SCSI (it supports the older Virtio controller).

On the new PVE 8.1, the VM hangs with lots of these errors:

BUG: soft lockup - CPU#1 stuck for 96s! [apache2:14807]
BUG: soft lockup - CPU#0 stuck for 96s! [swapper:0]
BUG: soft lockup - CPU#3 stuck for 96s! [rsyslogd:14853]

When it happens, it seems that it's the disk i/o that dies. No more i/o AT ALL. Cannot read, cannot write. Power cycle the VM and it works again fine.

Since it's something that happens with the virtual disk (at least in my opinion) I have tried switching from LSI 53C895A to Virtio (not virtio-scsi) but the issue persists.

The virtual disk is configured as default: no cache, iothread enabled, Async IO is default (io_uring).

I/O on the host seems to be fine, no errors on the host log, the other VMs (modern Debian) work just fine. The disks are 2 NVME disks in raid1 (mdadm) with LVM on top

The issues are not clearly related to anything (not to backups, for example). I/O on the whole host is quite low, iodelay is under 5% even at the (rare) peaks, and it's usually under 0,5%.

Any ideas?

Thanks a lot.
 
Hi,
please make sure you are running the latest updates. There was a fix for a similar-sounding issue addressed in pve-qemu-kvm >= 8.1.5-2
 
Hi,
please make sure you are running the latest updates. There was a fix for a similar-sounding issue addressed in pve-qemu-kvm >= 8.1.5-2

I have pve-qemu-kvm Version: 8.1.5-5 on my problematic server, so it should not be that specific issue.

I have an older 8.1.2-4 on another server that has no issues at all...
 
I have pve-qemu-kvm Version: 8.1.5-5 on my problematic server, so it should not be that specific issue.
Please share the VM configuration file qm config <ID>. You could try turning off iothread, to see if the issue is related to that.
I have an older 8.1.2-4 on another server that has no issues at all...
With the same VM?
 
Last edited:
No, with different VMs. The same VM was fine on a 7.x server which I cannot reach right now (it's switced off and not here).

This is the config of the problematic VM. I have just switched off iothread, I will see if the issue is still there as soon as it crashes again. It was enabled until right now.

Code:
root@pve01 ~ # qm config  100
balloon: 0
boot: order=virtio0;ide2
cores: 4
cpu: x86-64-v2-AES
ide2: none,media=cdrom
memory: 4096
meta: creation-qemu=8.1.2,ctime=1705511420
name: anu
net0: virtio=00:50:56:15:14:18,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsihw: lsi
smbios1: uuid=6986a6f7-dffd-475f-a87b-d2de6a8e292e
sockets: 1
tablet: 0
virtio0: local-lvm:vm-100-disk-0,size=235G
vmgenid: d33f2eba-ce10-42cd-9642-1fce7ce9b8cc
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!