virtual machines kernel panic with Proxmox 5.1

[baerm]

Hi there,

we have some problems with the new Proxmox 5.1:
We run a 4 node cluster with 3 AMD Opteron server and 1 Intel Xeon. The Xeon is already on Proxmox 5.1, the AMDs still run Proxmox 5.0.

There are two main issues:
1) Migrating VMs with Debian 9.2 (4.9.0-4-amd64) to and from the intel machine causes kernel panic in the VMs. VMs with Debian 8.9 (3.16.0-4-amd64) do not show this behaviour and have no problems at all.

2) virtual machine residing on the intel machine tend to die at odd times with cpu going up to 100% and kernel panic.

Soft reset of the virtual machine fixes the problem.

CPU type is set to kvm64 on all virtual machines.

Since I do not know any good method to get a full error trace (logs dont show anything) I can only give a partial screen of the error in proxmox console (see attached).

Any ideas what is causing this?
I am aware of the kernel downgrade option, but would prefer to stay on new kernel if possible.

Best regards,

Matthias

 

[tommisan]

Hi,

I also got a kernel panic (Proxmox 5.1) on a vm Debian 9.3 with PostgreSQL 10 (a OS disk and a data disk, both SCSI). I recently upgraded the vm kernel to Debian 4.9.65-3+deb9u2 (2018-01-04).

I am trying to understand what it's causing the problem.
From what I see it could be related to the emulated SCSI controller LSI 53c810 - sym53c8xx, but it seems different from what you got..

 

[tom]

Hi,

I also got a kernel panic (Proxmox 5.1) on a vm Debian 9.3 with PostgreSQL 10 (a OS disk and a data disk, both SCSI). I recently upgraded the vm kernel to Debian 4.9.65-3+deb9u2 (2018-01-04).

I am trying to understand what it's causing the problem.
From what I see it could be related to the emulated SCSI controller LSI 53c810 - sym53c8xx, but it seems different from what you got..

Please use virtio-scsi controller (instead of LSI)

 

[tommisan]

Hi Tom, thank you for your answer.

Since Proxmox 4.4 I have been using the default controller virtIO SCSI which seems to be the recommended one (this is the first kernel panic I got in years). 90% of my vms are Debian based (mostly jessie and stretch).

Why do you advise to use the older VirtIO Block? Is there a problem with the latest kernel and virtIO SCSI?

 

[tom]

Hi Tom, thank you for your answer.

Since Proxmox 4.4 I have been using the default controller virtIO SCSI which seems to be the recommended one (this is the first kernel panic I got in years). 90% of my vms are Debian based (mostly jessie and stretch).

Why do you advise to use the older VirtIO Block? Is there a problem with the latest kernel and virtIO SCSI?

I do not recommend VirtIO Block, I recommend VirtIO SCSI.

 

[tommisan]

The kernel panic I got was is a vm with VirtIO SCSI controller.
With lspci I see:
00:05.0 SCSI storage controller: LSI Logic / Symbios Logic 53c895a

In the Proxmox guide I read that with VirtIO SCSI Proxmox emulates by default a LSI 53C895A controller.
Could you please explain what you meaned by "use virtio-scsi controller (instead of LSI)"?

Thank you in advance

 

[tom]

Please your VM config:

> qm config VMID

lspci inside Debian should report:

> 00:05.0 SCSI storage controller: Red Hat, Inc Virtio SCSI

(And where did you read that VirtIO emulates a LSI 53C895A?)

 

[tommisan]

vm config:

root@pve5:~# qm config 105
boot: cdn
bootdisk: scsi0
cores: 4
ide2: none,media=cdrom
memory: 8192
name: dibbi1-xl
net0: virtio=...,bridge=vmbr0
net1: virtio=...,bridge=vmbr1
numa: 0
ostype: l26
scsi0: stovm:105/vm-105-disk-1.qcow2,size=52G
scsi1: stovm:105/vm-105-disk-2.qcow2,size=200G
smbios1: uuid=...
sockets: 1

lspci inside Debian 9.3:
00:05.0 SCSI storage controller: LSI Logic / Symbios Logic 53c895a

Proxmox guide:
"the SCSI controller, designed in 1985, is commonly found on server grade hardware, and can connect up to 14 storage devices. Proxmox VE emulates by default a LSI 53C895A controller."
https://pve.proxmox.com/wiki/Qemu/KVM_Virtual_Machines

 

[tom]

vm config:

root@pve5:~# qm config 105
boot: cdn
bootdisk: scsi0
cores: 4
ide2: none,media=cdrom
memory: 8192
name: dibbi1-xl
net0: virtio=...,bridge=vmbr0
net1: virtio=...,bridge=vmbr1
numa: 0
ostype: l26
scsi0: stovm:105/vm-105-disk-1.qcow2,size=52G
scsi1: stovm:105/vm-105-disk-2.qcow2,size=200G
smbios1: uuid=...
sockets: 1

lspci inside Debian 9.3:
00:05.0 SCSI storage controller: LSI Logic / Symbios Logic 53c895a

Proxmox guide:
"the SCSI controller, designed in 1985, is commonly found on server grade hardware, and can connect up to 14 storage devices. Proxmox VE emulates by default a LSI 53C895A controller."
https://pve.proxmox.com/wiki/Qemu/KVM_Virtual_Machines

thanks, and the next sentence tells you:

"A SCSI controller of type VirtIO SCSI is the recommended setting if you aim for performance and is automatically selected for newly created Linux VMs since Proxmox VE 4.3."

=> please stop your VM and go to Option: SCSI Controller and change to VirtIO SCSI and start your VM again.

 

[tommisan]

Perfect, I got:
00:05.0 SCSI storage controller: Red Hat, Inc Virtio SCSI

Thanks again for your time, best regards