kernel panic when creating VMs (centos 9 stream ISO)

Aug 15, 2018
55
4
13
47
Good morning friends

I have proxmox 7.1-10
updated.

I'm trying to create VM with centers
CentOS-Stream-9-latest-x86_64-dvd1.iso

but when VM starts to install the operating system

enter kernel panic.


jai if this was a problem, i installed this in my local virtual box and everything went well..

but in proxmox it doesn't work... can anyone help me?
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
5,207
676
118
hi,

I have proxmox 7.1-10
updated.

I'm trying to create VM with centers
CentOS-Stream-9-latest-x86_64-dvd1.iso

but when VM starts to install the operating system

enter kernel panic.
* are you getting the kernel panic inside the VM or on the PVE host?

depending on the answer to my above questions please post:
* pveversion -v
* qm config VMID (change VMID to yours)
* if you can post the trace here (you should be able to find that in dmesg), that would help too, you can use [code][/code] tags
 
Aug 15, 2018
55
4
13
47
inside the vm when the installation will start

1644502136786.png


pveversion -v
Linux xorn 5.13.19-4-pve #1 SMP PVE 5.13.19-9 (Mon, 07 Feb 2022 11:01:14 +0100) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Feb 9 11:07:31 -03 2022 on pts/0
root@xorn:~# pveversion -v
proxmox-ve: 7.1-1 (running kernel: 5.13.19-4-pve)
pve-manager: 7.1-10 (running version: 7.1-10/6ddebafe)
pve-kernel-helper: 7.1-10
pve-kernel-5.13: 7.1-7
pve-kernel-5.11: 7.0-10
pve-kernel-5.4: 6.4-4
pve-kernel-5.13.19-4-pve: 5.13.19-9
pve-kernel-5.13.19-3-pve: 5.13.19-7
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.13.19-1-pve: 5.13.19-3
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.11.22-5-pve: 5.11.22-10
pve-kernel-5.11.22-4-pve: 5.11.22-9
pve-kernel-5.11.22-3-pve: 5.11.22-7
pve-kernel-5.11.22-2-pve: 5.11.22-4
pve-kernel-5.4.124-1-pve: 5.4.124-1
ceph-fuse: 15.2.15-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.1
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-6
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.1-2
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.1-1
libpve-storage-perl: 7.1-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.3.0-1
proxmox-backup-client: 2.1.5-1
proxmox-backup-file-restore: 2.1.5-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-5
pve-cluster: 7.1-3
pve-container: 4.1-3
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-5
pve-ha-manager: 3.3-3
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.1-1
pve-xtermjs: 4.16.0-1
pve-zsync: 2.2.1
qemu-server: 7.1-4
smartmontools: 7.2-pve2
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.2-pve1

qm config VMID (106)

acpi: 1
boot: order=ide2;scsi0;scsi1;net0
cores: 2
ide2: local:iso/CentOS-Stream-9-latest-x86_64-dvd1.iso,media=cdrom,size=8214M
kvm: 1
memory: 4192
meta: creation-qemu=6.1.0,ctime=1644414135
name: VM106
net0: virtio=6E:7D:85:29:AB:76,bridge=vmbr1
numa: 0
ostype: l26
scsi0: local:106/vm-106-disk-0.qcow2,size=100G,ssd=1
scsi1: local:106/vm-106-disk-1.qcow2,size=500G,ssd=1
scsihw: virtio-scsi-pci
smbios1: uuid=51709630-b6c2-41a9-b01d-4b14527cc1e1
sockets: 1
vmgenid: d53fce54-5afc-4a31-af83-cd18ad454b34
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
5,207
676
118
thanks for sharing the outputs.

inside the vm when the installation will start
have you tried with another VM or another ISO? is this the only one with that issue?
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
5,207
676
118
I tested with debian 11.2 and it is normal...only with centos 9
okay interesting... could you check if that ISO is not corrupted? you could compare the checksums from where you downloaded it
 
Aug 15, 2018
55
4
13
47
I already did this 4 x

I downloaded it again

and same iso I used locally and it works.
only on proxmox does not work
 
Aug 15, 2018
55
4
13
47
sha256sum CentOS-Stream-9-latest-x86_64-dvd1.iso
2688febc19b7abca5423fed70882649b7f95f1a9750873aabae007b04d2ea693 CentOS-Stream-9-latest-x86_64-dvd1.iso
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
5,207
676
118
sha256sum CentOS-Stream-9-latest-x86_64-dvd1.iso
2688febc19b7abca5423fed70882649b7f95f1a9750873aabae007b04d2ea693 CentOS-Stream-9-latest-x86_64-dvd1.iso
thanks, i can verify that i have the same one here downloaded from [0], and i can reproduce the kernel panic from your screenshot when starting the installation.

in the meantime you could try installing from a different ISO and see if that works for you

[0]: https://www.centos.org/centos-stream/
 
Aug 15, 2018
55
4
13
47
does not work if configured like this centos 9 stream
1644598206419.png


but that's how it works
1644598330516.png

I believe that the problem is in the default( kvm64) of the proxmox
 
  • Like
Reactions: CanadaGuy

ivenae

New Member
Feb 11, 2022
2
2
3
38
You use 5.13.19-4
This Kernel has known bugs.
Switch back to 5.13.19-3 or – if not productive – forward to 5.15.
I would not use that kernel. Search forum for this kernel version.
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
5,207
676
118
You use 5.13.19-4
This Kernel has known bugs.
Switch back to 5.13.19-3 or – if not productive – forward to 5.15.
I would not use that kernel. Search forum for this kernel version.
that's true, though i was able to replicate the same errors on a 5.15 kernel setup. the panic happens inside the VM because the init process gets killed for some reason (though i haven't got around to debugging this yet). my guess would be that this specific ISO is causing the issue since it doesn't seem to happen with other ones.

centos 8 stream

installation works normal
could you try an older centos 9 ISO?
 

em3034

New Member
Nov 11, 2021
4
5
3
45
I'm encountering the same issue with the new CentOS Stream 9 ISO. I've tried it on 2 proxmox nodes, one based on Intel and the other one AMD.
The same works fine using a previous CentOS Stream 9 ISO from Begining January. It also works fine using the latest RHEL9 Beta.
It would be interesting to see if it can be installed on other Hypervisor than proxmox. Anybody tried?
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
5,207
676
118
I'm encountering the same issue with the new CentOS Stream 9 ISO. I've tried it on 2 proxmox nodes, one based on Intel and the other one AMD.
The same works fine using a previous CentOS Stream 9 ISO from Begining January. It also works fine using the latest RHEL9 Beta.
thanks for verifying that the older centos 9 stream ISO is working fine, then it's most likely something they've changed in the ISO that's causing the issue...
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
5,207
676
118
does not work if configured like this centos 9 stream
View attachment 34154


but that's how it works
View attachment 34155

I believe that the problem is in the default( kvm64) of the proxmox
i've just tried that now too, it seems to work fine with the host CPU type, thanks for checking!

It would be interesting to see if it can be installed on other Hypervisor than proxmox. Anybody tried?
no idea, but it works with the CPU type set to host.

if you have similar issues in different hypervisors then it might be a bug on centos' end (would make sense to report it to them if that's the case)
 
  • Like
Reactions: ZipTX

em3034

New Member
Nov 11, 2021
4
5
3
45
Actually the workaround (spotted by Robson earlier in this thread) to set the CPU type to Host instead of Default (kvm64) solves the problem.
But using the default doesn't while is used to work with previous Stream 9 isos.
 

sml42

New Member
Feb 14, 2022
1
1
1
50
The error I am getting with CPU Type KVM64

fatal glibc error: cpu does not support x86-64-v2
and kernel panic afterwards

I can confirm that with CPU type host everything works as expected.
 
  • Like
Reactions: CanadaGuy

Nicox

New Member
Feb 20, 2022
1
0
1
72
I get the exact same issue with Centos 9 Stream. Putting CPU type to host resolved the problem. Is it correct to keep this option for long term deployment ?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!