Proxmox AND Host freezes..

[berkaybulut]

Hello,
A machine in my proxmox cluster with 3 hosts is freezing. When I accessed the console and idrac, I saw that the screen was frozen. I have to restart it.

This is completely random. Has anyone experienced this or have any ideas?

Spoiler: packade version

proxmox-ve: 8.2.0 (running kernel: 6.8.4-3-pve)
pve-manager: 8.2.2 (running version: 8.2.2/9355359cd7afbae4)
proxmox-kernel-helper: 8.1.0
proxmox-kernel-6.8: 6.8.4-3
proxmox-kernel-6.8.4-3-pve-signed: 6.8.4-3
proxmox-kernel-6.8.4-2-pve-signed: 6.8.4-2
proxmox-kernel-6.5.13-5-pve-signed: 6.5.13-5
proxmox-kernel-6.5: 6.5.13-5
proxmox-kernel-6.5.13-3-pve-signed: 6.5.13-3
proxmox-kernel-6.5.11-8-pve-signed: 6.5.11-8
ceph-fuse: 17.2.7-pve2
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx8
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.1
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.3
libpve-access-control: 8.1.4
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.0.6
libpve-cluster-perl: 8.0.6
libpve-common-perl: 8.2.1
libpve-guest-common-perl: 5.1.1
libpve-http-server-perl: 5.1.0
libpve-network-perl: 0.9.8
libpve-rs-perl: 0.8.8
libpve-storage-perl: 8.2.1
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.4.0-3
proxmox-backup-client: 3.2.2-1
proxmox-backup-file-restore: 3.2.2-1
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.2.3
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.6
proxmox-widget-toolkit: 4.2.3
pve-cluster: 8.0.6
pve-container: 5.1.10
pve-docs: 8.2.2
pve-edk2-firmware: 4.2023.08-4
pve-esxi-import-tools: 0.7.0
pve-firewall: 5.0.7
pve-firmware: 3.11-1
pve-ha-manager: 4.0.4
pve-i18n: 3.2.2
pve-qemu-kvm: 8.1.5-6
pve-xtermjs: 5.3.0-3
qemu-server: 8.2.1
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.3-pve2

Spoiler: System Log

May 13 06:50:47 cmt7055 pmxcfs[4551]: [status] notice: received log
May 13 06:51:33 cmt7055 nfsidmap[1945486]: nss_getpwnam: name 'Administrators@BUILTIN' does not map into domain 'dchost.com'
May 13 06:51:33 cmt7055 nfsidmap[1945487]: nss_name_to_gid: name 'None@WIN-2R2K935S25H' does not map into domain 'dchost.com'
May 13 06:55:48 cmt7055 pmxcfs[4551]: [status] notice: received log
May 13 07:00:19 cmt7055 pmxcfs[4551]: [status] notice: received log
May 13 07:02:23 cmt7055 nfsidmap[1949525]: nss_getpwnam: name 'Administrators@BUILTIN' does not map into domain 'dchost.com'
May 13 07:02:23 cmt7055 nfsidmap[1949526]: nss_name_to_gid: name 'None@WIN-2R2K935S25H' does not map into domain 'dchost.com'
May 13 07:05:48 cmt7055 pmxcfs[4551]: [status] notice: received log
May 13 07:10:48 cmt7055 pmxcfs[4551]: [status] notice: received log
May 13 07:13:13 cmt7055 nfsidmap[1953394]: nss_getpwnam: name 'Administrators@BUILTIN' does not map into domain 'dchost.com'
May 13 07:13:13 cmt7055 nfsidmap[1953395]: nss_name_to_gid: name 'None@WIN-2R2K935S25H' does not map into domain 'dchost.com'
May 13 07:15:18 cmt7055 pmxcfs[4551]: [status] notice: received log
May 13 07:17:01 cmt7055 CRON[1954731]: pam_unix(cron:session): session opened for user root(uid=0) by (uid=0)
May 13 07:17:01 cmt7055 CRON[1954732]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
May 13 07:17:01 cmt7055 CRON[1954731]: pam_unix(cron:session): session closed for user root
May 13 07:17:32 cmt7055 pvestatd[4720]: metrics send error 'grafana': 500 Can't connect to grafana.dchost.com:8086 (Temporary failure in name resolution)
May 13 07:17:34 cmt7055 pvestatd[4720]: status update time (31.964 seconds)
May 13 07:20:32 cmt7055 smartd[4124]: Device: /dev/sdc [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 81 to 82
May 13 07:20:32 cmt7055 smartd[4124]: Device: /dev/sdc [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 81 to 82
May 13 07:23:55 cmt7055 nfsidmap[1957148]: nss_getpwnam: name 'Administrators@BUILTIN' does not map into domain 'dchost.com'
May 13 07:23:55 cmt7055 nfsidmap[1957149]: nss_name_to_gid: name 'None@WIN-2R2K935S25H' does not map into domain 'dchost.com'
May 13 07:25:48 cmt7055 pmxcfs[4551]: [status] notice: received log
May 13 07:27:33 cmt7055 pmxcfs[4551]: [status] notice: received log
May 13 07:30:18 cmt7055 pmxcfs[4551]: [status] notice: received log
May 13 07:34:05 cmt7055 nfsidmap[1960899]: nss_getpwnam: name 'Administrators@BUILTIN' does not map into domain 'dchost.com'
May 13 07:34:05 cmt7055 nfsidmap[1960900]: nss_name_to_gid: name 'None@WIN-2R2K935S25H' does not map into domain 'dchost.com'
May 13 07:35:48 cmt7055 pmxcfs[4551]: [status] notice: received log
May 13 07:38:20 cmt7055 pveproxy[1766851]: detected empty handle
May 13 07:41:41 cmt7055 pmxcfs[4551]: [dcdb] notice: data verification successful
May 13 07:44:55 cmt7055 nfsidmap[1964755]: nss_getpwnam: name 'Administrators@BUILTIN' does not map into domain 'dchost.com'
May 13 07:44:55 cmt7055 nfsidmap[1964756]: nss_name_to_gid: name 'None@WIN-2R2K935S25H' does not map into domain 'dchost.com'
May 13 07:45:17 cmt7055 pmxcfs[4551]: [status] notice: received log
May 13 07:50:32 cmt7055 smartd[4124]: Device: /dev/sdc [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 82 to 81
May 13 07:50:32 cmt7055 smartd[4124]: Device: /dev/sdc [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 82 to 81
May 13 07:50:32 cmt7055 smartd[4124]: Device: /dev/sdg [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 81 to 82
May 13 07:50:32 cmt7055 smartd[4124]: Device: /dev/sdg [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 81 to 82
May 13 07:50:48 cmt7055 pmxcfs[4551]: [status] notice: received log
May 13 07:55:05 cmt7055 nfsidmap[1968401]: nss_getpwnam: name 'Administrators@BUILTIN' does not map into domain 'dchost.com'
May 13 07:55:05 cmt7055 nfsidmap[1968402]: nss_name_to_gid: name 'None@WIN-2R2K935S25H' does not map into domain 'dchost.com'
-- Reboot --
May 13 13:42:54 cmt7055 kernel: Linux version 6.8.4-3-pve (build@proxmox) (gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT_DYNAMIC PMX 6.8.4-3 (2024-05-02T11:55Z) ()
May 13 13:42:54 cmt7055 kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-6.8.4-3-pve root=/dev/mapper/pve-root ro quiet
May 13 13:42:54 cmt7055 kernel: KERNEL supported cpus:
May 13 13:42:54 cmt7055 kernel: Intel GenuineIntel
May 13 13:42:54 cmt7055 kernel: AMD AuthenticAMD
May 13 13:42:54 cmt7055 kernel: Hygon HygonGenuine
May 13 13:42:54 cmt7055 kernel: Centaur CentaurHauls
May 13 13:42:54 cmt7055 kernel: zhaoxin Shanghai
May 13 13:42:54 cmt7055 kernel: BIOS-provided physical RAM map:
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x0000000000100000-0x0000000071ffdfff] usable
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x0000000071ffe000-0x0000000072051fff] type 20
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x0000000072052000-0x0000000072058fff] usable
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x0000000072059000-0x00000000720a2fff] type 20
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x00000000720a3000-0x00000000720a9fff] usable
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x00000000720aa000-0x0000000072109fff] type 20
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x000000007210a000-0x000000007a088fff] usable
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x000000007a089000-0x000000007a188fff] reserved
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x000000007a189000-0x000000007a288fff] type 20
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x000000007a289000-0x000000007af0afff] reserved
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x000000007af0b000-0x000000007b93afff] ACPI NVS
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x000000007b93b000-0x000000007bab4fff] ACPI data
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x000000007bab5000-0x000000007bae8fff] usable
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x000000007bae9000-0x000000007bafefff] ACPI data
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x000000007baff000-0x000000007bafffff] usable
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x00000000feda8000-0x00000000fedabfff] reserved
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x00000000ff310000-0x00000000ffffffff] reserved
May 13 13:42:54 cmt7055 kernel: BIOS-e820: [mem 0x0000000100000000-0x000000607fffffff] usable
May 13 13:42:54 cmt7055 kernel: NX (Execute Disable) protection: active
May 13 13:42:54 cmt7055 kernel: APIC: Static calls initialized
May 13 13:42:54 cmt7055 kernel: efi: EFI v2.4 by Dell Inc.
May 13 13:42:54 cmt7055 kernel: efi: ACPI 2.0=0x7bab4014 SMBIOS=0x7af0a000 ACPI=0x7bab4000 MOKvar=0x7a155000
May 13 13:42:54 cmt7055 kernel: efi: Remove mem247: MMIO range=[0x80000000-0x8fffffff] (256MB) from e820 map
May 13 13:42:54 cmt7055 kernel: e820: remove [mem 0x80000000-0x8fffffff] reserved
May 13 13:42:54 cmt7055 kernel: efi: Not removing mem248: MMIO range=[0xfeda8000-0xfedabfff] (16KB) from e820 map
May 13 13:42:54 cmt7055 kernel: efi: Remove mem249: MMIO range=[0xff310000-0xffffffff] (12MB) from e820 map
May 13 13:42:54 cmt7055 kernel: e820: remove [mem 0xff310000-0xffffffff] reserved

 

[antonin.chadima]

https://forum.proxmox.com/threads/kernel-6-8-4-2-causes-random-server-freezing.146327
do you have Ceph and NVMe drives?

Another thread: https://forum.proxmox.com/threads/random-6-8-4-2-pve-kernel-crashes.145760