Unexpected reboot

[kf4rtg]

I host some gaming servers and a few household apps on my proxmox, and this morning there was an unexpected reboot. I dove into the logs and I clearly see -REBOOT- but I'm hoping someone can help me determine why.

Also - can someone please confirm that these root sessions that are opening and closing are just a result of the cron job running.

May 09 03:10:01 pm1 CRON[1016201]: (root) CMD (test -e /run/systemd/system || SERVICE_MODE=1 /sbin/e2scrub_all -A -r)
May 09 03:10:01 pm1 CRON[1016199]: pam_unix(cron:session): session closed for user root
May 09 03:17:02 pm1 CRON[1019546]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
May 09 03:17:02 pm1 CRON[1019548]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
May 09 03:17:02 pm1 CRON[1019546]: pam_unix(cron:session): session closed for user root
May 09 03:27:08 pm1 kernel: audit: type=1400 audit(1778311628.717:263): apparmor="DENIED" operation="mount" class="mount" info="failed flags match" error=-13 profile="lxc-501_</var/lib/lxc>" name="/run/systemd/unit-root/proc/" pid=1024448 comm="(find)" fstype="proc" srcname="proc" flags="rw, nosuid, nodev, noexec"
May 09 03:27:08 pm1 kernel: audit: type=1400 audit(1778311628.745:264): apparmor="DENIED" operation="mount" class="mount" info="failed flags match" error=-13 profile="lxc-501_</var/lib/lxc>" name="/run/systemd/unit-root/proc/" pid=1024449 comm="(mandb)" fstype="proc" srcname="proc" flags="rw, nosuid, nodev, noexec"
May 09 03:52:04 pm1 smartd[1671]: Device: /dev/bus/0 [megaraid_disk_12] [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 71 to 70
May 09 04:14:16 pm1 systemd[1]: Starting man-db.service - Daily man-db regeneration...
May 09 04:14:16 pm1 systemd[1]: man-db.service: Deactivated successfully.
May 09 04:14:16 pm1 systemd[1]: Finished man-db.service - Daily man-db regeneration.
May 09 04:17:01 pm1 CRON[1048210]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
May 09 04:17:01 pm1 CRON[1048212]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
May 09 04:17:01 pm1 CRON[1048210]: pam_unix(cron:session): session closed for user root
May 09 04:52:05 pm1 smartd[1671]: Device: /dev/bus/0 [megaraid_disk_12] [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 70 to 69
May 09 05:17:01 pm1 CRON[1076995]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
May 09 05:17:01 pm1 CRON[1076997]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
May 09 05:17:01 pm1 CRON[1076995]: pam_unix(cron:session): session closed for user root
May 09 05:22:04 pm1 smartd[1671]: Device: /dev/bus/0 [megaraid_disk_12] [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 69 to 70
May 09 06:17:01 pm1 CRON[1105792]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
May 09 06:17:01 pm1 CRON[1105794]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
May 09 06:17:01 pm1 CRON[1105792]: pam_unix(cron:session): session closed for user root
May 09 06:25:01 pm1 CRON[1109689]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
May 09 06:25:01 pm1 CRON[1109691]: (root) CMD (test -x /usr/sbin/anacron || { cd / && run-parts --report /etc/cron.daily; })
May 09 06:25:01 pm1 CRON[1109689]: pam_unix(cron:session): session closed for user root
May 09 06:33:27 pm1 systemd[1]: Starting apt-daily-upgrade.service - Daily apt upgrade and clean activities...
May 09 06:33:27 pm1 systemd[1]: apt-daily-upgrade.service: Deactivated successfully.
May 09 06:33:27 pm1 systemd[1]: Finished apt-daily-upgrade.service - Daily apt upgrade and clean activities.
May 09 06:52:04 pm1 smartd[1671]: Device: /dev/bus/0 [megaraid_disk_12] [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 70 to 69
May 09 07:17:01 pm1 CRON[1134739]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
May 09 07:17:01 pm1 CRON[1134741]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
May 09 07:17:01 pm1 CRON[1134739]: pam_unix(cron:session): session closed for user root
May 09 07:52:05 pm1 smartd[1671]: Device: /dev/bus/0 [megaraid_disk_12] [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 69 to 70
May 09 08:17:01 pm1 CRON[1163476]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
May 09 08:17:01 pm1 CRON[1163478]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
May 09 08:17:01 pm1 CRON[1163476]: pam_unix(cron:session): session closed for user root
May 09 08:30:16 pm1 systemd[1]: Starting systemd-tmpfiles-clean.service - Cleanup of Temporary Directories...
May 09 08:30:16 pm1 systemd-tmpfiles[1169850]: /usr/lib/tmpfiles.d/legacy.conf:14: Duplicate line for path "/run/lock", ignoring.
May 09 08:30:16 pm1 systemd[1]: systemd-tmpfiles-clean.service: Deactivated successfully.
May 09 08:30:16 pm1 systemd[1]: Finished systemd-tmpfiles-clean.service - Cleanup of Temporary Directories.
May 09 08:52:04 pm1 smartd[1671]: Device: /dev/bus/0 [megaraid_disk_12] [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 70 to 69
May 09 09:17:01 pm1 CRON[1192243]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
May 09 09:17:01 pm1 CRON[1192245]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
May 09 09:17:01 pm1 CRON[1192243]: pam_unix(cron:session): session closed for user root
-- Reboot --
May 09 09:26:44 pm1 kernel: Linux version 6.14.11-4-pve (build@proxmox) (gcc (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44) #1 SMP PREEMPT_DYNAMIC PMX 6.14.11-4 (2025-10-10T08:04Z) ()
May 09 09:26:44 pm1 kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-6.14.11-4-pve root=/dev/mapper/pve-root ro nomodeset quiet
May 09 09:26:44 pm1 kernel: KERNEL supported cpus:
May 09 09:26:44 pm1 kernel:   Intel GenuineIntel
May 09 09:26:44 pm1 kernel:   AMD AuthenticAMD
May 09 09:26:44 pm1 kernel:   Hygon HygonGenuine
May 09 09:26:44 pm1 kernel:   Centaur CentaurHauls
May 09 09:26:44 pm1 kernel:   zhaoxin   Shanghai 
May 09 09:26:44 pm1 kernel: BIOS-provided physical RAM map:
May 09 09:26:44 pm1 kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
May 09 09:26:44 pm1 kernel: BIOS-e820: [mem 0x0000000000100000-0x000000007df60fff] usable
May 09 09:26:44 pm1 kernel: BIOS-e820: [mem 0x000000007df61000-0x000000007df9ffff] reserved
May 09 09:26:44 pm1 kernel: BIOS-e820: [mem 0x000000007dfa0000-0x000000007dfaffff] ACPI data
May 09 09:26:44 pm1 kernel: BIOS-e820: [mem 0x000000007dfb0000-0x000000007e2eafff] ACPI NVS
May 09 09:26:44 pm1 kernel: BIOS-e820: [mem 0x000000007e2eb000-0x000000007f352fff] reserved
May 09 09:26:44 pm1 kernel: BIOS-e820: [mem 0x000000007f353000-0x000000007f353fff] usable
May 09 09:26:44 pm1 kernel: BIOS-e820: [mem 0x000000007f354000-0x000000007f3d9fff] ACPI NVS
May 09 09:26:44 pm1 kernel: BIOS-e820: [mem 0x000000007f3da000-0x000000007f7fffff] usable
May 09 09:26:44 pm1 kernel: BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
May 09 09:26:44 pm1 kernel: BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
May 09 09:26:44 pm1 kernel: BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
May 09 09:26:44 pm1 kernel: BIOS-e820: [mem 0x0000000100000000-0x000000307fffffff] usable
May 09 09:26:44 pm1 kernel: NX (Execute Disable) protection: active
May 09 09:26:44 pm1 kernel: APIC: Static calls initialized
May 09 09:26:44 pm1 kernel: e820: update [mem 0x65c70018-0x65c80857] usable ==> usable
May 09 09:26:44 pm1 kernel: e820: update [mem 0x65c60018-0x65c6f857] usable ==> usable
May 09 09:26:44 pm1 kernel: e820: update [mem 0x65c50018-0x65c5f857] usable ==> usable
May 09 09:26:44 pm1 kernel: e820: update [mem 0x65c47018-0x65c4f057] usable ==> usable
May 09 09:26:44 pm1 kernel: extended physical RAM map:
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x0000000000000000-0x000000000009ffff] usable
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x0000000000100000-0x0000000065c47017] usable
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x0000000065c47018-0x0000000065c4f057] usable
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x0000000065c4f058-0x0000000065c50017] usable
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x0000000065c50018-0x0000000065c5f857] usable
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x0000000065c5f858-0x0000000065c60017] usable
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x0000000065c60018-0x0000000065c6f857] usable
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x0000000065c6f858-0x0000000065c70017] usable
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x0000000065c70018-0x0000000065c80857] usable
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x0000000065c80858-0x000000007df60fff] usable
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x000000007df61000-0x000000007df9ffff] reserved
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x000000007dfa0000-0x000000007dfaffff] ACPI data
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x000000007dfb0000-0x000000007e2eafff] ACPI NVS
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x000000007e2eb000-0x000000007f352fff] reserved
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x000000007f353000-0x000000007f353fff] usable
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x000000007f354000-0x000000007f3d9fff] ACPI NVS
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x000000007f3da000-0x000000007f7fffff] usable
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x0000000080000000-0x000000008fffffff] reserved
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
May 09 09:26:44 pm1 kernel: reserve setup_data: [mem 0x0000000100000000-0x000000307fffffff] usable
May 09 09:26:44 pm1 kernel: efi: EFI v2.3.1 by American Megatrends
May 09 09:26:44 pm1 kernel: efi: ESRT=0x7f2fad18 ACPI=0x7e203000 ACPI 2.0=0x7e203000 SMBIOS=0xf04c0 MOKvar=0x7f259000 INITRD=0x6f32d818
May 09 09:26:44 pm1 kernel: efi: Remove mem67: MMIO range=[0x80000000-0x8fffffff] (256MB) from e820 map
May 09 09:26:44 pm1 kernel: e820: remove [mem 0x80000000-0x8fffffff] reserved
May 09 09:26:44 pm1 kernel: efi: Not removing mem68: MMIO range=[0xfed1c000-0xfed1ffff] (16KB) from e820 map
May 09 09:26:44 pm1 kernel: efi: Not removing mem69: MMIO range=[0xfed20000-0xfed3ffff] (128KB) from e820 map
May 09 09:26:44 pm1 kernel: efi: Remove mem70: MMIO range=[0xff000000-0xffffffff] (16MB) from e820 map
May 09 09:26:44 pm1 kernel: e820: remove [mem 0xff000000-0xffffffff] reserved
May 09 09:26:44 pm1 kernel: SMBIOS 2.7 present.
May 09 09:26:44 pm1 kernel: DMI: Supermicro X9DRH-7TF/7F/iTF/iF/X9DRH-7TF/7F/iTF/iF, BIOS 3.0b 04/28/2014
May 09 09:26:44 pm1 kernel: DMI: Memory slots populated: 16/16
May 09 09:26:44 pm1 kernel: secureboot: Secure boot disabled
May 09 09:26:44 pm1 kernel: tsc: Fast TSC calibration using PIT
May 09 09:26:44 pm1 kernel: tsc: Detected 2399.972 MHz processor
May 09 09:26:44 pm1 kernel: e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
May 09 09:26:44 pm1 kernel: e820: remove [mem 0x000a0000-0x000fffff] usable
May 09 09:26:44 pm1 kernel: last_pfn = 0x3080000 max_arch_pfn = 0x400000000
May 09 09:26:44 pm1 kernel: total RAM covered: 196608M

 

[YaZoal]

Is this node in a cluster?
Please share journalctl log include ~4 before the reboot observed, for example:
root@pve ~# journalctl --since "2026-05-09 00:00" --until "2026-05-09 12:00" | gzip > /tmp/$(hostname)-syslog.txt.gz

For a cluster, it is worth checking the journal logs from all nodes during the time the issue was observed.

 

[68k]

Joining in, since I've seen a few unexpected reboots lately as well. Single node.

I'm able to reproduce these; on my machine the reboots happen during times of high I/O activity (rsync remote backup):

May 13 10:56:57 Qazaam kernel: CE: hpet increased min_delta_ns to 773268 nsec

May 13 11:02:36 Qazaam sshd-session[8049]: Accepted password for root from 192.168.16.153 port 60437 ssh2
May 13 11:02:36 Qazaam sshd-session[8049]: pam_unix(sshd:session): session opened for user root(uid=0) by root(uid=0)
May 13 11:02:36 Qazaam systemd-logind[1026]: New session 7 of user root.
May 13 11:05:36 Qazaam kernel: CE: hpet increased min_delta_ns to 1000000 nsec
May 13 11:06:38 Qazaam kernel: CE: Reprogramming failure. Giving up
...
May 13 11:12:36 Qazaam kernel: CE: Reprogramming failure. Giving up
...
May 13 11:15:07 Qazaam kernel: CE: Reprogramming failure. Giving up
May 13 11:15:07 Qazaam kernel: watchdog: CPU1: Watchdog detected hard LOCKUP on cpu 1
May 13 11:15:07 Qazaam kernel: Modules linked in: veth ebtable_filter ebtables ip_set ip6table_raw iptable_raw ip6table_filter ip6_tables iptable_filter scsi_transport_iscsi nf_tables softdog bonding tls binfmt_misc nfnetlink_log amdgpu amdxcp drm_panel_backlight_quirks gpu_sched drm_buddy radeon nfsd amd64_edac edac_mce_amd kvm_amd drm_ttm_helper auth_rpcgss nfs_acl ttm rtl2832_sdr videobuf2_vmalloc lockd videobuf2_memops ccp grace videobuf2_v4l2 drm_exec videobuf2_common drm_suballoc_helper sch_fq_codel sunrpc videodev r820t rtl2832 i2c_mux kvm drm_display_helper dvb_usb_rtl28xxu dvb_usb_v2 cec dvb_core i2c_algo_bit video input_leds vga16fb rc_core irqbypass pcspkr k10temp vgastate mc wmi apple_mfi_fastcharge mac_hid zfs(PO) spl(O) vhost_net vhost efi_pstore vhost_iotlb tap nfnetlink dmi_sysfs ip_tables x_tables autofs4 raid1 raid0 linear hid_generic usbkbd raid456 usbhid async_raid6_recov async_memcpy hid async_pq async_xor async_tx xor raid6_pq raid10 pata_acpi tg3 ohci_pci i2c_piix4 pata_atiixp ehci_pci i2c_smbus ehci_hcd ohci_hcd
May 13 11:15:07 Qazaam kernel: ahci libahci
May 13 11:15:07 Qazaam kernel: CPU: 1 UID: 0 PID: 1994 Comm: CPU 0/KVM Tainted: P O 7.0.2-2-pve #1 PREEMPT(lazy)
May 13 11:15:07 Qazaam kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE
May 13 11:15:07 Qazaam kernel: Hardware name: HP ProLiant MicroServer, BIOS O41 07/29/2011
May 13 11:15:07 Qazaam kernel: RIP: 0010:svm_vcpu_run+0x2ac/0x850 [kvm_amd]
May 13 11:15:07 Qazaam kernel: Code: 83 78 70 61 0f 84 54 05 00 00 f6 80 b8 00 00 00 01 75 14 48 8b b3 18 02 00 00 48 3b b0 70 06 00 00 0f 85 49 04 00 00 0f 01 dc <48> 8b 83 90 1a 00 00 48 83 78 70 61 0f 84 16 05 00 00 f6 83 08 02
May 13 11:15:07 Qazaam kernel: RSP: 0018:ffffcf7d046af8f8 EFLAGS: 00000046
May 13 11:15:07 Qazaam kernel: RAX: ffff8c928fb41000 RBX: ffff8c924563d868 RCX: 0000000000000000
May 13 11:15:07 Qazaam kernel: RDX: ffffffffae8021b1 RSI: 0000000000000000 RDI: 0000000000000000
May 13 11:15:07 Qazaam kernel: RBP: ffffcf7d046af920 R08: 0000000000000000 R09: 0000000000000000
May 13 11:15:07 Qazaam kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
May 13 11:15:07 Qazaam kernel: R13: ffff8c9245650000 R14: ffff8c928fb41000 R15: 0000000000000000
May 13 11:15:07 Qazaam kernel: FS: 000071681b5fc6c0(0000) GS:ffff8c9542a0f000(0000) knlGS:0000000000000000
May 13 11:15:07 Qazaam kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
May 13 11:15:07 Qazaam kernel: CR2: 00007fc298588008 CR3: 00000001b9eec000 CR4: 00000000000006f0
May 13 11:15:07 Qazaam kernel: Call Trace:
May 13 11:15:07 Qazaam kernel: <TASK>
May 13 11:15:07 Qazaam kernel: kvm_arch_vcpu_ioctl_run+0x691/0x18c0 [kvm]
May 13 11:15:07 Qazaam kernel: ? read_emulated+0x5e/0xc0 [kvm]
May 13 11:15:07 Qazaam kernel: ? x86_emulate_insn+0x1ef/0x1150 [kvm]
May 13 11:15:07 Qazaam kernel: kvm_vcpu_ioctl+0x312/0xba0 [kvm]
May 13 11:15:07 Qazaam kernel: ? x86_emulate_instruction+0x382/0x8d0 [kvm]
May 13 11:15:07 Qazaam kernel: ? kvm_mmu_page_fault+0x310/0x670 [kvm]
May 13 11:15:07 Qazaam kernel: __x64_sys_ioctl+0xa5/0x100
May 13 11:15:07 Qazaam kernel: x64_sys_call+0x103b/0x2390
May 13 11:15:07 Qazaam kernel: do_syscall_64+0x11c/0x14e0
May 13 11:15:07 Qazaam kernel: ? npf_interception+0x141/0x2f0 [kvm_amd]
May 13 11:15:07 Qazaam kernel: ? apic_update_ppr+0x2a/0x100 [kvm]
May 13 11:15:07 Qazaam kernel: ? kvm_arch_vcpu_put+0x1ab/0x210 [kvm]
May 13 11:15:07 Qazaam kernel: ? vcpu_put+0x22/0x60 [kvm]
May 13 11:15:07 Qazaam kernel: ? kvm_arch_vcpu_ioctl_run+0xa8b/0x18c0 [kvm]
May 13 11:15:07 Qazaam kernel: ? kvm_on_user_return+0x4a/0x90 [kvm]
May 13 11:15:07 Qazaam kernel: ? fire_user_return_notifiers+0x37/0x70
May 13 11:15:07 Qazaam kernel: ? kvm_vcpu_ioctl+0x2a7/0xba0 [kvm]
May 13 11:15:07 Qazaam kernel: ? fire_user_return_notifiers+0x37/0x70
May 13 11:15:07 Qazaam kernel: ? kvm_vcpu_ioctl+0x2a7/0xba0 [kvm]
May 13 11:15:07 Qazaam kernel: ? __x64_sys_ioctl+0xbf/0x100
May 13 11:15:07 Qazaam kernel: ? __x64_sys_ioctl+0xbf/0x100
May 13 11:15:07 Qazaam kernel: ? kvm_on_user_return+0x4a/0x90 [kvm]
May 13 11:15:07 Qazaam kernel: ? fire_user_return_notifiers+0x37/0x70
May 13 11:15:07 Qazaam kernel: ? do_syscall_64+0x567/0x14e0
May 13 11:15:07 Qazaam kernel: ? __x64_sys_ioctl+0xbf/0x100
May 13 11:15:07 Qazaam kernel: ? kvm_on_user_return+0x4a/0x90 [kvm]
May 13 11:15:07 Qazaam kernel: ? fire_user_return_notifiers+0x37/0x70
May 13 11:15:07 Qazaam kernel: ? do_syscall_64+0x567/0x14e0
May 13 11:15:07 Qazaam kernel: ? x64_sys_call+0x103b/0x2390
May 13 11:15:07 Qazaam kernel: ? do_syscall_64+0x15a/0x14e0
May 13 11:15:07 Qazaam kernel: ? x64_sys_call+0x198e/0x2390
May 13 11:15:07 Qazaam kernel: ? do_syscall_64+0x15a/0x14e0
May 13 11:15:07 Qazaam kernel: ? do_syscall_64+0x2df/0x14e0
May 13 11:15:07 Qazaam kernel: ? do_syscall_64+0x567/0x14e0
May 13 11:15:07 Qazaam kernel: ? __x64_sys_ioctl+0xbf/0x100
May 13 11:15:07 Qazaam kernel: ? kvm_on_user_return+0x4a/0x90 [kvm]
May 13 11:15:07 Qazaam kernel: ? fire_user_return_notifiers+0x37/0x70
May 13 11:15:07 Qazaam kernel: ? do_syscall_64+0x567/0x14e0
May 13 11:15:07 Qazaam kernel: entry_SYSCALL_64_after_hwframe+0x76/0x7e
May 13 11:15:07 Qazaam kernel: RIP: 0033:0x71682ce3091b
May 13 11:15:07 Qazaam kernel: Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1c 48 8b 44 24 18 64 48 2b 04 25 28 00 00
May 13 11:15:07 Qazaam kernel: RSP: 002b:000071681b5f7b30 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
May 13 11:15:07 Qazaam kernel: RAX: ffffffffffffffda RBX: 000000000000ae80 RCX: 000071682ce3091b
May 13 11:15:07 Qazaam kernel: RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 000000000000001b
May 13 11:15:07 Qazaam kernel: RBP: 000059bf730c5690 R08: 0000000000000000 R09: 0000000000000000
May 13 11:15:07 Qazaam kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
May 13 11:15:07 Qazaam kernel: R13: 0000000000000000 R14: 000000000000f0aa R15: 0000000000000000
May 13 11:15:07 Qazaam kernel: </TASK>
May 13 11:15:07 Qazaam kernel: INFO: NMI handler (perf_event_nmi_handler) took too long to run: 2.070 msecs
May 13 11:15:07 Qazaam kernel: perf: interrupt took too long (18210 > 2500), lowering kernel.perf_event_max_sample_rate to 10000
May 13 11:15:52 Qazaam kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
May 13 11:15:52 Qazaam kernel: rcu: 1-...!: (0 ticks this GP) idle=bba0/0/0x0 softirq=96274/96274 fqs=1 (false positive?)
May 13 11:15:52 Qazaam kernel: rcu: (detected by 0, t=60002 jiffies, g=197193, q=781 ncpus=2)
May 13 11:15:52 Qazaam kernel: Sending NMI from CPU 0 to CPUs 1:
May 13 11:15:52 Qazaam kernel: NMI backtrace for cpu 1 skipped: idling at cpu_idle_poll.isra.0+0x25/0x130
May 13 11:15:52 Qazaam kernel: rcu: rcu_preempt kthread timer wakeup didn't happen for 59995 jiffies! g197193 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
May 13 11:15:52 Qazaam kernel: rcu: Possible timer handling issue on cpu=1 timer-softirq=197278
May 13 11:15:52 Qazaam kernel: rcu: rcu_preempt kthread starved for 59998 jiffies! g197193 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
May 13 11:15:52 Qazaam kernel: rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
May 13 11:15:52 Qazaam kernel: rcu: RCU grace-period kthread stack dump:
May 13 11:15:52 Qazaam kernel: task:rcu_preempt state:I stack:0 pid:15 tgid:15 ppid:2 task_flags:0x208040 flags:0x00080000
May 13 11:15:52 Qazaam kernel: Call Trace:
May 13 11:15:52 Qazaam kernel: <TASK>
May 13 11:15:52 Qazaam kernel: __schedule+0x495/0x1760
May 13 11:15:52 Qazaam kernel: ? __mod_timer+0x11c/0x340
May 13 11:15:52 Qazaam kernel: schedule+0x27/0xf0
May 13 11:15:52 Qazaam kernel: schedule_timeout+0x89/0x110
May 13 11:15:52 Qazaam kernel: ? __pfx_process_timeout+0x10/0x10
May 13 11:15:52 Qazaam kernel: rcu_gp_fqs_loop+0x106/0x490
May 13 11:15:52 Qazaam kernel: ? __pfx_rcu_gp_kthread+0x10/0x10
May 13 11:15:52 Qazaam kernel: rcu_gp_kthread+0xa3/0x140
May 13 11:15:52 Qazaam kernel: kthread+0xf7/0x130
May 13 11:15:52 Qazaam kernel: ? __pfx_kthread+0x10/0x10
May 13 11:15:52 Qazaam kernel: ret_from_fork+0x2dc/0x3a0
May 13 11:15:52 Qazaam kernel: ? __pfx_kthread+0x10/0x10
May 13 11:15:52 Qazaam kernel: ret_from_fork_asm+0x1a/0x30
May 13 11:15:52 Qazaam kernel: </TASK>
May 13 11:17:28 Qazaam kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
May 13 11:17:28 Qazaam kernel: rcu: 1-...!: (1 GPs behind) idle=cd58/0/0x0 softirq=96274/96274 fqs=1 (false positive?)
May 13 11:17:28 Qazaam kernel: rcu: (detected by 0, t=60002 jiffies, g=197485, q=1470 ncpus=2)
May 13 11:17:28 Qazaam kernel: Sending NMI from CPU 0 to CPUs 1:
May 13 11:17:28 Qazaam kernel: NMI backtrace for cpu 1
May 13 11:17:28 Qazaam kernel: CPU: 1 UID: 0 PID: 0 Comm: swapper/1 Tainted: P O 7.0.2-2-pve #1 PREEMPT(lazy)
May 13 11:17:28 Qazaam kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE
May 13 11:17:28 Qazaam kernel: Hardware name: HP ProLiant MicroServer, BIOS O41 07/29/2011
May 13 11:17:28 Qazaam kernel: RIP: 0010:tick_check_broadcast_expired+0x13/0x20
May 13 11:17:28 Qazaam kernel: Code: cc cc cc 0f 1f 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 65 48 63 15 bc 32 94 01 48 8b 05 f9 41 42 01 48 0f a3 10 <0f> 92 c0 0f b6 c0 31 d2 c3 cc cc cc cc 90 90 90 90 90 90 90 90 90
May 13 11:17:28 Qazaam kernel: RSP: 0018:ffffcf7d000bbea0 EFLAGS: 00000247
May 13 11:17:28 Qazaam kernel: RAX: ffff8c91c00cf2a8 RBX: ffff8c91c09ad280 RCX: 0000000000000000
May 13 11:17:28 Qazaam kernel: RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000000
May 13 11:17:28 Qazaam kernel: RBP: ffffcf7d000bbeb0 R08: 0000000000000000 R09: 0000000000000000
May 13 11:17:28 Qazaam kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff8c91c09ad280
May 13 11:17:28 Qazaam kernel: R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000000
May 13 11:17:28 Qazaam kernel: FS: 0000000000000000(0000) GS:ffff8c9542a0f000(0000) knlGS:0000000000000000
May 13 11:17:28 Qazaam kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
May 13 11:17:28 Qazaam kernel: CR2: 000055927efd9e48 CR3: 00000001b9eec000 CR4: 00000000000006f0
May 13 11:17:28 Qazaam kernel: Call Trace:
May 13 11:17:28 Qazaam kernel: <TASK>
May 13 11:17:28 Qazaam kernel: ? cpu_idle_poll.isra.0+0x3b/0x130
May 13 11:17:28 Qazaam kernel: do_idle+0x5e/0x2c0
May 13 11:17:28 Qazaam kernel: cpu_startup_entry+0x29/0x30
May 13 11:17:28 Qazaam kernel: start_secondary+0x118/0x150
May 13 11:17:28 Qazaam kernel: common_startup_64+0x13e/0x141
May 13 11:17:28 Qazaam kernel: </TASK>
May 13 11:17:28 Qazaam kernel: rcu: rcu_preempt kthread timer wakeup didn't happen for 59995 jiffies! g197485 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
May 13 11:17:28 Qazaam kernel: rcu: Possible timer handling issue on cpu=1 timer-softirq=197278
May 13 11:17:28 Qazaam kernel: rcu: rcu_preempt kthread starved for 59998 jiffies! g197485 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
May 13 11:17:28 Qazaam kernel: rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
May 13 11:17:28 Qazaam kernel: rcu: RCU grace-period kthread stack dump:
May 13 11:17:28 Qazaam kernel: task:rcu_preempt state:I stack:0 pid:15 tgid:15 ppid:2 task_flags:0x208040 flags:0x00080000
May 13 11:17:28 Qazaam kernel: Call Trace:
May 13 11:17:28 Qazaam kernel: <TASK>
May 13 11:17:28 Qazaam kernel: __schedule+0x495/0x1760
May 13 11:17:28 Qazaam kernel: ? __mod_timer+0x11c/0x340
May 13 11:17:28 Qazaam kernel: schedule+0x27/0xf0
May 13 11:17:28 Qazaam kernel: schedule_timeout+0x89/0x110
May 13 11:17:28 Qazaam kernel: ? __pfx_process_timeout+0x10/0x10
May 13 11:17:28 Qazaam kernel: rcu_gp_fqs_loop+0x106/0x490
May 13 11:17:28 Qazaam kernel: ? __pfx_rcu_gp_kthread+0x10/0x10
May 13 11:17:28 Qazaam kernel: rcu_gp_kthread+0xa3/0x140
May 13 11:17:28 Qazaam kernel: kthread+0xf7/0x130
May 13 11:17:28 Qazaam kernel: ? __pfx_kthread+0x10/0x10
May 13 11:17:28 Qazaam kernel: ret_from_fork+0x2dc/0x3a0
May 13 11:17:28 Qazaam kernel: ? __pfx_kthread+0x10/0x10
May 13 11:17:28 Qazaam kernel: ret_from_fork_asm+0x1a/0x30
May 13 11:17:28 Qazaam kernel: </TASK>

Machine is a HP N54L (Turion64) w/16GB ECC, hardware setup hasn't changed and used to be stable (~months of uptime) before. I'm running the latest ProxMox kernel (7.0.2-2-pve).

I'm always happy to provide more diag data to pinpoint the issue.

Regards, 68k

 

[YaZoal]

Hi @68k,
This might be a diffrent issue, you might consider starting a new thread for it.
But did you observe the same issue using different Kernel? for example kernel 6.14.11-4-pve
The trace for the kernel panic also show possible OOM:
May 13 11:17:28 Qazaam kernel: rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.

 

[Ben.Tomokins]

I have also seen 2 intermittend Reboots on the same ProxMox VE Host this week, one on Monday at 6:55am and then again today at 7:55am. I know it might be coincidence as this has not happened previously but I updated the ProxMox servers on Sunday evening the night before the first Reboot with the the following fixes:

Package Name​	Installed Version​	Available Version​
proxmox-kernel-6.17	6.17.13-6	6.17.13-7
proxmox-kernel-6.17.13-7-pve-signed	6.17.13-7

Here is a scrap of the Log from this morning unexpected Reboot:

I have run the following journalctl --since "2026-05-14 07:00" --until "2026-05-14 09:00" | gzip > /tmp/$(hostname)-syslog.txt.gz and attached the mcohpdl380g10-1-syslog.txt.gz here. This latest Reboot occurred at 08:00.

Is anyone able to help me out by looking at the attached syslog as we have just migrated recently from ESXi to Proxmox about 6 weeks ago so it's all very new to me?

 

[YaZoal]

...Is anyone able to help me out by looking at the attached syslog as we have just migrated recently from ESXi to Proxmox about 6 weeks ago so it's all very new to me?

I have taken a look at the provided log, the log does not show any kernel panic, OOM, or thermal event logged.
I would suggest to double check iDRAC/IPMI logs for evidence of hardware reset or power event.

 

[68k]

Hi @68k,
This might be a diffrent issue, you might consider starting a new thread for it.
But did you observe the same issue using different Kernel? for example kernel 6.14.11-4-pve
The trace for the kernel panic also show possible OOM:
May 13 11:17:28 Qazaam kernel: rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.

Thanks YaZoal! I've switched back to 6.17.13-9-pve for now and wasn't able to reproduce the issue so far - hardware is still unchanged.

The OOM line you've mentioned is probably unrelated (AFAICT), since it didn't appear in most of the crash logs that I've been able to capture. vmstat showed plenty of usable memory left too.

I'll try and compile some more log output in the near future so we can figure out what we're running into here.