Random reboots / freezes

F

FlorinMarian

Well-Known Member
Nov 13, 2017
90
4
48
30
Hello Community,

I’m currently running a dedicated server at OVH with Proxmox installed, hosting several KVM virtual machines.

The server has been rented since May 16th, and starting May 23rd, I began experiencing serious issues — the server would completely freeze. The only way to bring it back online was through the IPMI console (which showed no relevant log entries), using the "Reset" button. This would restore functionality for a few hours, until it froze again.

I requested a hardware diagnosis from OVH, specifically asking for a RAM (DIMM) check. After four hours of testing, OVH reported no issues with the memory, but still proceeded to replace all four 32GB RAM modules as a precaution.

Now, 10 days later, I’ve encountered two spontaneous reboots and another system freeze similar to the one in May.

Looking into kernel.log and syslog, the only clue I’ve found is a line filled with repeated NULLNULLNULL....
Screenshot 2025-06-04 220047.png

What would you recommend I investigate next in order to identify the root cause?

Thank you in advance!

Software:
Code: 
proxmox-ve: 8.4.0 (running kernel: 6.8.12-11-pve)
pve-manager: 8.4.1 (running version: 8.4.1/2a5fa54a8503f96d)
proxmox-kernel-helper: 8.1.1
proxmox-kernel-6.8.12-11-pve-signed: 6.8.12-11
proxmox-kernel-6.8: 6.8.12-11
proxmox-kernel-6.8.12-10-pve-signed: 6.8.12-10
amd64-microcode: 3.20240820.1~deb12u1
ceph-fuse: 16.2.15+ds-0+deb12u1
corosync: 3.1.9-pve1
criu: 3.17.1-2+deb12u1
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx11
intel-microcode: 3.20250512.1~deb12u1
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libknet1: 1.30-pve2
libproxmox-acme-perl: 1.6.0
libproxmox-backup-qemu0: 1.5.1
libproxmox-rs-perl: 0.3.5
libpve-access-control: 8.2.2
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.1.0
libpve-cluster-perl: 8.1.0
libpve-common-perl: 8.3.1
libpve-guest-common-perl: 5.2.2
libpve-http-server-perl: 5.2.2
libpve-network-perl: 0.11.2
libpve-rs-perl: 0.9.4
libpve-storage-perl: 8.3.6
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.6.0-2
proxmox-backup-client: 3.4.1-1
proxmox-backup-file-restore: 3.4.1-1
proxmox-firewall: 0.7.1
proxmox-kernel-helper: 8.1.1
proxmox-mail-forward: 0.3.2
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.7
proxmox-widget-toolkit: 4.3.11
pve-cluster: 8.1.0
pve-container: 5.2.6
pve-docs: 8.4.0
pve-edk2-firmware: not correctly installed
pve-esxi-import-tools: 0.7.4
pve-firewall: 5.1.1
pve-firmware: 3.15-4
pve-ha-manager: 4.0.7
pve-i18n: 3.4.4
pve-qemu-kvm: 9.2.0-5
pve-xtermjs: 5.5.0-2
qemu-server: 8.3.12
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.7-pve2


Hardware Info: https://pastebin.com/MLLfuLQh
 
Last edited:
After a few more freezes, I saw this in the syslog (evne if this time the server didn't froze yet)
Code: 
2025-06-05T06:23:13.988450+00:00 sv1 kernel: [  648.778890] BUG: kernel NULL pointer dereference, address: 0000000000000020
2025-06-05T06:23:13.988466+00:00 sv1 kernel: [  648.786756] #PF: supervisor write access in kernel mode
2025-06-05T06:23:13.988466+00:00 sv1 kernel: [  648.792673] #PF: error_code(0x0002) - not-present page
2025-06-05T06:23:13.988467+00:00 sv1 kernel: [  648.798476] PGD 0 P4D 0
2025-06-05T06:23:13.988468+00:00 sv1 kernel: [  648.801360] Oops: 0002 [#1] PREEMPT SMP NOPTI
2025-06-05T06:23:13.988468+00:00 sv1 kernel: [  648.806291] CPU: 1 PID: 426305 Comm: tar Tainted: P           O       6.8.12-11-pve #1
2025-06-05T06:23:13.988468+00:00 sv1 kernel: [  648.815225] Hardware name: MSI MSIS366/S3661, BIOS ES366AOC.10NT01 03/27/2025
2025-06-05T06:23:13.988469+00:00 sv1 kernel: [  648.823278] RIP: 0010:mutex_unlock+0x10/0x40
2025-06-05T06:23:13.988469+00:00 sv1 kernel: [  648.828115] Code: 31 f6 eb d4 e8 51 49 ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 31 d2 65 48 8b 04 25 c0 43 03 00 <f0> 48 0f b1 17 75 0b 31 c0 31 d2 31 ff e9 49 2f 1a 00 55 48 89 e5
2025-06-05T06:23:13.988470+00:00 sv1 kernel: [  648.849254] RSP: 0018:ffffa309c1727f40 EFLAGS: 00010046
2025-06-05T06:23:13.988470+00:00 sv1 kernel: [  648.855161] RAX: ffff8a4b42b42f40 RBX: 0000000000000000 RCX: 0000000000000000
2025-06-05T06:23:13.988470+00:00 sv1 kernel: [  648.863212] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000020
2025-06-05T06:23:13.988471+00:00 sv1 kernel: [  648.871262] RBP: ffffa309c1727f48 R08: 0000000000000000 R09: 0000000000000000
2025-06-05T06:23:13.988471+00:00 sv1 kernel: [  648.879311] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
2025-06-05T06:23:13.988471+00:00 sv1 kernel: [  648.887365] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
2025-06-05T06:23:13.988471+00:00 sv1 kernel: [  648.895419] FS:  00007b85a21861c0(0000) GS:ffff8a689da80000(0000) knlGS:0000000000000000
2025-06-05T06:23:13.988472+00:00 sv1 kernel: [  648.904541] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2025-06-05T06:23:13.988472+00:00 sv1 kernel: [  648.911031] CR2: 0000000000000020 CR3: 000000170a7f4000 CR4: 0000000000f50ef0
2025-06-05T06:23:13.988472+00:00 sv1 kernel: [  648.919086] PKRU: 55555554
2025-06-05T06:23:13.988472+00:00 sv1 kernel: [  648.922163] Call Trace:
2025-06-05T06:23:13.988473+00:00 sv1 kernel: [  648.924940]  <TASK>
2025-06-05T06:23:13.988473+00:00 sv1 kernel: [  648.927332]  ? show_regs+0x6d/0x80
2025-06-05T06:23:13.988473+00:00 sv1 kernel: [  648.931189]  ? __die+0x24/0x80
2025-06-05T06:23:13.988473+00:00 sv1 kernel: [  648.934656]  ? page_fault_oops+0x176/0x500
2025-06-05T06:23:13.988474+00:00 sv1 kernel: [  648.939594]  ? do_user_addr_fault+0x2f5/0x660
2025-06-05T06:23:13.988474+00:00 sv1 kernel: [  648.944810]  ? syscall_exit_to_user_mode+0x86/0x260
2025-06-05T06:23:13.988474+00:00 sv1 kernel: [  648.950603]  ? exc_page_fault+0x83/0x1b0
2025-06-05T06:23:13.988475+00:00 sv1 kernel: [  648.955306]  ? asm_exc_page_fault+0x27/0x30
2025-06-05T06:23:13.988475+00:00 sv1 kernel: [  648.960299]  ? mutex_unlock+0x10/0x40
2025-06-05T06:23:13.988475+00:00 sv1 kernel: [  648.964707]  ? __f_unlock_pos+0x12/0x20
2025-06-05T06:23:13.988475+00:00 sv1 kernel: [  648.969306]  entry_SYSCALL_64_after_hwframe+0x5e/0x80
2025-06-05T06:23:13.988476+00:00 sv1 kernel: [  648.975272] RIP: 0033:0x7b85a2319300
2025-06-05T06:23:13.988476+00:00 sv1 kernel: [  648.979598] Code: 40 00 48 8b 15 01 9b 0d 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 80 3d e1 22 0e 00 00 74 17 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 48 83 ec 28 48 89
2025-06-05T06:23:13.988476+00:00 sv1 kernel: [  649.001521] RSP: 002b:00007ffeb464c8c8 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
2025-06-05T06:23:13.988476+00:00 sv1 kernel: [  649.010338] RAX: ffffffffffffffda RBX: 0000000000000200 RCX: 00007b85a2319300
2025-06-05T06:23:13.988477+00:00 sv1 kernel: [  649.018667] RDX: 0000000000000200 RSI: 0000620d22f3f000 RDI: 0000000000000001
2025-06-05T06:23:13.988477+00:00 sv1 kernel: [  649.026991] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
2025-06-05T06:23:13.988477+00:00 sv1 kernel: [  649.035314] R10: 00007b85a223a4f0 R11: 0000000000000202 R12: 0000620d22f3f000
2025-06-05T06:23:13.988478+00:00 sv1 kernel: [  649.043640] R13: 0000000000000001 R14: 0000000000000006 R15: 0000000000000000
2025-06-05T06:23:13.988478+00:00 sv1 kernel: [  649.051963]  </TASK>
2025-06-05T06:23:13.988478+00:00 sv1 kernel: [  649.054719] Modules linked in: tcp_diag inet_diag act_police cls_basic sch_ingress sch_htb rpcsec_gss_krb5 auth_rpcgss nfsv4 nfs lockd grace netfs veth ebt_arp ebtable_filter ebtables ip6t_REJECT nf_reject_ipv6 xt_mac ipt_REJECT nf_reject_ipv4 xt_mark xt_set xt_physdev xt_addrtype ip_set_hash_net ip_set softdog nf_tables bonding tls sunrpc binfmt_misc ip6table_filter ip6table_raw ip6_tables xt_limit xt_LOG nf_log_syslog xt_multiport nfnetlink_log iptable_filter nfnetlink xt_hashlimit xt_comment iptable_raw iptable_nat nf_nat xt_tcpmss xt_conntrack xt_tcpudp iptable_mangle tcp_bbr sch_fq intel_rapl_msr intel_rapl_common edac_mce_amd kvm_amd kvm irqbypass crct10dif_pclmul polyval_clmulni polyval_generic ghash_clmulni_intel irdma sha256_ssse3 sha1_ssse3 ipmi_ssif aesni_intel joydev input_leds crypto_simd i40e hid_generic cryptd acpi_ipmi usbkbd usbmouse ib_uverbs cdc_ether usbhid ipmi_si usbnet rapl hid wmi_bmof k10temp ccp mii ipmi_devintf ib_core i2c_algo_bit ipmi_msghandler amd_pmc mac_hid isofs zfs(PO) spl(O)
2025-06-05T06:23:13.988479+00:00 sv1 kernel: [  649.054852]  vhost_net vhost vhost_iotlb tap nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 efi_pstore dmi_sysfs ip_tables x_tables autofs4 uas usb_storage raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 xhci_pci xhci_pci_renesas nvme ice ahci nvme_core xhci_hcd libahci crc32_pclmul i2c_piix4 gnss nvme_auth video wmi
2025-06-05T06:23:13.988480+00:00 sv1 kernel: [  649.196557] CR2: 0000000000000020
2025-06-05T06:23:13.988480+00:00 sv1 kernel: [  649.200644] ---[ end trace 0000000000000000 ]---
2025-06-05T06:23:13.988480+00:00 sv1 kernel: [  649.347075] RIP: 0010:mutex_unlock+0x10/0x40
2025-06-05T06:23:13.988480+00:00 sv1 kernel: [  649.352342] Code: 31 f6 eb d4 e8 51 49 ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 31 d2 65 48 8b 04 25 c0 43 03 00 <f0> 48 0f b1 17 75 0b 31 c0 31 d2 31 ff e9 49 2f 1a 00 55 48 89 e5
2025-06-05T06:23:13.988481+00:00 sv1 kernel: [  649.374542] RSP: 0018:ffffa309c1727f40 EFLAGS: 00010046
2025-06-05T06:23:13.988481+00:00 sv1 kernel: [  649.380834] RAX: ffff8a4b42b42f40 RBX: 0000000000000000 RCX: 0000000000000000
2025-06-05T06:23:13.988481+00:00 sv1 kernel: [  649.389267] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000020
2025-06-05T06:23:13.988481+00:00 sv1 kernel: [  649.397703] RBP: ffffa309c1727f48 R08: 0000000000000000 R09: 0000000000000000
2025-06-05T06:23:13.988489+00:00 sv1 kernel: [  649.406135] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
2025-06-05T06:23:13.988489+00:00 sv1 kernel: [  649.414586] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
2025-06-05T06:23:13.988489+00:00 sv1 kernel: [  649.423018] FS:  00007b85a21861c0(0000) GS:ffff8a689da80000(0000) knlGS:0000000000000000
2025-06-05T06:23:13.988489+00:00 sv1 kernel: [  649.432536] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2025-06-05T06:23:13.988490+00:00 sv1 kernel: [  649.439400] CR2: 0000000000000020 CR3: 000000170a7f4000 CR4: 0000000000f50ef0
2025-06-05T06:23:13.988490+00:00 sv1 kernel: [  649.447835] PKRU: 55555554
2025-06-05T06:23:13.988490+00:00 sv1 kernel: [  649.451282] note: tar[426305] exited with irqs disabled
 
You must log in or register to reply here.
Top Bottom