[Help] Proxmox crashing every few days

[AsankaG]

Hi All,

Need some help trying to find out what might be causing the issues for proxmox to crash every few days. Basically the system becomes unresponsive and we have force restart the system.

Any help you guys can give me would be a great help.

The system Specs are below:
AMD Ryzen 5 3600
MSI B450
32GB RAM
1X250GB SSD Boot drive
4x 1TB HDD (Raid-Z 10)

Kernel Version : Linux 5.4.106-1-pve #1 SMP PVE 5.4.106-1 (Fri, 19 Mar 2021 11:08:47 +0100
PVE Manager Version : pve-manager/6.3-6/2184247e


Crash log:

Apr 26 16:05:04 s1 kernel: [269519.454069] hrtimer: interrupt took 8270 ns
Apr 26 21:21:14 s1 kernel: [288489.593802] general protection fault: 0000 [#2] SMP NOPTI
Apr 26 21:21:14 s1 kernel: [288489.594290] CPU: 7 PID: 3199 Comm: pve-firewall Tainted: P      D    O      5.4.106-1-pve #1
Apr 26 21:21:14 s1 kernel: [288489.594775] Hardware name: Micro-Star International Co., Ltd. MS-7B85/B450 GAMING PRO CARBON AC (MS-7B85), BIOS 1.C2 06/10/2020
Apr 26 21:21:14 s1 kernel: [288489.595774] RIP: 0010:anon_vma_interval_tree_insert+0x43/0xa0
Apr 26 21:21:14 s1 kernel: [288489.596277] Code: 00 00 00 48 8b 41 08 48 2b 01 48 89 e5 48 89 f1 48 c1 e8 0c 49 8d 7c 02 ff eb 07 48 8d 48 10 49 89 c1 48 8b 01 48 85 c0 74 20 <48> 3b 78 18 76 04 48 89 78 18 48 8b 48 e0 4c 3b 91 98 00 00 00 72
Apr 26 21:21:14 s1 kernel: [288489.597313] RSP: 0018:ffffb47c0ad83cf8 EFLAGS: 00010206
Apr 26 21:21:14 s1 kernel: [288489.597839] RAX: 0800000000000000 RBX: ffff9372c0f556c0 RCX: ffff9373fe05cca0
Apr 26 21:21:14 s1 kernel: [288489.598361] RDX: ffff9375ee67d2c0 RSI: ffff9373fe05cca0 RDI: 0000000000000004
Apr 26 21:21:14 s1 kernel: [288489.598881] RBP: ffffb47c0ad83cf8 R08: 0000000000000001 R09: 0000000000000000
Apr 26 21:21:14 s1 kernel: [288489.599396] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000000000
Apr 26 21:21:14 s1 kernel: [288489.599900] R13: ffff937916f7de10 R14: ffff9373fe05cc60 R15: ffff9375ee67d2c0
Apr 26 21:21:14 s1 kernel: [288489.600399] FS:  00007f43dc9911c0(0000) GS:ffff93793e9c0000(0000) knlGS:0000000000000000
Apr 26 21:21:14 s1 kernel: [288489.600915] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 26 21:21:14 s1 kernel: [288489.601423] CR2: 0000561a6044ed78 CR3: 00000007d6f32000 CR4: 0000000000340ee0
Apr 26 21:21:14 s1 kernel: [288489.601943] Call Trace:
Apr 26 21:21:14 s1 kernel: [288489.602442]  anon_vma_fork+0xf6/0x140
Apr 26 21:21:14 s1 kernel: [288489.602930]  dup_mm+0x4c8/0x5c0
Apr 26 21:21:14 s1 kernel: [288489.603417]  copy_process+0x18a9/0x1b60
Apr 26 21:21:14 s1 kernel: [288489.603901]  _do_fork+0x85/0x350
Apr 26 21:21:14 s1 kernel: [288489.604378]  ? recalc_sigpending+0x1b/0x60
Apr 26 21:21:14 s1 kernel: [288489.604846]  ? __set_task_blocked+0x72/0x90
Apr 26 21:21:14 s1 kernel: [288489.605305]  __x64_sys_clone+0x8f/0xb0
Apr 26 21:21:14 s1 kernel: [288489.605764]  do_syscall_64+0x57/0x190
Apr 26 21:21:14 s1 kernel: [288489.606218]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
Apr 26 21:21:14 s1 kernel: [288489.606675] RIP: 0033:0x7f43dca927be
Apr 26 21:21:14 s1 kernel: [288489.607136] Code: db 0f 85 25 01 00 00 64 4c 8b 0c 25 10 00 00 00 45 31 c0 4d 8d 91 d0 02 00 00 31 d2 31 f6 bf 11 00 20 01 b8 38 00 00 00 0f 05 <48> 3d 00 f0 ff ff 0f 87 b6 00 00 00 41 89 c4 85 c0 0f 85 c3 00 00
Apr 26 21:21:14 s1 kernel: [288489.608095] RSP: 002b:00007ffda83682a0 EFLAGS: 00000246 ORIG_RAX: 0000000000000038
Apr 26 21:21:14 s1 kernel: [288489.608577] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f43dca927be
Apr 26 21:21:14 s1 kernel: [288489.609067] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000001200011
Apr 26 21:21:14 s1 kernel: [288489.609547] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007f43dc9911c0
Apr 26 21:21:14 s1 kernel: [288489.610028] R10: 00007f43dc991490 R11: 0000000000000246 R12: 0000561a5ef94d38
Apr 26 21:21:14 s1 kernel: [288489.610507] R13: 00007ffda83682e0 R14: 0000561a5e4f2260 R15: 0000000000000000
Apr 26 21:21:14 s1 kernel: [288489.610991] Modules linked in: act_police cls_basic sch_ingress sch_htb nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 nfs lockd grace fscache veth ebtable_filter ebtables ip_set ip6table_raw iptable_raw ip6table_filter ip6_tables sctp iptable_filter bpfilter softdog nfnetlink_log nfnetlink snd_hda_codec_hdmi snd_hda_codec_realtek edac_mce_amd snd_hda_codec_generic iwlmvm ledtrig_audio kvm_amd snd_hda_intel mac80211 kvm snd_intel_dspcfg libarc4 snd_hda_codec irqbypass snd_hda_core btusb crct10dif_pclmul snd_hwdep crc32_pclmul ghash_clmulni_intel btrtl snd_pcm iwlwifi btbcm aesni_intel snd_timer btintel crypto_simd bluetooth snd cryptd glue_helper soundcore ecdh_generic cfg80211 ecc k10temp joydev ccp pcspkr wmi_bmof mac_hid zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) vhost_net vhost tap ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi sunrpc scsi_transport_iscsi ip_tables x_tables autofs4 btrfs xor zstd_compress raid6_pq
Apr 26 21:21:14 s1 kernel: [288489.611021]  dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio libcrc32c hid_generic usbhid hid i2c_piix4 igb xhci_pci i2c_algo_bit dca ahci xhci_hcd libahci wmi gpio_amdpt gpio_generic
Apr 26 21:21:14 s1 kernel: [288489.616682] ---[ end trace 8d0a0e996d0b81cf ]---
Apr 26 21:21:14 s1 kernel: [288489.617334] RIP: 0010:nfs_pgio_result+0x31/0x80 [nfs]
Apr 26 21:21:14 s1 kernel: [288489.617970] Code: 89 e5 41 55 41 54 49 89 fc 53 48 89 f3 4c 8b 2e f6 05 2a f5 40 ff 08 0f 85 03 ea 00 00 48 8b 43 60 4c 89 ea 48 89 de 4c 89 e7 <48> 8b 40 10 e8 d6 0b 08 cf 85 c0 75 1c 41 8b 74 24 04 85 f6 78 1a
Apr 26 21:21:14 s1 kernel: [288489.619269] RSP: 0018:ffffb47c15f4bdc0 EFLAGS: 00010246
Apr 26 21:21:14 s1 kernel: [288489.619911] RAX: f7ffffffc0f91d40 RBX: ffff937410f7a680 RCX: 0000000000000018
Apr 26 21:21:14 s1 kernel: [288489.620555] RDX: ffff93789b674e60 RSI: ffff937410f7a680 RDI: ffff937410f7a710
Apr 26 21:21:14 s1 kernel: [288489.621194] RBP: ffffb47c15f4bdd8 R08: ffff93721432aa40 R09: 0000000000000001
Apr 26 21:21:14 s1 kernel: [288489.621816] R10: 8080808080808080 R11: 0000000000000001 R12: ffff937410f7a710
Apr 26 21:21:14 s1 kernel: [288489.622435] R13: ffff93789b674e60 R14: ffffffffc0358c60 R15: ffffffffc0358c60
Apr 26 21:21:14 s1 kernel: [288489.623052] FS:  00007f43dc9911c0(0000) GS:ffff93793e9c0000(0000) knlGS:0000000000000000
Apr 26 21:21:14 s1 kernel: [288489.623666] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 26 21:21:14 s1 kernel: [288489.624280] CR2: 0000561a6044ed78 CR3: 00000007d6f32000 CR4: 0000000000340ee0
Apr 27 00:00:01 s1 rsyslogd:  [origin software="rsyslogd" swVersion="8.1901.0" x-pid="2757" x-info="https://www.rsyslog.com"] rsyslogd was HUPed
Apr 27 00:00:07 s1 kernel: [298022.469550] fwbr101i0: port 2(tap101i0) entered disabled state
Apr 27 00:00:07 s1 kernel: [298022.506167] fwbr101i0: port 1(fwln101i0) entered disabled state
Apr 27 00:00:07 s1 kernel: [298022.506848] vmbr0: port 2(fwpr101p0) entered disabled state
Apr 27 00:00:07 s1 kernel: [298022.507550] device fwln101i0 left promiscuous mode
Apr 27 00:00:07 s1 kernel: [298022.508164] fwbr101i0: port 1(fwln101i0) entered disabled state
Apr 27 00:00:07 s1 kernel: [298022.528518] device fwpr101p0 left promiscuous mode
Apr 27 00:00:07 s1 kernel: [298022.529262] vmbr0: port 2(fwpr101p0) entered disabled state
Apr 27 00:00:08 s1 kernel: [298023.197967] device tap101i0 entered promiscuous mode
Apr 27 00:00:08 s1 kernel: [298023.215695] fwbr101i0: port 1(fwln101i0) entered blocking state
Apr 27 00:00:08 s1 kernel: [298023.216324] fwbr101i0: port 1(fwln101i0) entered disabled state
Apr 27 00:00:08 s1 kernel: [298023.217030] device fwln101i0 entered promiscuous mode
Apr 27 00:00:08 s1 kernel: [298023.217967] fwbr101i0: port 1(fwln101i0) entered blocking state
Apr 27 00:00:08 s1 kernel: [298023.218678] fwbr101i0: port 1(fwln101i0) entered forwarding state
Apr 27 00:00:08 s1 kernel: [298023.221647] vmbr0: port 2(fwpr101p0) entered blocking state
Apr 27 00:00:08 s1 kernel: [298023.222284] vmbr0: port 2(fwpr101p0) entered disabled state
Apr 27 00:00:08 s1 kernel: [298023.222960] device fwpr101p0 entered promiscuous mode
Apr 27 00:00:08 s1 kernel: [298023.223610] vmbr0: port 2(fwpr101p0) entered blocking state
Apr 27 00:00:08 s1 kernel: [298023.224232] vmbr0: port 2(fwpr101p0) entered forwarding state
Apr 27 00:00:08 s1 kernel: [298023.227638] fwbr101i0: port 2(tap101i0) entered blocking state
Apr 27 00:00:08 s1 kernel: [298023.228246] fwbr101i0: port 2(tap101i0) entered disabled state
Apr 27 00:00:08 s1 kernel: [298023.228946] fwbr101i0: port 2(tap101i0) entered blocking state
Apr 27 00:00:08 s1 kernel: [298023.229540] fwbr101i0: port 2(tap101i0) entered forwarding state

 

[Dunuin]

You don't got ECC RAM?. You could boot into memtest86 to check if a bad RAM chip is causing errors.