[SOLVED] Kernel panics related to ZFS causing system to become unstable

[viertaxa]

Hello,

I'm receiving the following kernel panic on my system, causing ZFS to become unstable. I'm fully up to date on the paid plan.

Sep 28 12:19:42 pve kernel: [ 1470.872048] general protection fault: 0000 [#1] SMP NOPTI
Sep 28 12:19:42 pve kernel: [ 1470.872062] CPU: 11 PID: 507 Comm: z_rd_int Tainted: P           O      5.4.60-1-pve #1
Sep 28 12:19:42 pve kernel: [ 1470.872067] Hardware name: Gigabyte Technology Co., Ltd. B450M DS3H/B450M DS3H-CF, BIOS F51 07/29/2020
Sep 28 12:19:42 pve kernel: [ 1470.872111] RIP: 0010:zio_remove_child+0x6f/0x140 [zfs]
Sep 28 12:19:42 pve kernel: [ 1470.872116] Code: e8 03 00 00 48 89 45 d0 e8 ce b5 23 fb 48 8b 45 d0 49 89 84 24 e8 03 00 00 48 8b 83 50 01 00 00 4c 01 e8 48 8b 08 48 8b 50 08 <48> 89 51 08 48 89 0a 48 b9 00 01 00 00 00 00 a
d de 48 ba 22 01 00
Sep 28 12:19:42 pve kernel: [ 1470.872123] RSP: 0018:ffffa7d359cbfd10 EFLAGS: 00010286
Sep 28 12:19:42 pve kernel: [ 1470.872127] RAX: ffff88b39647c2f0 RBX: ffff88ad9fc104d8 RCX: ffef88b39647c890
Sep 28 12:19:42 pve kernel: [ 1470.872135] RDX: ffff88b39647c5f0 RSI: ffff88ad9fc11360 RDI: ffff88ad9fc11720
Sep 28 12:19:42 pve kernel: [ 1470.872139] RBP: ffffa7d359cbfd40 R08: ffff88ad9fc11368 R09: ffffa7d359cbfd70
Sep 28 12:19:42 pve kernel: [ 1470.872142] R10: ffff88aefd401838 R11: 0000000000000001 R12: ffff88ad9fc11360
Sep 28 12:19:42 pve kernel: [ 1470.872146] R13: ffff88b39647c2d0 R14: ffff88ad9fc10898 R15: ffff88ad9fc11720
Sep 28 12:19:42 pve kernel: [ 1470.872150] FS:  0000000000000000(0000) GS:ffff88b41eac0000(0000) knlGS:0000000000000000
Sep 28 12:19:42 pve kernel: [ 1470.872154] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Sep 28 12:19:42 pve kernel: [ 1470.872157] CR2: 0000558e97c43198 CR3: 0000000fb59ca000 CR4: 0000000000340ee0
Sep 28 12:19:42 pve kernel: [ 1470.872161] Call Trace:
Sep 28 12:19:42 pve kernel: [ 1470.872194]  zio_done+0x4f6/0x1030 [zfs]
Sep 28 12:19:42 pve kernel: [ 1470.872221]  zio_execute+0x99/0xf0 [zfs]
Sep 28 12:19:42 pve kernel: [ 1470.872227]  taskq_thread+0x2ec/0x4d0 [spl]
Sep 28 12:19:42 pve kernel: [ 1470.872232]  ? wake_up_q+0x80/0x80
Sep 28 12:19:42 pve kernel: [ 1470.872257]  ? zio_taskq_member.isra.12.constprop.17+0x70/0x70 [zfs]
Sep 28 12:19:42 pve kernel: [ 1470.872262]  kthread+0x120/0x140
Sep 28 12:19:42 pve kernel: [ 1470.872266]  ? task_done+0xb0/0xb0 [spl]
Sep 28 12:19:42 pve kernel: [ 1470.872270]  ? kthread_park+0x90/0x90
Sep 28 12:19:42 pve kernel: [ 1470.872273]  ret_from_fork+0x22/0x40
Sep 28 12:19:42 pve kernel: [ 1470.872276] Modules linked in: tcp_diag inet_diag veth ebtable_filter ebtables ip_set ip6table_raw iptable_raw ip6table_filter ip6_tables iptable_filter bpfilter 8021q garp mrp softdog nfnetli
nk_log nfnetlink zfs(PO) zunicode(PO) zlua(PO) zavl(PO) icp(PO) edac_mce_amd snd_hda_codec_hdmi kvm_amd kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel snd_hda_codec_realtek snd_hda_codec_generic ledtrig_aud
io snd_hda_intel snd_intel_dspcfg aesni_intel crypto_simd cryptd glue_helper nouveau mxm_wmi video ttm pcspkr wmi_bmof drm_kms_helper k10temp snd_hda_codec snd_hda_core drm snd_hwdep i2c_algo_bit snd_pcm fb_sys_fops ccp sys
copyarea snd_timer sysfillrect sysimgblt snd soundcore input_leds mac_hid zcommon(PO) znvpair(PO) spl(O) vhost_net vhost tap ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi sunrpc ip
_tables x_tables autofs4 btrfs xor zstd_compress raid6_pq dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio libcrc32c hid_generic usbkbd
Sep 28 12:19:42 pve kernel: [ 1470.872300]  usbhid hid i2c_piix4 e1000e r8169 xhci_pci realtek ahci xhci_hcd libahci wmi gpio_amdpt gpio_generic
Sep 28 12:19:42 pve kernel: [ 1470.872328] ---[ end trace 0f974716c5f0bac8 ]---
Sep 28 12:19:42 pve kernel: [ 1470.872353] RIP: 0010:zio_remove_child+0x6f/0x140 [zfs]
Sep 28 12:19:42 pve kernel: [ 1470.872356] Code: e8 03 00 00 48 89 45 d0 e8 ce b5 23 fb 48 8b 45 d0 49 89 84 24 e8 03 00 00 48 8b 83 50 01 00 00 4c 01 e8 48 8b 08 48 8b 50 08 <48> 89 51 08 48 89 0a 48 b9 00 01 00 00 00 00 a
d de 48 ba 22 01 00
Sep 28 12:19:42 pve kernel: [ 1470.872362] RSP: 0018:ffffa7d359cbfd10 EFLAGS: 00010286
Sep 28 12:19:42 pve kernel: [ 1470.872365] RAX: ffff88b39647c2f0 RBX: ffff88ad9fc104d8 RCX: ffef88b39647c890
Sep 28 12:19:42 pve kernel: [ 1470.872368] RDX: ffff88b39647c5f0 RSI: ffff88ad9fc11360 RDI: ffff88ad9fc11720
Sep 28 12:19:42 pve kernel: [ 1470.872371] RBP: ffffa7d359cbfd40 R08: ffff88ad9fc11368 R09: ffffa7d359cbfd70
Sep 28 12:19:42 pve kernel: [ 1470.872374] R10: ffff88aefd401838 R11: 0000000000000001 R12: ffff88ad9fc11360
Sep 28 12:19:42 pve kernel: [ 1470.872377] R13: ffff88b39647c2d0 R14: ffff88ad9fc10898 R15: ffff88ad9fc11720
Sep 28 12:19:42 pve kernel: [ 1470.872380] FS:  0000000000000000(0000) GS:ffff88b41eac0000(0000) knlGS:0000000000000000
Sep 28 12:19:42 pve kernel: [ 1470.872383] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Sep 28 12:19:42 pve kernel: [ 1470.872386] CR2: 0000558e97c43198 CR3: 0000000fb59ca000 CR4: 0000000000340ee0

Any help you might be able to provide is appreciated.

 

[viertaxa]

Marking this as likely solved. I ran Memtest86 on the hardware and am getting a ton of errors, so it's hard to trust anything on the system. Hopefully the RMA hardware will help sort the issue out.