[SOLVED] Kernel panic on 5.3.13-1-pve and ZFS 0.8.2-pve2

[kristian.kirilov]

Hello,
I'm not sure this is the right place for that, but today I noticed some kind of kernel panic related to ZFS. Also one of my boxes just hang on, after few working hours.
Have you ever seen this before?

[    6.462783] intel_rapl_common: Found RAPL domain core
[    6.462784] intel_rapl_common: Found RAPL domain uncore
[    6.462786] intel_rapl_common: Found RAPL domain dram
[    6.462788] intel_rapl_common: RAPL package-0 domain package locked by BIOS
[    6.462792] intel_rapl_common: RAPL package-0 domain dram locked by BIOS
[    6.863865] ZFS: Loaded module v0.8.2-pve2, ZFS pool version 5000, ZFS filesystem version 5
[    6.949896] ------------[ cut here ]------------
[    6.949900] General protection fault in user access. Non-canonical address?
[    6.949906] WARNING: CPU: 2 PID: 747 at arch/x86/mm/extable.c:126 ex_handler_uaccess+0x52/0x60
[    6.949909] Modules linked in: intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel sn
d_hda_codec_hdmi zfs(PO) aesni_intel zunicode(PO) aes_x86_64 crypto_simd zlua(PO) cryptd zavl(PO) glue_helper mei_hdcp icp(PO) intel_cstate snd_hda_codec_realtek snd_hda_codec_generic ledtri
g_audio intel_rapl_perf pcspkr i915 wmi_bmof snd_hda_intel snd_hda_codec drm_kms_helper snd_hda_core snd_hwdep drm snd_pcm snd_timer snd mei_me i2c_algo_bit soundcore fb_sys_fops syscopyarea
 sysfillrect sysimgblt mei ie31200_edac mac_hid intel_smartconnect zcommon(PO) znvpair(PO) spl(O) vhost_net vhost tap ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi
_transport_iscsi drbd sunrpc lru_cache ip_tables x_tables autofs4 raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear raid
1 tg3 lpc_ich ahci i2c_i801 libahci e1000e wmi video
[    6.949949] CPU: 2 PID: 747 Comm: kworker/u8:3 Tainted: P           O      5.3.13-1-pve #1
[    6.949951] Hardware name: LENOVO 30A0A086BL/SHARKBAY, BIOS FBKTD8AUS 09/17/2019
[    6.949954] RIP: 0010:ex_handler_uaccess+0x52/0x60
[    6.949956] Code: c4 08 b8 01 00 00 00 5b 5d c3 80 3d 85 d6 78 01 00 75 db 48 c7 c7 58 10 34 a7 48 89 75 f0 c6 05 71 d6 78 01 01 e8 ff a1 01 00 <0f> 0b 48 8b 75 f0 eb bc 66 0f 1f 44 00 00
 0f 1f 44 00 00 55 80 3d
[    6.949959] RSP: 0018:ffffa1504085bcc0 EFLAGS: 00010282
[    6.949960] RAX: 0000000000000000 RBX: ffffffffa6e02448 RCX: 0000000000000000
[    6.949962] RDX: 000000000000003f RSI: ffffffffa7b83f7f RDI: 0000000000000246
[    6.949963] RBP: ffffa1504085bcd0 R08: ffffffffa7b83f40 R09: 0000000000029fc0
[    6.949965] R10: 000000101930c1f0 R11: ffffffffa7b83f40 R12: 000000000000000d
[    6.949966] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[    6.949968] FS:  0000000000000000(0000) GS:ffff8c9dbe900000(0000) knlGS:0000000000000000
[    6.949970] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    6.949971] CR2: 0000558ace707018 CR3: 00000007f9058004 CR4: 00000000001606e0
[    6.949973] Call Trace:
[    6.949977]  fixup_exception+0x4a/0x61
[    6.949981]  do_general_protection+0x4e/0x150
[    6.949984]  general_protection+0x28/0x30
[    6.949987] RIP: 0010:strnlen_user+0x4c/0x110
[    6.949989] Code: f8 0f 86 e1 00 00 00 48 29 f8 45 31 c9 0f 1f 00 0f ae e8 48 39 c6 49 89 fa 48 0f 46 c6 41 83 e2 07 48 83 e7 f8 31 c9 4c 01 d0 <4c> 8b 1f 85 c9 0f 85 96 00 00 00 42 8d 0c
 d5 00 00 00 00 41 b8 01
[    6.949991] RSP: 0018:ffffa1504085bde8 EFLAGS: 00010206
[    6.949993] RAX: 0000000000020000 RBX: 363ba6253fedee00 RCX: 0000000000000000
[    6.949994] RDX: 363ba6253fedee00 RSI: 0000000000020000 RDI: 363ba6253fedee00
[    6.949996] RBP: ffffa1504085bdf8 R08: 8080808080808080 R09: 0000000000000000
[    6.949997] R10: 0000000000000000 R11: 0000000000000000 R12: 00007fffffffefe6
[    6.949999] R13: ffff8c9dabf39fe6 R14: 0000000000000000 R15: fffff8195fafce40
[    6.950003]  ? _copy_from_user+0x3e/0x60
[    6.950006]  copy_strings.isra.35+0x92/0x380
[    6.950008]  __do_execve_file.isra.42+0x5b5/0x9d0
[    6.950011]  ? kmem_cache_alloc+0x110/0x220
[    6.950013]  do_execve+0x25/0x30
[    6.950016]  call_usermodehelper_exec_async+0x188/0x1b0
[    6.950018]  ? call_usermodehelper+0xb0/0xb0
[    6.950021]  ret_from_fork+0x35/0x40
[    6.950023] ---[ end trace a45d447443d33a31 ]---
[   13.062258]  zd0: p1
[   13.441957]  zd16: p1 p2 < p5 p6 p7 >
[   13.667304]  zd32: p1 p2
[   13.880428]  zd48: p1 p2 p3
[   14.475730]  zd64: p1 p2 < p5 >

 

[Stefan_R]

The trace you posted stems from a known ZFS issue (https://github.com/zfsonlinux/zfs/issues/9417#issuecomment-548085631), but can be considered harmless. And just FYI, that's not a panic, it's a warning, with a panic the system would always hang.

The hang you saw on one of your machines is *probably* unrelated, more logs would be required to figure that one out. If you continue seeing it, consider setting up kdump to get more info.

 

[kristian.kirilov]

Thanks a lot for your explanation @Stefan_R, will continue to watch the behavior of the system, and if it freezes again, will setup kdump as you said.
Do I have to close the thread? Or this doesn't matter for you?

 

[Stefan_R]

Glad it helped You can mark it as 'solved' if you want.

 

[kristian.kirilov]

Yeah, I want to mark it as solved, but to be honest, I'm not sure how to do it?

 

[Stefan_R]

'More Options' (3 dots ...) above the first post -> 'Edit Thread', and select the 'SOLVED' prefix

 

[kristian.kirilov]

It's done. Thanks a lot @Stefan_R