We are experiencing kernel panic and system freeze when we try to stop VM with PCI in KVM machine. When we stop VM, we got this and node reboot is needed.
Trace:
[ 878.355774] CPU: 34 PID: 45959 Comm: ip Tainted: P O 5.3.13-1-pve #1
[ 878.355825] Hardware name: Dell Inc. PowerEdge R630/0CNCJW, BIOS 2.9.1 12/04/2018
[ 878.355898] RIP: 0010:i40e_config_vf_promiscuous_mode+0x177/0x330 [i40e]
[ 878.355946] Code: 8b 00 83 d1 00 48 85 c0 75 ef 48 83 c6 08 48 39 75 c8 75 dd 85 c9 74 79 0f b6 45 b8 45 31 d2 89 45 d4 4d 8b 2f 4d 85 ed 74 58 <41> 0f b7 4d 16 66 81 f9 ff 0f 77 43 0f b7 b3 ea 0c 00 00 8b 55 d4
[ 878.356061] RSP: 0018:ffff9a7046a97560 EFLAGS: 00010202
[ 878.356100] RAX: 0000000000000000 RBX: ffff889cd7f0d000 RCX: 0000000000000000
[ 878.356147] RDX: 0000000000000000 RSI: 0000000006000000 RDI: 0000000000000000
[ 878.356194] RBP: ffff9a7046a975b8 R08: 0000000000000000 R09: 0000000000000022
[ 878.356240] R10: 0000000000000000 R11: ffff9a7046a97500 R12: ffff889dcfcfc008
[ 878.356287] R13: 4d2d4b7e20d769c2 R14: 0000000000000000 R15: ffff889cd7f0d338
[ 878.356335] FS: 00007f67256f8e40(0000) GS:ffff889ddf840000(0000) knlGS:0000000000000000
[ 878.356388] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 878.356427] CR2: 00007f6725a4c730 CR3: 00000006eea66003 CR4: 00000000001626e0
[ 878.356475] Call Trace:
[ 878.356518] i40e_ndo_set_vf_port_vlan+0x1d1/0x2d0 [i40e]
[ 878.356561] do_setlink+0x572/0xe40
[ 878.356593] __rtnl_newlink+0x538/0x900
[ 878.356627] ? __nla_validate_parse+0x50/0x7f0
[ 878.356665] ? mem_cgroup_try_charge+0x71/0x190
[ 878.356700] ? __alloc_pages_nodemask+0x16a/0x330
[ 878.356735] ? page_counter_try_charge+0x5b/0xd0
[ 878.356775] ? __mutex_lock.isra.10+0x3c0/0x4c0
[ 878.356810] ? _cond_resched+0x19/0x30
[ 878.356840] ? kmem_cache_alloc_trace+0x15e/0x230
[ 878.356875] rtnl_newlink+0x48/0x70
[ 878.356903] rtnetlink_rcv_msg+0x2b6/0x370
[ 878.356936] ? _cond_resched+0x19/0x30
[ 878.356965] ? rtnl_calcit.isra.35+0x100/0x100
[ 878.357001] netlink_rcv_skb+0x4f/0x120
[ 878.357031] rtnetlink_rcv+0x15/0x20
[ 878.357061] netlink_unicast+0x197/0x220
[ 878.357093] netlink_sendmsg+0x213/0x3c0
[ 878.357125] sock_sendmsg+0x63/0x70
[ 878.357153] ___sys_sendmsg+0x29f/0x320
[ 878.357186] ? call_rcu+0x10/0x20
[ 878.358471] ? destroy_inode+0x65/0x80
[ 878.359747] ? evict+0x139/0x1a0
[ 878.360996] ? _cond_resched+0x19/0x30
[ 878.362225] ? __dentry_kill+0x124/0x170
[ 878.363433] __sys_sendmsg+0x5c/0xa0
[ 878.364620] __x64_sys_sendmsg+0x1f/0x30
[ 878.365805] do_syscall_64+0x5a/0x130
[ 878.366993] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 878.368196] RIP: 0033:0x7f6725aa9914
[ 878.369379] Code: 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b5 0f 1f 80 00 00 00 00 48 8d 05 e9 5d 0c 00 8b 00 85 c0 75 13 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 41 54 41 89 d4 55 48 89 f5 53
[ 878.371882] RSP: 002b:00007ffd55473218 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 878.373128] RAX: ffffffffffffffda RBX: 000000005df26cc1 RCX: 00007f6725aa9914
[ 878.374367] RDX: 0000000000000000 RSI: 00007ffd55473280 RDI: 0000000000000005
[ 878.375599] RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000010
[ 878.376790] R10: fffffffffffff466 R11: 0000000000000246 R12: 0000000000000001
[ 878.377942] R13: 0000555ce4170560 R14: 0000000000000000 R15: 00007ffd55473a58
[ 878.379077] Modules linked in: rbd libceph ebtable_filter ebtables ip_set ip6table_raw iptable_raw ip6table_filter ip6_tables sctp iptable_filter bpfilter openvswitch nsh nf_conncount nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 softdog nfnetlink_log nfnetlink ipmi_ssif intel_rapl_msr intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel mgag200 aes_x86_64 drm_vram_helper crypto_simd cryptd ttm glue_helper dcdbas drm_kms_helper intel_cstate intel_rapl_perf drm pcspkr joydev input_leds fb_sys_fops syscopyarea sysfillrect sysimgblt mxm_wmi mei_me mei ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter mac_hid vhost_net vhost tap ib_iser rdma_cm iw_cm ib_cm ib_core sunrpc iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi vfio_pci vfio_virqfd irqbypass vfio_iommu_type1 vfio ip_tables x_tables autofs4 zfs(PO) zunicode(PO) zlua(PO) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) btrfs xor
[ 878.379139] zstd_compress raid6_pq libcrc32c hid_generic usbkbd usbmouse usbhid hid i40e(O) ixgbe igb lpc_ich i2c_algo_bit xfrm_algo ahci dca libahci mdio megaraid_sas wmi
[ 878.392543] ---[ end trace f48e2fe75d2d811a ]---
[ 878.431337] RIP: 0010:i40e_config_vf_promiscuous_mode+0x177/0x330 [i40e]
[ 878.433138] Code: 8b 00 83 d1 00 48 85 c0 75 ef 48 83 c6 08 48 39 75 c8 75 dd 85 c9 74 79 0f b6 45 b8 45 31 d2 89 45 d4 4d 8b 2f 4d 85 ed 74 58 <41> 0f b7 4d 16 66 81 f9 ff 0f 77 43 0f b7 b3 ea 0c 00 00 8b 55 d4
[ 878.436303] RSP: 0018:ffff9a7046a97560 EFLAGS: 00010202
[ 878.437873] RAX: 0000000000000000 RBX: ffff889cd7f0d000 RCX: 0000000000000000
[ 878.439483] RDX: 0000000000000000 RSI: 0000000006000000 RDI: 0000000000000000
[ 878.441073] RBP: ffff9a7046a975b8 R08: 0000000000000000 R09: 0000000000000022
[ 878.442666] R10: 0000000000000000 R11: ffff9a7046a97500 R12: ffff889dcfcfc008
[ 878.444292] R13: 4d2d4b7e20d769c2 R14: 0000000000000000 R15: ffff889cd7f0d338
[ 878.445931] FS: 00007f67256f8e40(0000) GS:ffff889ddf840000(0000) knlGS:0000000000000000
[ 878.447665] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 878.449311] CR2: 00007f6725a4c730 CR3: 00000006eea66003 CR4: 00000000001626e0
Kindly,
Pau
Trace:
[ 878.355774] CPU: 34 PID: 45959 Comm: ip Tainted: P O 5.3.13-1-pve #1
[ 878.355825] Hardware name: Dell Inc. PowerEdge R630/0CNCJW, BIOS 2.9.1 12/04/2018
[ 878.355898] RIP: 0010:i40e_config_vf_promiscuous_mode+0x177/0x330 [i40e]
[ 878.355946] Code: 8b 00 83 d1 00 48 85 c0 75 ef 48 83 c6 08 48 39 75 c8 75 dd 85 c9 74 79 0f b6 45 b8 45 31 d2 89 45 d4 4d 8b 2f 4d 85 ed 74 58 <41> 0f b7 4d 16 66 81 f9 ff 0f 77 43 0f b7 b3 ea 0c 00 00 8b 55 d4
[ 878.356061] RSP: 0018:ffff9a7046a97560 EFLAGS: 00010202
[ 878.356100] RAX: 0000000000000000 RBX: ffff889cd7f0d000 RCX: 0000000000000000
[ 878.356147] RDX: 0000000000000000 RSI: 0000000006000000 RDI: 0000000000000000
[ 878.356194] RBP: ffff9a7046a975b8 R08: 0000000000000000 R09: 0000000000000022
[ 878.356240] R10: 0000000000000000 R11: ffff9a7046a97500 R12: ffff889dcfcfc008
[ 878.356287] R13: 4d2d4b7e20d769c2 R14: 0000000000000000 R15: ffff889cd7f0d338
[ 878.356335] FS: 00007f67256f8e40(0000) GS:ffff889ddf840000(0000) knlGS:0000000000000000
[ 878.356388] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 878.356427] CR2: 00007f6725a4c730 CR3: 00000006eea66003 CR4: 00000000001626e0
[ 878.356475] Call Trace:
[ 878.356518] i40e_ndo_set_vf_port_vlan+0x1d1/0x2d0 [i40e]
[ 878.356561] do_setlink+0x572/0xe40
[ 878.356593] __rtnl_newlink+0x538/0x900
[ 878.356627] ? __nla_validate_parse+0x50/0x7f0
[ 878.356665] ? mem_cgroup_try_charge+0x71/0x190
[ 878.356700] ? __alloc_pages_nodemask+0x16a/0x330
[ 878.356735] ? page_counter_try_charge+0x5b/0xd0
[ 878.356775] ? __mutex_lock.isra.10+0x3c0/0x4c0
[ 878.356810] ? _cond_resched+0x19/0x30
[ 878.356840] ? kmem_cache_alloc_trace+0x15e/0x230
[ 878.356875] rtnl_newlink+0x48/0x70
[ 878.356903] rtnetlink_rcv_msg+0x2b6/0x370
[ 878.356936] ? _cond_resched+0x19/0x30
[ 878.356965] ? rtnl_calcit.isra.35+0x100/0x100
[ 878.357001] netlink_rcv_skb+0x4f/0x120
[ 878.357031] rtnetlink_rcv+0x15/0x20
[ 878.357061] netlink_unicast+0x197/0x220
[ 878.357093] netlink_sendmsg+0x213/0x3c0
[ 878.357125] sock_sendmsg+0x63/0x70
[ 878.357153] ___sys_sendmsg+0x29f/0x320
[ 878.357186] ? call_rcu+0x10/0x20
[ 878.358471] ? destroy_inode+0x65/0x80
[ 878.359747] ? evict+0x139/0x1a0
[ 878.360996] ? _cond_resched+0x19/0x30
[ 878.362225] ? __dentry_kill+0x124/0x170
[ 878.363433] __sys_sendmsg+0x5c/0xa0
[ 878.364620] __x64_sys_sendmsg+0x1f/0x30
[ 878.365805] do_syscall_64+0x5a/0x130
[ 878.366993] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 878.368196] RIP: 0033:0x7f6725aa9914
[ 878.369379] Code: 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b5 0f 1f 80 00 00 00 00 48 8d 05 e9 5d 0c 00 8b 00 85 c0 75 13 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 41 54 41 89 d4 55 48 89 f5 53
[ 878.371882] RSP: 002b:00007ffd55473218 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 878.373128] RAX: ffffffffffffffda RBX: 000000005df26cc1 RCX: 00007f6725aa9914
[ 878.374367] RDX: 0000000000000000 RSI: 00007ffd55473280 RDI: 0000000000000005
[ 878.375599] RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000010
[ 878.376790] R10: fffffffffffff466 R11: 0000000000000246 R12: 0000000000000001
[ 878.377942] R13: 0000555ce4170560 R14: 0000000000000000 R15: 00007ffd55473a58
[ 878.379077] Modules linked in: rbd libceph ebtable_filter ebtables ip_set ip6table_raw iptable_raw ip6table_filter ip6_tables sctp iptable_filter bpfilter openvswitch nsh nf_conncount nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 softdog nfnetlink_log nfnetlink ipmi_ssif intel_rapl_msr intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel mgag200 aes_x86_64 drm_vram_helper crypto_simd cryptd ttm glue_helper dcdbas drm_kms_helper intel_cstate intel_rapl_perf drm pcspkr joydev input_leds fb_sys_fops syscopyarea sysfillrect sysimgblt mxm_wmi mei_me mei ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter mac_hid vhost_net vhost tap ib_iser rdma_cm iw_cm ib_cm ib_core sunrpc iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi vfio_pci vfio_virqfd irqbypass vfio_iommu_type1 vfio ip_tables x_tables autofs4 zfs(PO) zunicode(PO) zlua(PO) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) btrfs xor
[ 878.379139] zstd_compress raid6_pq libcrc32c hid_generic usbkbd usbmouse usbhid hid i40e(O) ixgbe igb lpc_ich i2c_algo_bit xfrm_algo ahci dca libahci mdio megaraid_sas wmi
[ 878.392543] ---[ end trace f48e2fe75d2d811a ]---
[ 878.431337] RIP: 0010:i40e_config_vf_promiscuous_mode+0x177/0x330 [i40e]
[ 878.433138] Code: 8b 00 83 d1 00 48 85 c0 75 ef 48 83 c6 08 48 39 75 c8 75 dd 85 c9 74 79 0f b6 45 b8 45 31 d2 89 45 d4 4d 8b 2f 4d 85 ed 74 58 <41> 0f b7 4d 16 66 81 f9 ff 0f 77 43 0f b7 b3 ea 0c 00 00 8b 55 d4
[ 878.436303] RSP: 0018:ffff9a7046a97560 EFLAGS: 00010202
[ 878.437873] RAX: 0000000000000000 RBX: ffff889cd7f0d000 RCX: 0000000000000000
[ 878.439483] RDX: 0000000000000000 RSI: 0000000006000000 RDI: 0000000000000000
[ 878.441073] RBP: ffff9a7046a975b8 R08: 0000000000000000 R09: 0000000000000022
[ 878.442666] R10: 0000000000000000 R11: ffff9a7046a97500 R12: ffff889dcfcfc008
[ 878.444292] R13: 4d2d4b7e20d769c2 R14: 0000000000000000 R15: ffff889cd7f0d338
[ 878.445931] FS: 00007f67256f8e40(0000) GS:ffff889ddf840000(0000) knlGS:0000000000000000
[ 878.447665] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 878.449311] CR2: 00007f6725a4c730 CR3: 00000006eea66003 CR4: 00000000001626e0
Kindly,
Pau
If hardware is needed we can give remote access.