Hello All,
I'm Facing an issue where one of my network interfaces crashes after i boot the server by few mins. here's the error message in the logs
Feb 17 21:29:54 proxmox kernel: NETDEV WATCHDOG: enp7s0 (r8169): transmit queue 0 timed out
Feb 17 21:29:54 proxmox kernel: WARNING: CPU: 35 PID: 0 at net/sched/sch_generic.c:477 dev_watchdog+0x273/0x280
Feb 17 21:29:54 proxmox kernel: Modules linked in: tcp_diag inet_diag ebtable_filter ebtables ip_set ip6table_raw iptable_raw ip6table_filter ip6_tables iptable_filter bpfilter nf_tables 8021q garp mrp bonding tls softdog nfnetlink_log nfnetlink nouveau intel_rapl_msr intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp mxm_wmi video snd_hda_codec_realtek kvm_intel snd_hda_codec_generic drm_ttm_helper kvm ledtrig_audio irqbypass ttm drm_kms_helper crct10dif_pclmul snd_hda_codec_hdmi ghash_clmulni_intel cec mt7601u rc_core snd_hda_intel fb_sys_fops syscopyarea sysfillrect sysimgblt snd_intel_dspcfg snd_intel_sdw_acpi aesni_intel snd_hda_codec snd_hda_core crypto_simd cryptd mac80211 rtsx_usb_ms snd_hwdep snd_pcm snd_timer cfg80211 mei_me snd intel_wmi_thunderbolt think_lmi soundcore firmware_attributes_class wmi_bmof mei rapl intel_cstate memstick pcspkr efi_pstore libarc4 mac_hid zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) vhost_net vhost
Feb 17 21:29:54 proxmox kernel: vhost_iotlb tap ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp drm libiscsi_tcp libiscsi scsi_transport_iscsi sunrpc ip_tables x_tables autofs4 dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio btrfs blake2b_generic xor zstd_compress rtsx_usb_sdmmc rtsx_usb raid6_pq libcrc32c crc32_pclmul xhci_pci igb xhci_pci_renesas i2c_i801 i2c_algo_bit r8169 i2c_smbus lpc_ich dca realtek ehci_pci e1000e ahci ehci_hcd xhci_hcd pata_acpi libahci wmi
Feb 17 21:29:54 proxmox kernel: CPU: 35 PID: 0 Comm: swapper/35 Tainted: P O 5.15.12-1-pve #1
Feb 17 21:29:54 proxmox kernel: Hardware name: LENOVO 30A8S0BM00/1030, BIOS A5KTA1AUS 10/15/2018
Feb 17 21:29:54 proxmox kernel: RIP: 0010:dev_watchdog+0x273/0x280
Feb 17 21:29:54 proxmox kernel: Code: eb 97 48 8b 5d d0 c6 05 1a bd 52 01 01 48 89 df e8 12 84 f9 ff 44 89 e1 48 89 de 48 c7 c7 80 f8 69 94 48 89 c2 e8 f1 ec 1b 00 <0f> 0b eb 80 e9 6e 02 25 00 0f 1f 40 00 0f 1f 44 00 00 55 49 89 ca
Feb 17 21:29:54 proxmox kernel: RSP: 0018:ffff985406bb4e70 EFLAGS: 00010282
Feb 17 21:29:54 proxmox kernel: RAX: 0000000000000000 RBX: ffff897ad96c0000 RCX: ffff8989ffde09c8
Feb 17 21:29:54 proxmox kernel: RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff8989ffde09c0
Feb 17 21:29:54 proxmox kernel: RBP: ffff985406bb4ea8 R08: 0000000000000003 R09: 0000000000000001
Feb 17 21:29:54 proxmox kernel: R10: 0000000000ffff0a R11: 0000000000000001 R12: 0000000000000000
Feb 17 21:29:54 proxmox kernel: R13: ffff897ac415f280 R14: 0000000000000001 R15: ffff897ad96c04c0
Feb 17 21:29:54 proxmox kernel: FS: 0000000000000000(0000) GS:ffff8989ffdc0000(0000) knlGS:0000000000000000
Feb 17 21:29:54 proxmox kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Feb 17 21:29:54 proxmox kernel: CR2: 00007f02c4d2621c CR3: 0000001ccd010001 CR4: 00000000001726e0
Feb 17 21:29:54 proxmox kernel: Call Trace:
Feb 17 21:29:54 proxmox kernel: <IRQ>
Feb 17 21:29:54 proxmox kernel: ? pfifo_fast_enqueue+0x150/0x150
Feb 17 21:29:54 proxmox kernel: call_timer_fn+0x2b/0x120
Feb 17 21:29:54 proxmox kernel: __run_timers.part.0+0x1db/0x250
Feb 17 21:29:54 proxmox kernel: ? ktime_get+0x46/0xc0
Feb 17 21:29:54 proxmox kernel: ? lapic_next_deadline+0x2c/0x40
Feb 17 21:29:54 proxmox kernel: ? clockevents_program_event+0xa7/0x120
Feb 17 21:29:54 proxmox kernel: run_timer_softirq+0x2a/0x50
Feb 17 21:29:54 proxmox kernel: __do_softirq+0xd9/0x2e6
Feb 17 21:29:54 proxmox kernel: irq_exit_rcu+0x8c/0xb0
Feb 17 21:29:54 proxmox kernel: sysvec_apic_timer_interrupt+0x7c/0x90
Feb 17 21:29:54 proxmox kernel: </IRQ>
Feb 17 21:29:54 proxmox kernel: <TASK>
Feb 17 21:29:54 proxmox kernel: asm_sysvec_apic_timer_interrupt+0x12/0x20
Feb 17 21:29:54 proxmox kernel: RIP: 0010:cpuidle_enter_state+0xd9/0x620
Feb 17 21:29:54 proxmox kernel: Code: 3d 34 02 64 6c e8 b7 23 72 ff 49 89 c7 0f 1f 44 00 00 31 ff e8 98 2f 72 ff 80 7d d0 00 0f 85 5a 01 00 00 fb 66 0f 1f 44 00 00 <45> 85 f6 0f 88 66 01 00 00 4d 63 ee 49 83 fd 09 0f 87 e1 03 00 00
Feb 17 21:29:54 proxmox kernel: RSP: 0018:ffff98540642be38 EFLAGS: 00000246
Feb 17 21:29:54 proxmox kernel: RAX: ffff8989ffdf0f80 RBX: ffffb843ff7f8300 RCX: 0000000000000000
Feb 17 21:29:54 proxmox kernel: RDX: 000000000000007c RSI: 0000000033518eeb RDI: 0000000000000000
Feb 17 21:29:54 proxmox kernel: RBP: ffff98540642be88 R08: 000000e9d8eb993b R09: 0000000000061a80
Feb 17 21:29:54 proxmox kernel: R10: 0000000000000005 R11: 071c71c71c71c71c R12: ffffffff94ed3740
Feb 17 21:29:54 proxmox kernel: R13: 0000000000000004 R14: 0000000000000004 R15: 000000e9d8eb993b
Feb 17 21:29:54 proxmox kernel: ? cpuidle_enter_state+0xc8/0x620
Feb 17 21:29:54 proxmox kernel: cpuidle_enter+0x2e/0x40
Feb 17 21:29:54 proxmox kernel: do_idle+0x209/0x2b0
Feb 17 21:29:54 proxmox kernel: cpu_startup_entry+0x20/0x30
Feb 17 21:29:54 proxmox kernel: start_secondary+0x12a/0x180
Feb 17 21:29:54 proxmox kernel: secondary_startup_64_no_verify+0xc2/0xcb
Feb 17 21:29:54 proxmox kernel: </TASK>
Feb 17 21:29:54 proxmox kernel:
---[ end trace 727774c916316baf ]---
i noticed this happened after i updated the PVE Kernel to (pve-kernel-5.15). i noticed after the upgrade that the overall CPU usage decreased for all the virtual machines and that's good.
so when i run apt install pve-kernel- i can see a lot of newer PVEs but, i'm afraid to upgrade as it may break anything else. So, any idea how to fix this issue?
Thanks in advance
Shady K
I'm Facing an issue where one of my network interfaces crashes after i boot the server by few mins. here's the error message in the logs
Feb 17 21:29:54 proxmox kernel: NETDEV WATCHDOG: enp7s0 (r8169): transmit queue 0 timed out
Feb 17 21:29:54 proxmox kernel: WARNING: CPU: 35 PID: 0 at net/sched/sch_generic.c:477 dev_watchdog+0x273/0x280
Feb 17 21:29:54 proxmox kernel: Modules linked in: tcp_diag inet_diag ebtable_filter ebtables ip_set ip6table_raw iptable_raw ip6table_filter ip6_tables iptable_filter bpfilter nf_tables 8021q garp mrp bonding tls softdog nfnetlink_log nfnetlink nouveau intel_rapl_msr intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp mxm_wmi video snd_hda_codec_realtek kvm_intel snd_hda_codec_generic drm_ttm_helper kvm ledtrig_audio irqbypass ttm drm_kms_helper crct10dif_pclmul snd_hda_codec_hdmi ghash_clmulni_intel cec mt7601u rc_core snd_hda_intel fb_sys_fops syscopyarea sysfillrect sysimgblt snd_intel_dspcfg snd_intel_sdw_acpi aesni_intel snd_hda_codec snd_hda_core crypto_simd cryptd mac80211 rtsx_usb_ms snd_hwdep snd_pcm snd_timer cfg80211 mei_me snd intel_wmi_thunderbolt think_lmi soundcore firmware_attributes_class wmi_bmof mei rapl intel_cstate memstick pcspkr efi_pstore libarc4 mac_hid zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) vhost_net vhost
Feb 17 21:29:54 proxmox kernel: vhost_iotlb tap ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp drm libiscsi_tcp libiscsi scsi_transport_iscsi sunrpc ip_tables x_tables autofs4 dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio btrfs blake2b_generic xor zstd_compress rtsx_usb_sdmmc rtsx_usb raid6_pq libcrc32c crc32_pclmul xhci_pci igb xhci_pci_renesas i2c_i801 i2c_algo_bit r8169 i2c_smbus lpc_ich dca realtek ehci_pci e1000e ahci ehci_hcd xhci_hcd pata_acpi libahci wmi
Feb 17 21:29:54 proxmox kernel: CPU: 35 PID: 0 Comm: swapper/35 Tainted: P O 5.15.12-1-pve #1
Feb 17 21:29:54 proxmox kernel: Hardware name: LENOVO 30A8S0BM00/1030, BIOS A5KTA1AUS 10/15/2018
Feb 17 21:29:54 proxmox kernel: RIP: 0010:dev_watchdog+0x273/0x280
Feb 17 21:29:54 proxmox kernel: Code: eb 97 48 8b 5d d0 c6 05 1a bd 52 01 01 48 89 df e8 12 84 f9 ff 44 89 e1 48 89 de 48 c7 c7 80 f8 69 94 48 89 c2 e8 f1 ec 1b 00 <0f> 0b eb 80 e9 6e 02 25 00 0f 1f 40 00 0f 1f 44 00 00 55 49 89 ca
Feb 17 21:29:54 proxmox kernel: RSP: 0018:ffff985406bb4e70 EFLAGS: 00010282
Feb 17 21:29:54 proxmox kernel: RAX: 0000000000000000 RBX: ffff897ad96c0000 RCX: ffff8989ffde09c8
Feb 17 21:29:54 proxmox kernel: RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff8989ffde09c0
Feb 17 21:29:54 proxmox kernel: RBP: ffff985406bb4ea8 R08: 0000000000000003 R09: 0000000000000001
Feb 17 21:29:54 proxmox kernel: R10: 0000000000ffff0a R11: 0000000000000001 R12: 0000000000000000
Feb 17 21:29:54 proxmox kernel: R13: ffff897ac415f280 R14: 0000000000000001 R15: ffff897ad96c04c0
Feb 17 21:29:54 proxmox kernel: FS: 0000000000000000(0000) GS:ffff8989ffdc0000(0000) knlGS:0000000000000000
Feb 17 21:29:54 proxmox kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Feb 17 21:29:54 proxmox kernel: CR2: 00007f02c4d2621c CR3: 0000001ccd010001 CR4: 00000000001726e0
Feb 17 21:29:54 proxmox kernel: Call Trace:
Feb 17 21:29:54 proxmox kernel: <IRQ>
Feb 17 21:29:54 proxmox kernel: ? pfifo_fast_enqueue+0x150/0x150
Feb 17 21:29:54 proxmox kernel: call_timer_fn+0x2b/0x120
Feb 17 21:29:54 proxmox kernel: __run_timers.part.0+0x1db/0x250
Feb 17 21:29:54 proxmox kernel: ? ktime_get+0x46/0xc0
Feb 17 21:29:54 proxmox kernel: ? lapic_next_deadline+0x2c/0x40
Feb 17 21:29:54 proxmox kernel: ? clockevents_program_event+0xa7/0x120
Feb 17 21:29:54 proxmox kernel: run_timer_softirq+0x2a/0x50
Feb 17 21:29:54 proxmox kernel: __do_softirq+0xd9/0x2e6
Feb 17 21:29:54 proxmox kernel: irq_exit_rcu+0x8c/0xb0
Feb 17 21:29:54 proxmox kernel: sysvec_apic_timer_interrupt+0x7c/0x90
Feb 17 21:29:54 proxmox kernel: </IRQ>
Feb 17 21:29:54 proxmox kernel: <TASK>
Feb 17 21:29:54 proxmox kernel: asm_sysvec_apic_timer_interrupt+0x12/0x20
Feb 17 21:29:54 proxmox kernel: RIP: 0010:cpuidle_enter_state+0xd9/0x620
Feb 17 21:29:54 proxmox kernel: Code: 3d 34 02 64 6c e8 b7 23 72 ff 49 89 c7 0f 1f 44 00 00 31 ff e8 98 2f 72 ff 80 7d d0 00 0f 85 5a 01 00 00 fb 66 0f 1f 44 00 00 <45> 85 f6 0f 88 66 01 00 00 4d 63 ee 49 83 fd 09 0f 87 e1 03 00 00
Feb 17 21:29:54 proxmox kernel: RSP: 0018:ffff98540642be38 EFLAGS: 00000246
Feb 17 21:29:54 proxmox kernel: RAX: ffff8989ffdf0f80 RBX: ffffb843ff7f8300 RCX: 0000000000000000
Feb 17 21:29:54 proxmox kernel: RDX: 000000000000007c RSI: 0000000033518eeb RDI: 0000000000000000
Feb 17 21:29:54 proxmox kernel: RBP: ffff98540642be88 R08: 000000e9d8eb993b R09: 0000000000061a80
Feb 17 21:29:54 proxmox kernel: R10: 0000000000000005 R11: 071c71c71c71c71c R12: ffffffff94ed3740
Feb 17 21:29:54 proxmox kernel: R13: 0000000000000004 R14: 0000000000000004 R15: 000000e9d8eb993b
Feb 17 21:29:54 proxmox kernel: ? cpuidle_enter_state+0xc8/0x620
Feb 17 21:29:54 proxmox kernel: cpuidle_enter+0x2e/0x40
Feb 17 21:29:54 proxmox kernel: do_idle+0x209/0x2b0
Feb 17 21:29:54 proxmox kernel: cpu_startup_entry+0x20/0x30
Feb 17 21:29:54 proxmox kernel: start_secondary+0x12a/0x180
Feb 17 21:29:54 proxmox kernel: secondary_startup_64_no_verify+0xc2/0xcb
Feb 17 21:29:54 proxmox kernel: </TASK>
Feb 17 21:29:54 proxmox kernel:
---[ end trace 727774c916316baf ]---
i noticed this happened after i updated the PVE Kernel to (pve-kernel-5.15). i noticed after the upgrade that the overall CPU usage decreased for all the virtual machines and that's good.
so when i run apt install pve-kernel- i can see a lot of newer PVEs but, i'm afraid to upgrade as it may break anything else. So, any idea how to fix this issue?
Thanks in advance
Shady K