updated from 7.4 to 8, since regulary freezing.

[anoo222]

Hello,

I just updated from 7.4 to 8 using the wiki, all went fine.
Before and after the update pve7to8 returned 0 warnings.

Since updating (2 hours ago), proxmox freezed 2 times.
When it freezes, obviously all my services running on it don't work anymore.
I can't ping/ssh to the proxmox server.
The only solution is to gy physically press the reset button.

I've read a little bit on the forum, and saw similar (older) threads where it could be a hardware issue (ie ram).

Note that i'm running this server for several months without any freezings, just now after the update it started.

Anyone any idea what could be the issue?

This is the log from when it crashed last time, not sure if it is relevant and the right part in the log:

2023-07-07T12:11:55.874928+02:00 proxmox kernel: [ 4716.562914] NETDEV WATCHDOG: enp1s0 (r8169): transmit queue 0 timed out 2023-07-07T12:11:55.874929+02:00 proxmox kernel: [ 4716.562927] WARNING: CPU: 5 PID: 0 at net/sched/sch_generic.c:525 dev_watchdog+0x23a/0x250 2023-07-07T12:11:55.874929+02:00 proxmox kernel: [ 4716.562933] Modules linked in: bluetooth ecdh_generic ecc cfg80211 nf_conntrack_netlink xt_nat xfrm_user xfrm_algo xt_addrtype overlay ip6table_na t iptable_nat xt_mark nft_ct nft_fib_ipv4 nft_fib wireguard curve25519_x86_64 libchacha20poly1305 chacha_x86_64 poly1305_x86_64 libcurve25519_generic libchacha ip6_udp_tunnel udp_tunnel nft_chain_na t xt_MASQUERADE nf_nat nft_limit xt_LOG nf_log_syslog xt_limit xt_comment xt_tcpudp xt_state xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat tcp_diag inet_diag veth ebtable_filter ebtables ip_set ip6table_raw iptable_raw ip6table_filter ip6_tables iptable_filter bpfilter nf_tables scsi_transport_iscsi bonding tls softdog sunrpc nfnetlink_log nfnetlink binfmt_misc snd_hda_cod ec_hdmi snd_ctl_led snd_hda_codec_realtek snd_hda_codec_generic snd_sof_pci_intel_cnl snd_sof_intel_hda_common soundwire_intel soundwire_generic_allocation soundwire_cadence snd_sof_intel_hda snd_so f_pci snd_sof_xtensa_dsp snd_sof snd_sof_utils snd_soc_hdac_hda 2023-07-07T12:11:55.874934+02:00 proxmox kernel: [ 4716.562981] snd_hda_ext_core snd_soc_acpi_intel_match snd_soc_acpi soundwire_bus intel_rapl_msr intel_rapl_common snd_soc_core intel_tcc_cooling x86_pkg_temp_thermal snd_compress intel_powerclamp ac97_bus coretemp snd_pcm_dmaengine snd_hda_intel kvm_intel snd_intel_dspcfg snd_intel_sdw_acpi mei_hdcp mei_pxp i915 kvm irqbypass drm_buddy snd_h da_codec crct10dif_pclmul polyval_clmulni ttm polyval_generic ghash_clmulni_intel snd_hda_core sha512_ssse3 drm_display_helper dell_wmi aesni_intel cec snd_hwdep crypto_simd rc_core ledtrig_audio sn d_pcm drm_kms_helper snd_timer cryptd i2c_algo_bit dell_smbios syscopyarea snd sysfillrect rapl dcdbas intel_cstate cmdlinepart sysimgblt soundcore spi_nor dell_wmi_sysman mei_me pcspkr intel_pch_th ermal mtd sparse_keymap dell_wmi_descriptor firmware_attributes_class mei wmi_bmof ee1004 acpi_pad mac_hid zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) vhost _net vhost vhost_iotlb tap drm efi_pstore dmi_sysfs ip_tables 2023-07-07T12:11:55.874935+02:00 proxmox kernel: [ 4716.563047] x_tables autofs4 btrfs blake2b_generic xor raid6_pq simplefb ses enclosure scsi_transport_sas dm_thin_pool dm_persistent_data dm_bio_ prison dm_bufio libcrc32c uas usb_storage crc32_pclmul spi_intel_pci xhci_pci i2c_i801 xhci_pci_renesas spi_intel i2c_smbus ahci r8169 realtek libahci xhci_hcd video wmi 2023-07-07T12:11:55.874935+02:00 proxmox kernel: [ 4716.563066] CPU: 5 PID: 0 Comm: swapper/5 Tainted: P O 6.2.16-3-pve #1 2023-07-07T12:11:55.874935+02:00 proxmox kernel: [ 4716.563068] Hardware name: Dell Inc. OptiPlex 3060/04Y8V0, BIOS 1.15.0 09/16/2021 2023-07-07T12:11:55.874936+02:00 proxmox kernel: [ 4716.563069] RIP: 0010:dev_watchdog+0x23a/0x250 2023-07-07T12:11:55.874936+02:00 proxmox kernel: [ 4716.563071] Code: 00 e9 2b ff ff ff 48 89 df c6 05 8a 6f 7d 01 01 e8 6b 08 f8 ff 44 89 f1 48 89 de 48 c7 c7 58 64 00 9f 48 89 c2 e8 06 ab 30 ff <0 f> 0b e9 1c ff ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 2023-07-07T12:11:55.874937+02:00 proxmox kernel: [ 4716.563073] RSP: 0018:ffffadaa00230e38 EFLAGS: 00010246 2023-07-07T12:11:55.874937+02:00 proxmox kernel: [ 4716.563074] RAX: 0000000000000000 RBX: ffff9c0181c58000 RCX: 0000000000000000 2023-07-07T12:11:55.874937+02:00 proxmox kernel: [ 4716.563076] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 2023-07-07T12:11:55.874938+02:00 proxmox kernel: [ 4716.563077] RBP: ffffadaa00230e68 R08: 0000000000000000 R09: 0000000000000000 2023-07-07T12:11:55.874938+02:00 proxmox kernel: [ 4716.563077] R10: 0000000000000000 R11: 0000000000000000 R12: ffff9c0181c584c8 2023-07-07T12:11:55.874938+02:00 proxmox kernel: [ 4716.563078] R13: ffff9c0181c5841c R14: 0000000000000000 R15: 0000000000000000 2023-07-07T12:11:55.874939+02:00 proxmox kernel: [ 4716.563079] FS: 0000000000000000(0000) GS:ffff9c04ae540000(0000) knlGS:0000000000000000 2023-07-07T12:11:55.874939+02:00 proxmox kernel: [ 4716.563081] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 2023-07-07T12:11:55.874939+02:00 proxmox kernel: [ 4716.563082] CR2: 00007f7956b2c8c8 CR3: 00000003b7e10003 CR4: 00000000003706e0 2023-07-07T12:11:55.874940+02:00 proxmox kernel: [ 4716.563083] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 2023-07-07T12:11:55.874940+02:00 proxmox kernel: [ 4716.563084] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 2023-07-07T12:11:55.874940+02:00 proxmox kernel: [ 4716.563085] Call Trace: 2023-07-07T12:11:55.874941+02:00 proxmox kernel: [ 4716.563086] <IRQ> 2023-07-07T12:11:55.874941+02:00 proxmox kernel: [ 4716.563089] ? __pfx_dev_watchdog+0x10/0x10 2023-07-07T12:11:55.874941+02:00 proxmox kernel: [ 4716.563092] call_timer_fn+0x29/0x160 2023-07-07T12:11:55.874941+02:00 proxmox kernel: [ 4716.563094] ? __pfx_dev_watchdog+0x10/0x10 2023-07-07T12:11:55.874942+02:00 proxmox kernel: [ 4716.563096] __run_timers+0x259/0x310 2023-07-07T12:11:55.874943+02:00 proxmox kernel: [ 4716.563100] run_timer_softirq+0x1d/0x40 2023-07-07T12:11:55.874943+02:00 proxmox kernel: [ 4716.563102] __do_softirq+0xd6/0x346 2023-07-07T12:11:55.874943+02:00 proxmox kernel: [ 4716.563104] ? hrtimer_interrupt+0x11f/0x250 2023-07-07T12:11:55.874943+02:00 proxmox kernel: [ 4716.563107] __irq_exit_rcu+0xa2/0xd0 2023-07-07T12:11:55.874944+02:00 proxmox kernel: [ 4716.563110] irq_exit_rcu+0xe/0x20 2023-07-07T12:11:55.874944+02:00 proxmox kernel: [ 4716.563111] sysvec_apic_timer_interrupt+0x92/0xd0 2023-07-07T12:11:55.874944+02:00 proxmox kernel: [ 4716.563114] </IRQ> 2023-07-07T12:11:55.874945+02:00 proxmox kernel: [ 4716.563115] <TASK> 2023-07-07T12:11:55.874945+02:00 proxmox kernel: [ 4716.563116] asm_sysvec_apic_timer_interrupt+0x1b/0x20 2023-07-07T12:11:55.874945+02:00 proxmox kernel: [ 4716.563117] RIP: 0010:cpuidle_enter_state+0xde/0x6f0 2023-07-07T12:11:55.874946+02:00 proxmox kernel: [ 4716.563120] Code: 2a d7 61 e8 54 7e 4a ff 8b 53 04 49 89 c7 0f 1f 44 00 00 31 ff e8 82 86 49 ff 80 7d d0 00 0f 85 eb 00 00 00 fb 0f 1f 44 00 00 <4 5> 85 f6 0f 88 12 02 00 00 4d 63 ee 49 83 fd 09 0f 87 c7 04 00 00 2023-07-07T12:11:55.874946+02:00 proxmox kernel: [ 4716.563121] RSP: 0018:ffffadaa00127e38 EFLAGS: 00000246 2023-07-07T12:11:55.874946+02:00 proxmox kernel: [ 4716.563122] RAX: 0000000000000000 RBX: ffffcda9ffd40010 RCX: 0000000000000000 2023-07-07T12:11:55.874947+02:00 proxmox kernel: [ 4716.563123] RDX: 0000000000000005 RSI: 0000000000000000 RDI: 0000000000000000 2023-07-07T12:11:55.874947+02:00 proxmox kernel: [ 4716.563124] RBP: ffffadaa00127e88 R08: 0000000000000000 R09: 0000000000000000 2023-07-07T12:11:55.874947+02:00 proxmox kernel: [ 4716.563125] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff9fac33a0 2023-07-07T12:11:55.874948+02:00 proxmox kernel: [ 4716.563126] R13: 0000000000000004 R14: 0000000000000004 R15: 0000044a290e818e 2023-07-07T12:11:55.874948+02:00 proxmox kernel: [ 4716.563129] ? cpuidle_enter_state+0xce/0x6f0 2023-07-07T12:11:55.874948+02:00 proxmox kernel: [ 4716.563132] cpuidle_enter+0x2e/0x50 2023-07-07T12:11:55.874948+02:00 proxmox kernel: [ 4716.563133] do_idle+0x216/0x2a0 2023-07-07T12:11:55.874949+02:00 proxmox kernel: [ 4716.563136] cpu_startup_entry+0x1d/0x20 2023-07-07T12:11:55.874949+02:00 proxmox kernel: [ 4716.563138] start_secondary+0x122/0x160 2023-07-07T12:11:55.874949+02:00 proxmox kernel: [ 4716.563140] secondary_startup_64_no_verify+0xe5/0xeb 2023-07-07T12:11:55.874950+02:00 proxmox kernel: [ 4716.563145] </TASK> 2023-07-07T12:11:55.874950+02:00 proxmox kernel: [ 4716.563145] ---[ end trace 0000000000000000 ]---

 

[Chris]

Hi,
are you by any chance running a r8168/69 NIC? MIght be related to https://forum.proxmox.com/threads/networking-issues-pve8.129742/
Installing the dkms driver solved these issues in that case.

Edit: Fixed typo.

 

[anoo222]

Hi Chris,

Thanks for yr response.
I'm not sure tbh, but i don't think so;

root@proxmox:~# lspci | egrep -i --color 'network|ethernet|wireless|wi-fi' 01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)

 

[Chris]

Hi Chris,

Thanks for yr response.
I'm not sure tbh, but i don't think so;

root@proxmox:~# lspci | egrep -i --color 'network|ethernet|wireless|wi-fi' 01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)

Sorry, there was a typo, the other thread talks about the r8168, so you are affected afaik, see also https://packages.debian.org/sid/r8168-dkms

 

[anoo222]

Thanks,

I'm going to install the R8618-dkms package and observe from there.

For others sake maybe reading this in the future, if i don't come back to this thread, it means installing R8618-dkms package was the fix.
If the issue persist, i will report back.

Thanks again Chris for the fast support.