Proxmox server occasionally loses connection and disappears from ARP and DHCP on the router

[Nixellion]

Hi!

I started having a weird issue where one of two Proxmox servers I have stops responding to pings and loses connection. It disappears from ARP and DHCP on the router (Mikrotik RB2011). All VMs and CTs also stop responding to pings and disappear from IP reservation lists. However they still work because my home automation that's not operating through LAN (Zigbee, ZWave) keeps working. Router shows ether4 port as advertising 100mbps connection both router and client, though it's a gigabit connection when its working.

This current setup with this router worked since September and only started having this issue like maybe a week or two ago, and it happened about 3 times already.

If I disable and enable ether4 connection on the router then I can ping Proxmox host and connecting to it's web UI. Also VMs and CTs that have static IP set in Proxmox work. But those that rely on DHCP dont.

Router reboot solved it once. But router reboot is not a solution even on consumer routers, but especially on Mikrotik where it should kind of work 24\7\365. And still it did not work this time, I had to reboot Proxmox host as well.

What may be the problem? Is it router? Cables? Proxmox? Where should I look for any additional clues?

Thank you.

 

[Stoiko Ivanov]

What may be the problem? Is it router? Cables? Proxmox? Where should I look for any additional clues?

check the journal of your PVE-node (`journalctl -b`) - that should tell you if the node sees some problems

 

[Nixellion]

check the journal of your PVE-node (`journalctl -b`) - that should tell you if the node sees some problems

Will do if\when it happens again. However do I have to do it when it's down or will it show problems after reconnecting? I don't have a monitor attached to the server, can't check it at that exact moment, only after re-enabling link to the router (from the router's settings), which helps.

 

[Stoiko Ivanov]

Will do if\when it happens again. However do I have to do it when it's down or will it show problems after reconnecting?

the journal should be present at least since the last reboot/reset - you can (and imho should) enable persistent journalling to keep the journal across reboots. [0]

The one exception that happens every now and then is that when a host is hard-reset the last few entries in the journal are lost because they were not written to disk (sadly those last lines are often where the relevant information is)

in any case - simply try logging in via ssh after the next link-loss and run `journalctl -r` - you should see whether there is anything (relevant) in the journal

[0] usually done with `mkdir /var/log/journal && systemctl restart systemd-journald - see https://www.freedesktop.org/software/systemd/man/journald.conf.html