Proxmox cluster rebooted when I power cycled a switch

aarcane

Renowned Member
Jul 28, 2015
35
3
73
So uh, I'm sure I know why this happened, but here goes.

I was having some communication issues on one node after a disaster recovery and reinstall. I checked all the config, but VLANs on this VM just wouldn't communicate outside the host. So I decided to attempt to reboot the switch. (Note, this is a home office dev & prod environment, so no redundant LAN yet). When I did, I noticed that one of my pve hosts was also rebooting, so I plugged the monitor into the others, and they were *all* power cycling.

So here's my hypothesis. I think each node thought it lost contact with the other PVE nodes in the cluster, thought it was in a fault state, and attempted to reboot itself to resolve the issue. A watchdog of some sort. A sane default most of the time. The thing is, I didn't set that behaviour up, and I wouldn't want it to happen in the future if I reset the switch for any other reasons in the future.

The questions: Is my assumption correct? Is there any way I can permanently, or preferably, temporarily, disable this feature, short of providing a secondary redundant cluster link?
 
So here's my hypothesis. I think each node thought it lost contact with the other PVE nodes in the cluster, thought it was in a fault state, and attempted to reboot itself to resolve the issue. A watchdog of some sort. A sane default most of the time. The thing is, I didn't set that behaviour up, and I wouldn't want it to happen in the future if I reset the switch for any other reasons in the future.
Do you have HA services? As only then the watchdog is active, at least from PVE site.
https://pve.proxmox.com/pve-docs/chapter-ha-manager.html#ha_manager_fencing
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!