Fencing and "isolation response"

I was wondering how fencing works in detail.
I've never experienced real problems with Proxmox fencing thanks to the watchdog mechanism.
Nevertheless how does it works exactly ?

For instance : a cluster with 3 nodes running in HA, VM100 is running on host1.
Host1 stop responding on the corosync network but the VMs working on it continues to run (they're on another network and runs normally).
To my mind, in that case, the HA cluster will move VM100 on host 2 or host 3 but the VM100 is still running.
In some others platforms, there is a control on the locks of the virtual hard drive of the VM to prevent that kind of problems.

What about Proxmox ?
Proxmox will self-fence the isolated host : host1 will be forcibly rebooted, so VM100 is for sure not running anymore. While host1 is rebooting, VM100 will be relocated to either host2 or host3
each node with HA active has to regularly write a timestamp to pmxcfs (/etc/pve). this will fail if a node is not part of the quorate partition of the cluster. the watchdog on each node monitors this timestamp, if it is not updated for a certain amount of time, the watchdog expires and the node is self-fenced. the CRM master (running on one node on the quorate partition) will monitor the timestamps as well, notice that one is expired, and will redistribute the HA resources belonging to that node accordingly. on the next boot of the fenced node, it will first sync up with the rest of the cluster, then it will see that all the resources went elsewhere and not be able to start a VM running on another node. the CRM will notice the node coming back up, and depending on HA settings, distribute HA resources to it again.


The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!