if one node looses qorum for seconds all other nodes show red lights

felipe

Well-Known Member
Oct 28, 2013
222
6
58
since some time during backups it happenes that a node looses qorum. (searching for why is another task)
the node (which lost qorum) itself afterwards is the only node which show all other nodes in green and with data.
i checked that all nodes have qorum at this time.
running a /etc/init.d/pvestatd restart helps to get green light for the other nodes (on the other nodes).

why do all other nodes except the node which lost qorum for some seconds get stuck?
 
since some time during backups it happenes that a node looses qorum. (searching for why is another task)
the node (which lost qorum) itself afterwards is the only node which show all other nodes in green and with data.
i checked that all nodes have qorum at this time.
running a /etc/init.d/pvestatd restart helps to get green light for the other nodes (on the other nodes).

why do all other nodes except the node which lost qorum for some seconds get stuck?

Usually you have red lights at the node which cannot be reached for the moment by cluster communication network. These lights are independent from Quorum (if a Cluster is split into 2 parts only one part can have Quorum - but when you connect to portal into the part without quorum you will see green lights for the nodes in this parts where the others will be red).

After connection works again in the whole cluster all will be green again. No daemon restart necessary in a normal case.

Note that the reaction time are some seconds since there are timeouts until cluster communication is considered as "failed". When the timeout is very high you may never get red lights ....
 
Although not very frequently, i have seen this also many times during backup. Once i separated backup on a dedicated NIC this issue somewhat went away. Also another way is to reduce the backup bandwidth (in kbps) in vzdump.conf if both backup and cluster traffic shares same NIC.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!