PVE 9.1.11: HA CRM stuck in wait_for_quorum despite corosync quorum OK

[reshma]

Environment:
- Proxmox VE 9.1.11 (Debian 13.4 Trixie), kernel 7.0.2-4-pve
- 3-node cluster, non-enterprise repositories
- pve-ha-manager 5.2.4, libpve-cluster-perl 9.1.6, libpve-rs-perl 0.15.3 (aligned on all nodes)
- corosync 3.1.10-pve2, watchdog configured (softdog)

Problem:
pve-ha-crm starts successfully but immediately enters "wait_for_quorum" state and never progresses. This has persisted for several days across multiple restarts/reboots.

Evidence:
- Corosync shows quorum: `corosync-quorumtool` reports 3/3 nodes, quorate: Yes
- ha-manager status: "quorum OK", no master elected, no LRMs showing
- corosync-cpgtool: Shows only pve_kvstore_v1 and pve_dcdb_v1 groups - NO HA CPG groups (pve_ha_crm, pve_ha_lrm missing)
- strace on pve-ha-crm: Shows 5-second loop checking /etc/pve/local, sending watchdog signals, never progressing
- journalctl: Only shows "status change startup => wait_for_quorum" - no errors, no failures
- PVE_HA_DEBUG=1: Produces no additional output

All nodes show identical behavior. /etc/pve/ha/ contains only empty crm_commands and resources.cfg files.

Is this a known issue with PVE 9.1.11 or pve-ha-manager 5.2.4? What condition is CRM checking that's failing silently?

 

[dakralex]

Hi!

- ha-manager status: "quorum OK", no master elected, no LRMs showing
[ ... ]
- strace on pve-ha-crm: Shows 5-second loop checking /etc/pve/local, sending watchdog signals, never progressing
- journalctl: Only shows "status change startup => wait_for_quorum" - no errors, no failures

This would be the expected behavior if there are no and have never been any active HA resource present (i.e. running) in the cluster. The CRM will only transition into an active state as soon as there are any active HA resource present, the same for the LRM but only if there are any active HA resource present on its node.

- corosync-cpgtool: Shows only pve_kvstore_v1 and pve_dcdb_v1 groups - NO HA CPG groups (pve_ha_crm, pve_ha_lrm missing)
[ ... ]
- PVE_HA_DEBUG=1: Produces no additional output

The HA stack does not create its own Corosync CPG groups.

Additionally, there is no PVE_HA_DEBUG environment variable, which enables/disables any additional functionality or output.

 

[kfietz]

All nodes show identical behavior. /etc/pve/ha/ contains only empty crm_commands and resources.cfg files.

Did I get that right, you have no HA Resources configured at all?

That would explain why all services stay idle. Nothing to do.
You probably configured a HA Resource once and removed it afterwards. This would explain the current behavior.

Or did the resources just disappear?

 

[reshma]

yes I have removed the nodes from HA as I couldn't add 3rd node and that is when I'm not able to bring back nodes to HA. But Thanks, it worked now when I added the VM's from all the nodes to HA resources and then it worked. I could see all 3 nodes part of HA. Thanks !