cluster node does not go into maintenance mode

[SteelVortex]

I'm seeing a strange problem after updating a cluster node from 8.4.14 to 9.1.4. When I run ha-manager crm-command node-maintenance enable <NODE-NAME> the server never enters into maintenance mode. The below output is from the problematic node. Also, what is the best method for upgrading cluster nodes? For example, is it preferred to do in place upgrades, or should I migrate the VMs and containers, remove the node from the cluster, do a clean install, and add the server back to the cluster? I'm new to Proxmox clustering, so please let me know if I need to provide more information.


quorum OK
master PV1 (idle, Fri Jan 9 18:54:52 2026)
lrm PV1 (idle, Fri Jan 9 19:14:44 2026)
lrm PV2 (idle, Fri Jan 9 19:14:44 2026)
full cluster state:
{
"lrm_status" : {
"PV1" : {
"mode" : "active",
"results" : {},
"state" : "wait_for_agent_lock",
"timestamp" : 1768007684
},
"PV2" : {
"mode" : "active",
"results" : {},
"state" : "wait_for_agent_lock",
"timestamp" : 1768007684
}
},
"manager_status" : {
"master_node" : "PV1",
"node_request" : {
"PV1" : {},
"PV2" : {}
},
"node_status" : {
"PV1" : "online",
"PV2" : "online"
},
"service_status" : {},
"timestamp" : 1768006492
},
"quorum" : {
"node" : "PV2",
"quorate" : "1"
}
}






Cluster information
-------------------
Name: ProxCluster
Config Version: 3
Transport: knet
Secure auth: on

Quorum information
------------------
Date: Fri Jan 9 19:16:18 2026
Quorum provider: corosync_votequorum
Nodes: 2
Node ID: 0x00000002
Ring ID: 1.c4
Quorate: Yes

Votequorum information
----------------------
Expected votes: 3
Highest expected: 3
Total votes: 3
Quorum: 2
Flags: Quorate Qdevice

Membership information
----------------------
Nodeid Votes Qdevice Name
0x00000001 1 A,V,NMW 192.168.16.1
0x00000002 1 A,V,NMW 192.168.16.2 (local)
0x00000000 1 Qdevice



root@PV2:~# systemctl status pve-ha-crm.service
● pve-ha-crm.service - PVE Cluster HA Resource Manager Daemon
Loaded: loaded (/usr/lib/systemd/system/pve-ha-crm.service; enabled; preset: enabled)
Active: active (running) since Fri 2026-01-09 17:01:57 CST; 2h 16min ago
Invocation: fc4f22040a664e0ab91d2d4cbf796c7f
Main PID: 2639 (pve-ha-crm)
Tasks: 1 (limit: 309351)
Memory: 115.4M (peak: 135M)
CPU: 1.598s
CGroup: /system.slice/pve-ha-crm.service
└─2639 pve-ha-crm

Jan 09 17:01:56 PV2 systemd[1]: Starting pve-ha-crm.service - PVE Cluster HA Resource Manager Daemon...
Jan 09 17:01:57 PV2 pve-ha-crm[2639]: starting server
Jan 09 17:01:57 PV2 pve-ha-crm[2639]: status change startup => wait_for_quorum
Jan 09 17:01:57 PV2 systemd[1]: Started pve-ha-crm.service - PVE Cluster HA Resource Manager Daemon.




root@PV2:~# systemctl status pve-ha-lrm.service
● pve-ha-lrm.service - PVE Local HA Resource Manager Daemon
Loaded: loaded (/usr/lib/systemd/system/pve-ha-lrm.service; enabled; preset: enabled)
Active: active (running) since Fri 2026-01-09 17:02:05 CST; 2h 18min ago
Invocation: 22d01cffd1f74b229629550341542525
Main PID: 2668 (pve-ha-lrm)
Tasks: 1 (limit: 309351)
Memory: 114.5M (peak: 134M)
CPU: 2.066s
CGroup: /system.slice/pve-ha-lrm.service
└─2668 pve-ha-lrm

Jan 09 17:02:04 PV2 systemd[1]: Starting pve-ha-lrm.service - PVE Local HA Resource Manager Daemon...
Jan 09 17:02:05 PV2 pve-ha-lrm[2668]: starting server
Jan 09 17:02:05 PV2 pve-ha-lrm[2668]: status change startup => wait_for_agent_lock
Jan 09 17:02:05 PV2 systemd[1]: Started pve-ha-lrm.service - PVE Local HA Resource Manager Daemon.

 

[SteveITS]

Did you upgrade one or both servers?

They have instructions for https://pve.proxmox.com/wiki/Upgrade_from_8_to_9#In-place_upgrade

 

[SteelVortex]

I planned to upgrade one node at a time. Also, how exactly should I address the qdevice? The device is running on a Raspberry Pi running Debian. I'm basically asking when you upgrade a cluster with a qdevice, is it required to update the corosync package? I ask because I couldn't find any documentation on how to handle clusters with devices. Also, according to the link you shared, it looks like all cluster nodes should be upgraded.

For Clusters​

• Check that all nodes are up and running on the latest package versions.If not, continue the upgrade on the next node, start over at #Prerequisites
• Proxmox VE 9 deprecates HA groups in favor of HA rules. If you are using HA and HA groups, HA groups will be automatically migrated to HA rules once all cluster nodes have been upgraded to Proxmox VE 9.If you encounter any issues with HA groups or rules after having upgraded all cluster nodes, check the logs of pve-ha-crm on the active CRM node for errors with journalctl -eu pve-ha-crm.