PVE Cluster with failed node..start vm temporarily from another node

[general07]

Dear Folks

i have a scenario with a cluster of 3 x proxmox 7.3.3 (not HA) with activated replication on zfs pools
given that you have 2 failed nodes (temporarily) what is the proper technique to start a VM from the working node ? (just temporarily)

as i undestand from this link -> https://pve.proxmox.com/wiki/Cluster_Manager
firstly i have to set pvecm expected 1 ..i am wondering: is this only temporary command (will not save changes after reboot)?? ,i dont want to destroy my cluster by running this command
i assume after running this command (given that its safe and temporary command) ,i have to mv /etc/pve/nodes/qemu-server/pve01/XXX.conf /etc/pve/qemu-server/WORKING_NODEPVE and i will start the VM from working node storage (replication is working)..
** i tried to cp conf file from /etc/pve/nodes/qemu-server/pve01/ to working node(without running pvecm expected 1 but its read-only (i cannot write ,pobably because cluster is running)

 

[fiona]

Hi,
I'd really recommend getting one of the two nodes up again to make the cluster quorate. Setting expected votes to 1 should only be used as a last resort (e.g. to remove dead nodes from a cluster if you can't reach quorum otherwise). The filesystem in /etc/pve is shared using corosync, so it needs a quorum to function properly.

 

[general07]

well noted and thanks for clarification .. therefore as i understand if i have 2/3 working nodes(or 1/2 nodes) quorum is working and i can move configuration from /etc/pve/nodes/pveXX/qemu-server/ to another working node without the need to stop corosync ..is that correct?
Setting expected votes to 1 should only be used as a last resort --> well understood , BUT please clarify(for curiosity reasons) if by running this command pvecm expected 1 ,the changes will be permanent or will be lost after reboot.

 

[fiona]

well noted and thanks for clarification .. therefore as i understand if i have 2/3 working nodes(or 1/2 nodes) quorum is working and i can move configuration from /etc/pve/nodes/pveXX/qemu-server/ to another working node without the need to stop corosync ..is that correct?

If you have two nodes, you need both for quorum. Many people use a QDevice for vote support to be able to have quorum even when one node is down.

Setting expected votes to 1 should only be used as a last resort --> well understood , BUT please clarify(for curiosity reasons) if by running this command pvecm expected 1 ,the changes will be permanent or will be lost after reboot.

The number of votes will be reset after reboot. I haven't tested what happens with changes, because again, it should not be done, but what I would guess is:

• If the other two nodes both come up after the remaining one is shut down, they will overrule the remaining one and changes will be lost.
• But if one of the two nodes comes up while the remaining one is running, the changes will not be lost.