how to reinstall node in cluster Two nodes with quorum device

[auranext]

Hi,

Based on the documentation I understand that I need to remove qdevice before adding node to cluster.
I don t know how to do that in secure manner.
Otherwise I can restore node files like /var/lib/pve-cluster/config.db but I don t know what exact files I need to import and what the exact procedure.

thank you

maxime

 

[aaron]

If I understand you correctly, you have currently have a 2 node cluster + qdevice?

• In that case, first remove the qdevice: pvecm qdevice remove
• Then check the pvecm status confirming that only 2 votes are expected at max
• Move all guests from the node that is to be reinstalled
• Remove the one node following the guide https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_remove_a_cluster_node
• Make sure that there are no leftovers (/etc/pve/nodes/<old node>, in /etc/pve/priv/authorized_keys, ...)
• reinstall the node
• add it to the cluster
• install qdevice packages on the reinstalled node
• do a pvecm qdevice setup again

That way everything should work fine. Should you encounter some issues later, for example in a live migration regarding SSH keys. Run pvecm updatecerts on all nodes.

And of course, having backups of your guests, just in case, is always a good idea

 

[auranext]

thank you for this procedure.
what is worry me is if I remove the quorum device and a node, the resulting node will be isolated and will perhaps shut down because corosync does not have enough votes.
In my experience of Cluster 2nodes in PVE5.3 without qdev I have to set exceptionally corosync expeted vote to "1" before invoque delnode.
Is it useless with PVE6.2 ?

 

[aaron]

Once the node is alone, expected votes should be 1 since it is the only node in the cluster. If you use HA it is a good idea to stop the HA services until everything is back to where you want it

systemctl stop pve-ha-lrm
systemctl stop pve-ha-crm

 

[auranext]

ok it is more clear for me, thank you.

 

[auranext]

hum , sorry I m back (one more time !)
I m testing removing one node and I need a little clarification at this step

• first remove the qdevice: pvecm qdevice remove
• the command need to be executed on all the cluster nodes ? or just the node that is to be reinstalled ?

 

[aaron]

no, should be enough to run on one node. Compare the output of pvecm status before and after. The qdevice will still be listed, but will not have a vote and the overall number of expected votes should also be down.

 

[auranext]

thank you !