Moving a node with Ceph from one cluster to a different one

Paspao

Active Member
Aug 1, 2017
69
2
28
55
Hello,

I decided to redistribute my cluster nodes in a different way, so I want to remove one node from one cluster to another.

Which is the correct sequence to remove the node from Ceph cluster?
Docs only describe node addition and not removal. Can it be done just in GUI?

After node removal do you suggest a complete Proxmox reinstall or I may reliably add it to another cluster after removing osds and ceph configuration?

Thank you.
P
 
Which is the correct sequence to remove the node from Ceph cluster?
To remove a node from a hyper-converged cluster, first the ceph services need to be removed. Once the ceph cluster is healthy again, the PVE node needs to be removed from the cluster itself.

Docs only describe node addition and not removal. Can it be done just in GUI?
True, the docs need updating. ;) Not all step can be done on the GUI (PVE 5.x). :/

With the pveceph tool you can destroy all the configured Ceph services (eg. MON, MGR, MDS, OSD). Then best, disconnect the host and run through the doc - 'remove a cluster node'. The current documentation for your Proxmox VE version can be found on the upper right on the Proxmox VE GUI.
https://your-node/pve-docs/chapter-pvecm.html#_remove_a_cluster_node

After node removal do you suggest a complete Proxmox reinstall or I may reliably add it to another cluster after removing osds and ceph configuration?
Possible but better re-install the host, as there might be many config artifacts that you might not want to take over.
 
  • Like
Reactions: Paspao
Thank you. It worked.

I stopped OSDs and removed them.

Then I zapped them to bring disk at default status:

ceph-volume lvm zap /dev/sdX --destroy

Then I stopped and removed MON from GUI (seems to remove MGR too).

I finally removed node with pvecm delnode NODE-NAME

Only one thing :
I still see node (marked red) in GUI , but it is not listed in the cluster nodes, even after re-logging in and a service pveproxy restart.

Thanks.
P.
 
I still see node (marked red) in GUI , but it is not listed in the cluster nodes, even after re-logging in and a service pveproxy restart.
The directory '/etc/pve/nodes/<node_name>' still exists. If there is no needed content in that folder anymore, then you can remove the directory.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!