Recreate all cluster / node by node

[TwiX]

Hi,

I recently upgraded a 6 nodes Ceph PVE cluster 6.4 to 7
We discovered higher CPU usage, performance issues.

I want to recreate all cluster.
What I plan to do for each node :

- Migrate all VM to another node
- Stop all OSD and then remove them from config
- shutdown the node.
- remove node from cluster
- reinstall node from PVE 7 iso
- reintegrate node from cluster
- reinstall ceph 16
- recreate all OSDs

Is it the right way to do this ?

Thanks !

 

[Stefan_R]

Always recommend playing through your changes in a test environment, but that seems like a reasonable plan for a clean re-install. Make sure to wait for Ceph to create the replicas/balancing between each node.

 

[TwiX]

Thanks seems to be OK on my VirtualBox test environment

I just forgot the following :

- Migrate all VM to another node
- Stop all OSD and then remove them from config
- remove existing ceph monitor/manager
- shutdown the node.
- remove node from cluster
- remove host from ceph config => ceph osd crush rm node_name
- reinstall node from PVE 7 iso
- reintegrate node from cluster
- reinstall ceph 16
- recreate all OSDs
- recreate ceph monitor/manager (You also have to manually edit the ceph.conf to remove the persistent iterations of monitors you want to recreate)