convert existing CEPH cluster to Proxmox?

[jamin.collins]

As the subject states, is there a way to move an existing CEPH cluster to Proxmox?

I have an existing cluster of 5 storage OSD nodes, 3 monitors, and a few virtualization hosts that I'd like to convert to proxmox.

 

[alexskysilk]

there is probably a way to do this, but its full of sharp edges you will most likely get cut on. I would advise to back up all assets on the cluster, and reinstall everything from scratch. cumbersome, but safe.

Alternatively, leave the existing cluster alone and attach a separate proxmox cluster to it.

 

[jamin.collins]

I'm not afraid of sharp edges, I've been manually managing and recovering my CEPH cluster for several years now. I'd rather not have the extended downtime that dumping everything out of the cluster (~50TB) and importing it would entail.

 

[alexskysilk]

Maybe I wasnt clear.

sharp edges include TOTAL LOSS OF DATA. does that change anything?

 

[jamin.collins]

I've got nightly backups that I can retrieve the data from. I'd just prefer to assimilate the existing cluster and minimize the downtime. Ideally by converting each node from their existing OS to Proxmox.

But it's looking like I'll need to take the entire cluster offline while rebuilding each host to Proxmox and then reimport all the data. Seems like a day (or several) in overall downtime. I'm sure I'll have some of the hosts back up within an hour or so, but importing 50TB of data will take time.

It's sad there isn't a route to assimilate an existing CEPH cluster.

 

[Darkk]

Technically under the hood a CEPH host is the same for ProxMox just with different tools installed. I actually have thought about using three beefy servers with lots of disks just for pure CEPH without ProxMox installed for performance reasons. I've had bad performance issues whenever CEPH cluster is rebalancing due to failed OSD or node and the VMs started to stall. This is with CEPH/Proxmox combo on 7 nodes in a cluster.

Depending on how many VMs and physical host servers it might make sense to keep things separate. If you only have like 3 or 5 servers then it makes sense to do the CEPH/Proxmox combo. Just make sure you have a minimum 10 gig network for CEPH traffic.

 

[jamin.collins]

Mine is a home cluster and I'm using dedicated 2.5G networks for OSD and Client traffic. I know that constrains my overall performance, but I've been using CEPH since 2014. It's mainly the live migration and self healing that I'm after. I've even managed to tweak the CEPH recovery so it's not terrible for my VM performance when it happens.

 

[ibravo]

Wouldn't this be a matter of having the same configuration files between the Proxmox/Ceph Hosts and the pure Ceph cluster?
I am too, looking for a way to get some hosts dedicated purely to Ceph and not be Hyperconverged.

 

[alexskysilk]

looking for a way to get some hosts dedicated purely to Ceph and not be Hyperconverged.

You dont need to do anything special to achieve that. just remove it from your ha-groups and dont assign any vms there manually.