Building a separate ceph storage cluster

Sep 20, 2012
117
3
43
We're evaluating different shared storage and we're contemplating using ceph. We've tried this in the past with a hyper- converged set up and it didn't go so well so we're wanting to build a separate ceph cluster to integrate with our proxmox cluster. We've read the benchmark document and have an idea of the best hardware/network but we are wondering what we should use to build the cluster?

  1. Should we just build the cluster using proxmox nodes with no VM's on it and use the GUI?
  2. Should we try to build it by hand using something like ceph-deploy? I am a little leery of that route because we are not ceph experts by any stretch of the imagination.
  3. Should we look to a vendor to build a ceph cluster for us and provide support?
  4. Something other option?
I am just looking for advice. I appreciate it!


P.S. Forgot to mention, we are looking for 50TB of usable storage to begin with and scale up to 100TB in 6 months. Over the next year or two we'll probably want to be at 150-200TB.

Best regards,
Eric
 
Last edited:
The easiest solution would probably be to use PVE for the setup.
With Basic or higher subscriptions, you will be able to get support via my.proxmox.com, should there be any issue.

If it is part of a bigger cluster, you can configure it so that no VMs/CTs can be created on the Ceph nodes.
If it is a standalone cluster, you can still manage it easily through the Web GUI.


As to other solutions, I don't have experience with those. But there might be some nice solutions by other providers. This of course means no support from our side for the Ceph cluster.
 
For OP:
4. Rook or any configuration management solutions

Anyway, we are calculation external Ceph storage for our PVE too.
PVE Staff, are there some requirements for external clusters? For example, version difference etc? PVE documentation handles hyperconverged/updates mainly.
 
By default on PVE 7 the Debian Ceph packages are used. Which are 14.2 in bullseye. So anything that is compatible with that version.
If you require support for newer versions, you can always add our Ceph repositories and run apt update and apt full-upgrade to get Octopus or Pacific versions of the already installed Ceph packages.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!