Ceph RBD Namespace Support - a different use case

cpalmer

New Member
Feb 1, 2021
3
0
1
Although there are already a couple of older threads concerning lack of support for Ceph RBD namespaces, I'm posting this as a new thread as our use case is different and potentially simpler to solve.

We have a single external Ceph cluster, and multiple Proxmox installations (some clusters, some standalone), and we want all of the Proxmox installations to be able to use the one Ceph cluster. The problem is that Proxmox generates RBD image names very predictably, which pretty much guarantees name clashes if more than one Proxmox installation is using the same Ceph pool.

We currently have to allocate an entire Ceph pool for each Proxmox installation. This scales very badly, as Ceph pools are heavyweight entities carrying a large amount of runtime overhead. The ideal for us would be to use a single Ceph pool, but to allocate an RBD Namespace within it for each Proxmox instance. The Ceph keys/capabilities would then be arranged such that each Proxmox installation only had access to its own namespace. To meet our requirement it need only be something configured in Proxmox soon after installation, with the namespace not even being visible to the rest of Proxmox - i.e. just a fixed value presented in RBD operations.

(This is different to other use cases that have been mentioned, which wanted multiple RBD namespaces to be available for selection within a single Proxmox installation).

An added benefit appears when moving a virtual disk between Proxmox installations - a suitably privileged Ceph user could simply simply rename the volume across namespaces, without requiring any actual data movement.

Has anyone else tackled this problem? I had a quick look at some of the Proxmox code and while the places Ceph was called did seem constrained to a small-ish number of files, it did seem that each and every call may have to be modified.

Thanks, Chris
 
are you using PVE to manage ceph as well, or an external Ceph cluster not managed with the PVE ceph tooling? to support your use case with the latter should be quite straight-forward (allow setting a namespace when defining a storage, and just passing that through everywhere we talk to Ceph). to support setting up namespaces is more involved (as it would also entail setting up the keys and their permissions accordingly).
 
Hi Fabian

It's the simpler case: the Ceph cluster is managed entirely separately, not with PVE tooling. In our scenario we wouldn't expect (or even want) PVE to have any administrative control over the Ceph cluster at all.

Thanks, Chris
 
maybe file an enhancement request at https://bugzilla.proxmox.com - I think this should be easily do-able as 'advanced' or CLI-only option
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!