Although there are already a couple of older threads concerning lack of support for Ceph RBD namespaces, I'm posting this as a new thread as our use case is different and potentially simpler to solve.
We have a single external Ceph cluster, and multiple Proxmox installations (some clusters, some standalone), and we want all of the Proxmox installations to be able to use the one Ceph cluster. The problem is that Proxmox generates RBD image names very predictably, which pretty much guarantees name clashes if more than one Proxmox installation is using the same Ceph pool.
We currently have to allocate an entire Ceph pool for each Proxmox installation. This scales very badly, as Ceph pools are heavyweight entities carrying a large amount of runtime overhead. The ideal for us would be to use a single Ceph pool, but to allocate an RBD Namespace within it for each Proxmox instance. The Ceph keys/capabilities would then be arranged such that each Proxmox installation only had access to its own namespace. To meet our requirement it need only be something configured in Proxmox soon after installation, with the namespace not even being visible to the rest of Proxmox - i.e. just a fixed value presented in RBD operations.
(This is different to other use cases that have been mentioned, which wanted multiple RBD namespaces to be available for selection within a single Proxmox installation).
An added benefit appears when moving a virtual disk between Proxmox installations - a suitably privileged Ceph user could simply simply rename the volume across namespaces, without requiring any actual data movement.
Has anyone else tackled this problem? I had a quick look at some of the Proxmox code and while the places Ceph was called did seem constrained to a small-ish number of files, it did seem that each and every call may have to be modified.
Thanks, Chris
We have a single external Ceph cluster, and multiple Proxmox installations (some clusters, some standalone), and we want all of the Proxmox installations to be able to use the one Ceph cluster. The problem is that Proxmox generates RBD image names very predictably, which pretty much guarantees name clashes if more than one Proxmox installation is using the same Ceph pool.
We currently have to allocate an entire Ceph pool for each Proxmox installation. This scales very badly, as Ceph pools are heavyweight entities carrying a large amount of runtime overhead. The ideal for us would be to use a single Ceph pool, but to allocate an RBD Namespace within it for each Proxmox instance. The Ceph keys/capabilities would then be arranged such that each Proxmox installation only had access to its own namespace. To meet our requirement it need only be something configured in Proxmox soon after installation, with the namespace not even being visible to the rest of Proxmox - i.e. just a fixed value presented in RBD operations.
(This is different to other use cases that have been mentioned, which wanted multiple RBD namespaces to be available for selection within a single Proxmox installation).
An added benefit appears when moving a virtual disk between Proxmox installations - a suitably privileged Ceph user could simply simply rename the volume across namespaces, without requiring any actual data movement.
Has anyone else tackled this problem? I had a quick look at some of the Proxmox code and while the places Ceph was called did seem constrained to a small-ish number of files, it did seem that each and every call may have to be modified.
Thanks, Chris
