Multisite & CEPH: rbd-mirror and radosgw

[Orchal]

Hi,

I'm trying to set up a CEPH cluster replicated to another CEPH cluster for disaster recovery.

I tried first to set up rbd-mirror which is a lot easier to configure. I managed to get it work but I have the following issue: when I enable exclusive-lock and journaling on any LXC image, the LXC container won't run anymore. Proxmox can't mount the image because of exclusive-lock. I think I know the answer but I'm gonna ask:
- it is working as intended?
- more specifically, is LXC containers could run with exclusive-lock or only KVM machines (I didn't try but I red it works)?
- will I lose HA features if I use exclusive-lock?
However, I noticed that I can run the LXC container then enable the features, but it's not a viable way for me.

On the other hand I'm trying to set up a rados gateway to have a multisite synchronization but I'm new to CEPH and I'm not enough skilled yet to get it work. I don't understand how custom pools can be configured and synced. Actually, nothing works yet...

So the other question is: someone already runs a multisite Proxmox clusters using radosgw? Is it the right way to do even in a small environment or should I stick with vendors storage systems?

Thank you very much!
Orchal

 

[Alwin]

- it is working as intended?

This is an expected behavior.

- more specifically, is LXC containers could run with exclusive-lock or only KVM machines (I didn't try but I red it works)?

ATM, only with KVM.

- will I lose HA features if I use exclusive-lock?

Very likely that the containers will not start, even if the node with the lock is gone.

In general, this is on the list of features. But not yet implemented.
https://bugzilla.proxmox.com/show_bug.cgi?id=1686

 

[Orchal]

HI,

Thank you for your answer. It appears I'm not yet completely stupid haha.

I'm actually working on a rbd export / export-diff backup workflow in order to do the job.
It's not as cool as rbd-mirror or radosgw features but as least it works pretty well.

Orchal