Replication on CEPH RBD - are we going to see this soon?

[DC-CA1]

Hi , Proxmox is as killer hypervisor for competition , everybody know that but one of the most important feature that we lack for is REPLICATION of virtual machine on other file system than ZFS, why is this still not available ?

we have alot of VM we cant move from HYPER-V and Vmware because of that.

1= why it take so long to get this feature
2= are we going to see something experimental like we have now for QM-REMOTE-MIGRATE that work like a charm anytime soon ?

we can of course create a Rsync of /pve vms on another cluster , and then create a CEPH RBD snapshot ( if it exist , im not sure ) but far from been desired ...

 

[Whatever]

You can follow this tutorial with some changes (journaling -> snapshot) with some help from posts on this forum (try search).
We've managed to set it up. It works no really smoothly but works

https://pve.proxmox.com/wiki/Ceph_RBD_Mirroring

 

[DC-CA1]

Hi is there anybody concerned about not having Replication available on anything else than ZFS Local ?

@fabian can we hope to see this soon ?

 

[Whatever]

Seems this feature is not often requested and quite tricky to implement and maintain

 

[fabian]

Taking a look at snapshot-based RBD replication (instead of the rbd-mirror variant) is definitely on my TODO list (in the context of remote migration especially). What also is requested from time to time is lifting the restriction that the target storage has to be named the same as the source one - e.g., supporting a storage map like for migration. That wouldn't be too hard to implement per se, but would make recovery in case of a node failure harder (because all the volume IDs would be wrong in the config). RBD replication within a cluster doesn't make that much sense IMHO, but if it's easy to add I wouldn't mind either (it would require that storage map feature first though, since obviously local replication from Ceph to Ceph only makes sense if the two Ceph pools are different and not one and the same). Other storages usually lack a nice export/import mechanism that allows sane incremental sending, and there also isn't really that much of an advantage to supporting multiple local storages when there already is one with good built-in support

There are other rough edges in the remote migration context that are definitely more pressing though:
- adding regular export/import to shared storages that are currently lacking it (e.g., the RBD plugin )
- handling snapshots
- handling pending changes
- improving error handling
- finding and improving other edge cases
- ...

 

[spirit]

Personnaly, I was a lot of customers with the need of disaster recovery, with 2 differents cluster on different datacenter and storage mirroring.
(they can't do 1 stretched cluster because they don't have a 3rd datacenter for quorum, and also latencies can high between the 2 DC)


Currently, I'm doing it manually with custom scripts for rbd|zfs snaphot send|receiv + rsync of vm config.