RFE: Replication feature for Ceph

[kellogs]

Currently if user is using ZFS we could have access to the replication feature which is fantastic in term of RTO/RPO

Are we able to have this replication feature available for Ceph user please?

So we can replicate this across to another cluster and on its own storage for disaster recovery scenario

 

[cheiss]

Hi,

you are probably looking for Ceph RBD Mirroring? This should probably do what you want.

 

[kellogs]

hello @cheiss

yes i was looking at the Ceph RBD Mirroring as well but to this https://forum.proxmox.com/threads/ceph-rbd-mirror-force-promote.160339/#post-758050 it is not as good as it seems to be.

 

[aaron]

Since the VM replication feature based on ZFS is done within one cluster, and Ceph is a shared storage within one cluster, the direct equivalent is not necessary. In the context of the Datacenter Manager, we do plan to have site-replication and (manual) fail over between different Proxmox VE clusters on the roadmap.

Off-site replication copies of guest for manual recovery on DC failure (not HA!)

https://pve.proxmox.com/wiki/Proxmox_Datacenter_Manager_Roadmap

 

[kellogs]

Hello @aaron

Are we able to have near time for site replication or at least few minutes? If we could do that it would be a killer feature. As of now seems like the product which can do near time VM replication is Zerto but unfortunately it does not support proxmox.

Also i have tried to use PBS to backup every 1 hour but i noticed that everywhere when it is doing backup, the backup VM is frozen.

 

[_gabriel]

Also i have tried to use PBS to backup every 1 hour but i noticed that everywhere when it is doing backup, the backup VM is frozen

it's not excepted.
is PBS local to PVE ?
is PBS have fast storage ?
PBS + qemu incremental thanks to bitmap is really fast.
But guest can slowdown during backup, if high IO load like many writes or SQL on Windows, because qemu do an embedded snapshot which froze guest filesystem to get consistency data.
There is fleecing option to mitigate when fast local storage available.
Check backup log to find how many data transfered.
The PBS drawback is the not speedy restore, but mitigate with Live Restore.

 

[kellogs]

PBS is local to PVE on very fast network (100G)
PBS is on HDD raid 6.

What is the disk setup on your PBS?

 

[_gabriel]

PBS is on HDD raid 6

seems the bottleneck.
PBS does not backup sequential.
PBS does not read sequential too.
As all data are dedup to chunks.

What is the disk setup on your PBS?

I install PBS alongside PVE.
I have dozen at customers.
Mainly 1 HDD as datastore because with 1-2 VM mainly Windows.
Flash as datastore for critical one.
+ Remote Sync to HDD or external.

 

[kellogs]

actually my mistake. It is HDD in RAID10