3 Day Snapshot Retention for Ceph RBD Storage

[cph-cvm]

We are still investigating if Ceph will be good for our production storage. We love the high availability and speed of Ceph, but there are certain aspects that we miss from VMWare and iSCSI storage.

One of them being snapping a storage pool. We know this would be possible with Ceph if Proxmox supported CephFS as VM Storage. We also understand the underlying issue with Ceph where if we were to do temporary snapshots for a VM with RBD disks, we should definitely not snapshot the storage pool as well, as there could be issues.

What I am looking to find out is if we were to setup a script to schedule snapshots with the Proxmox API, and keep a retention of 3 for 3 days, would there be any significant issues? From my quick testing the performance difference does not seem to be much different, but I am mostly worried about Proxmox Backup and the Fleecing option, as we use this option for all of our backups.

Any insight would be greatly appreciated.

 

[waltar]

You can mount your cephfs on all nodes and define in datacenter storage dir shared to reach that. Do test vm's and lxc's, take multiple snapshots, backup's and even test restores.

 

[cph-cvm]

I did not know this was possible @waltar, I will give this a shot as well. I just got it mounted, thanks @waltar .

 

[cph-cvm]

Ended up not working, the snaps of the running VM's would break the snapped vm disk, would not boot correctly for Ubuntu Linux or Windows 11, legacy and UEFI.

 

[kellogs]

how about using PBS for your regular backups and then setup another ceph cluster for https://pve.proxmox.com/wiki/Ceph_RBD_Mirroring ?

I am going to setup a DR Cluster based on the link. so in the event there is a total disaster on the main DC, i have something to fall back

 

[waltar]

Hey @cph-cvm, try instead for vm "qm suspend <vmid>", do ceph snapshot (see manual) of and "qm resume <vmid>" after, for lxc do analog "lxc-freeze -n <vmis>" and "lxc-unfreeze -n <vmid>" !!

 

[alexskysilk]

Ceph if Proxmox supported CephFS as VM Storage.

you CAN do that, but its really not a good idea when RBD exists.

try instead for vm "qm suspend <vmid>", do ceph snapshot (see manual) of and "qm resume <vmid>" after,

When using the pve api, that is the sequence that happens; @cph-cvm did you issue the snapshots from the proxmox interface? if so, post the task log.