PBR Remote Sync - Limiting the sync

Apr 24, 2020
22
4
8
28
Our main backup server has 20+TBs of available storage (our colo server), while our offsite only has 12TB (public bare metal cloud).
Since I can't easily get additional storage at my offsite location and I only need to offsite store a smaller set of last-resort backups (say, the 5 most recent and 2 annual), is there a way to only sync the 5 most recent backups per VM, for example?

I can set a more aggressive prune policy at my offsite location to keep the storage usage in line, but as I understand it, every time it tries to pull the primary datastore it will pull it's entire contents, event the backups which would be immediately pruned at the destination.
 
Apr 24, 2020
22
4
8
28
One method I was considering (but it means I lose some de-duplication) is to have a 'Target' datastore and a 'Deep Archive' datastore at the primary site.
PVEs backup to the target, which keeps the last 2-5, then does a local sync to Deep Archive. then offsite runs a daily sync job to pull from Target.

Anyone have experience with this kind of setup?
 

dietmar

Proxmox Staff Member
Staff member
Apr 28, 2005
17,137
533
133
Austria
www.proxmox.com
I can set a more aggressive prune policy at my offsite location to keep the storage usage in line, but as I understand it, every time it tries to pull the primary datastore it will pull it's entire contents, event the backups which would be immediately pruned at the destination.
No, it only sync backups newer than the last local backup.
 

dietmar

Proxmox Staff Member
Staff member
Apr 28, 2005
17,137
533
133
Austria
www.proxmox.com
Oh! that's excellent news - but how would I get the initial sync (I.E. copy the latest set of backups) to the datastore, if a full sync will fill the destination drives? @dietmar
We are working on such feature ...
 
Apr 24, 2020
22
4
8
28
perhaps I could fool it by copying some of these directories? even if the verifications fail and I delete them afterwards, would that serve as a starting point for a given vm?
1632503099389.png
 
Apr 24, 2020
22
4
8
28
For any future users, I did in fact SSH from my off-site server to my primary backup server, copied the /vm and /ct folders in the datastore to my new server, setup a more aggressive prune rule and then started replication.

It starts transferring only the newest images, and once you have a few transferred over, kick off a verification task on the datastore. It will mark as failed any of the 'fake' folders you copied over via ssh, and then you can delete the old items.
 
  • Like
Reactions: Sourcenux

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!