Proxmox backup client & incremental backup
- Thread starter m3a2r1
- Start date
at the moment the client has to read all data on each backup, but minimizes the network load, see here for an explanation
https://git.proxmox.com/?p=proxmox-...bf116a2a64999da93e11ace8;hb=refs/heads/master
https://git.proxmox.com/?p=proxmox-...bf116a2a64999da93e11ace8;hb=refs/heads/master
we have no concrete plans on how we could do that... most backup systems (e.g. rsync) use metadata of a file (size + modification time) to reuse data, but with our current architecture that is not possible
(chunks have no direct relation to files; see the link i posted), and even if we could, that could always lead to wrong data backed up... also, there is no feature to track only changed files reliably for file systems..
(chunks have no direct relation to files; see the link i posted), and even if we could, that could always lead to wrong data backed up... also, there is no feature to track only changed files reliably for file systems..
You could do a full backup every week or month using proxmox-backup-client, and combine it with a rsync proces that creates a separate folder with a copy of "changed files in the last X days", and made then also a backup of that using proxmox-backup-client.
So you will have 2 backups and select the most apropiate.
Then you can prune or delete differential (rsync created) copies, when a new full backup is done.
You can use something like that, to create a folder with a copy of files modified in the last day, that then you will backup much faster:
https://unix.stackexchange.com/questions/173958/find-files-newer-than-a-day-and-copy
export DAYS=1
export BASE_DIR=/mnt/data (sample, folder with lot of files and TBs that I full copied previously, and I want to copy incrementally)
export DATE_DIR=PBS_$(date "+%Y-%m-%d")
mkdir -p "$BASE_DIR/$DATE_DIR" (new folder that will contain only a copy of new files)
cd $BASE_DIR
find . -type f -mtime -$DAYS -not \( -path ./$DATE_DIR\* -prune \) -exec rsync --relative {} $BASE_DIR/$DATE_DIR \;
proxmox-backup-client backup sample.pxar:$BASE_DIR/$DATE_DIR --backup-id 'sample-incremental-1day'
So you will have 2 backups and select the most apropiate.
Then you can prune or delete differential (rsync created) copies, when a new full backup is done.
You can use something like that, to create a folder with a copy of files modified in the last day, that then you will backup much faster:
https://unix.stackexchange.com/questions/173958/find-files-newer-than-a-day-and-copy
export DAYS=1
export BASE_DIR=/mnt/data (sample, folder with lot of files and TBs that I full copied previously, and I want to copy incrementally)
export DATE_DIR=PBS_$(date "+%Y-%m-%d")
mkdir -p "$BASE_DIR/$DATE_DIR" (new folder that will contain only a copy of new files)
cd $BASE_DIR
find . -type f -mtime -$DAYS -not \( -path ./$DATE_DIR\* -prune \) -exec rsync --relative {} $BASE_DIR/$DATE_DIR \;
proxmox-backup-client backup sample.pxar:$BASE_DIR/$DATE_DIR --backup-id 'sample-incremental-1day'