[SOLVED] Host incremental backups?
- Thread starter devilkin
- Start date
Well, I'm seeing huge transfer out towards pbs using the following command (even repeated), so I guess i'm going something wrong them :/
PBS_REPOSITORY and PBS_PASSWORD are set in environment
/usr/bin/proxmox-backup-client backup etc.pxar:/etc home.pxar:/homePBS_REPOSITORY and PBS_PASSWORD are set in environment
As far as I understand it the pbs client will create something like a archive that contains all the files/folders. Then this archive will be split into chunks and the hash of each chunk will be computed. Then the pbs client will ask the pbs server if there is already a chunk saved with that hash. If it already exists it will only send the metadata, if the hash doesn't exist it will send the chunk and metadata.
the backup client output will contain summaries of how much deduplication happened for each archive/index.. for host backups it will always have to read all data (to create the archive stream that gets chunked), but will only send chunks which are not part of the previous snapshot.
Would it be possible to implement something like librsync does for file based backup? Having to read each file for each backup run is not possible with a 13TB filesystem.
I think that wont work. PBS needs to read all files to be able to create the hashes that are need for deduplication and to see if that chunk is already stored on the PBS datastore. With rsync it also takes a long time to compare two folders if you enable hashing for file comparison instead of edit time or size. If you don't compute the hashes again and again you can't be sure about data integrity. For that you would need to use dirty bitmapping and that is only available for VMs and not LXCs.
Last edited: