Remote backup over ADSL

[Fathi]

Hello,
1) I am willing to backup two containers from a proxmox ve (connected to the internet via a 20Mbit/s FO), to a home proxmox backup server (connected to the internet over an ADSL 8Mbit/s down link and 1Mbit/uplink). Amount of data to backup: uncompressed 250~300 Gb with small amount of data changing every day.
2) I could do the first backup on site.
When speaking about deduplication of proxmox backup server: a) does this happen on the pbs, so all data is sent from pve over the network but only "new data" is stored on pbs or b) "new data" is identified on pve and only that data is sent over the network to pbs ?
This is very important to know for how to plan backups. If case b), I cloud do daily backups with such a poor network connection. If case a), I have to plan weekly backups to be able to send all data over the network or maybe find another solution.
TIA

 

[fabian]

kind of both if a previous snapshot exists, the index/indices will be downloaded, and any chunks referenced there that would be generated by the current backup will not be uploaded a second time (X). additionally, the PBS server will not store a chunk twice, so if a chunk is uploaded that already exists on the server, it will be discarded and the old copy will be referenced by the new backup snapshot (Y). additionally, any chunks occuring multiple times withing a single backup snapshot (usually the "zero chunk" consisting of all zero bytes) will be deduplicated by the client already and only uploaded once (or not at all, if (X) is the case ) - (Z).

now, Z is mainly relevant for VM backups as chunks are a fixed size there so the likelyhood of a chunk repeating is bigger if your data is repetitive. Y will happen mostly if you have many similar backup groups (e.g., lots of guests created from the same templates). X can only happen for a new snapshot in an existing group, so the first backup creating a group can never benefit from it (this is where your onsite backup will help).

 

[leesteken]

Maybe I can give you an idea about the time your backups might take after the first: My backups to a remote PBS over a 10Mbit internet uplink have a reuse of 85-95%. Small containers (1G actual data) take half a minute, small service VMs (6G disk) take two minutes and a desktop VM (100G+) takes half an hour. Since the backups are off-site for disaster recovery only, and a very busy uplink reduces the downlink speed, I only do this weekly.