Compression & Incremental Backups

[Ramalama]

Hi, i have the following Examples:

Private Example:
- PVE Server Hosted at Hetzner
- Backupserver at Home

Commercial Example:
- A lot of PVE-Servers + PBS at the Companys main Location.
- Company External sub Location has an PVE-Server

In both situations we want to Backup VM's, but the WAN bandwith/speed is limited.

In those situations i need for example Daily Backups, but i want to transfer as little Data over WAN as possible.

1. Do incremental Backups, reads the whole storage and sends the whole disk to PBS, then PBS deduplicated+deletes the data?
- Or does the PVE-Server sends already only the "incremental" Part over WAN?

2. Is the data that goes over WAN already Compressed, or is it Uncompressed and the Compression happens only on PBS via LZ4 ZFS Compression?

Thanks

 

[Dunuin]

1. Do incremental Backups, reads the whole storage and sends the whole disk to PBS, then PBS deduplicated+deletes the data?
- Or does the PVE-Server sends already only the "incremental" Part over WAN?

It only sends the chunks that are missing (so only the new data).

2. Is the data that goes over WAN already Compressed, or is it Uncompressed and the Compression happens only on PBS via LZ4 ZFS Compression?

PBS always uses ZSTD compression.

Private Example:
- PVE Server Hosted at Hetzner
- Backupserver at Home

Commercial Example:
- A lot of PVE-Servers + PBS at the Companys main Location.
- Company External sub Location has an PVE-Server

Ideally you want to local PBS + offsite PBS you sync your backup snapshots to:
a.) local PBS means faster restores so less downtime
b.) backup and live restore not only requires a good bandwidth but also a low latency
c.) ransomware protection

 

[Ramalama]

It only sends the chunks that are missing (so only the new data).


PBS always uses ZSTD compression.


Ideally you want to local PBS + offsite PBS you sync your backup snapshots to:
a.) local PBS means faster restores so less downtime
b.) backup and live restore not only requires a good bandwidth but also a low latency
c.) ransomware protection

Hey Dunuin
As you see im very new to PBS, but glad to see you again after all the time

Thanks for the Answers & for the Tip!
I Thought already of an local PBS+Sync either, but i have almost no Storage at that Crap Server.
In Short i don't have a clue of how Sync works, if its more like an 1:1 mirror, or if i can delete all the Backups with a Prune Job on the local PBS.

But thats something i will find out once i setup it anyway.

Cheers

 

[Dunuin]

In Short i don't have a clue of how Sync works, if its more like an 1:1 mirror, or if i can delete all the Backups with a Prune Job on the local PBS.

You backup to the local PBS only for fast backups + low latency so your running VMs won't be slowed down that much. Especially live restore, where you could already run that VM while it is still restoring to reduce downtime even further, needs that low latency. Both PBS will work independently. You can set some more strict pruning on the local PBS like only keeping backup snapshots for 3 days. For the syncing you tell the offsite PBS to pull the latest backup snapshots from the local PBS via internet or VPN. The offsite PBS got its own pruning settings and could for example keep the backup snapshots for some months/years. And you could tell it to only pull the last X backup snapshots.
You do the restores from the local PBS and in case you will ever lose your local PBS you could directly restore the VMs over the internet/VPN from the offsite PBS.

I would also recommend you read the documentation about ransomware protection: https://pbs.proxmox.com/docs/storage.html#ransomware-protection-recovery