Archiving to external HDD and deduplication

[Fathi]

Hi,
I have a promox ve server, backing to a proxmox backup server with all the benefits of pbs.
I would like to also backup/archive to an external hdd once a week and take that hdd off-site offline.
This VM disk is around 3.2TB of size and could go up to 5.7TB in the future without adding additional disks to pve. As I would like to have ~ 6 months of backups, this is approximately 27 snapshots, so approximately 87TB of external HDD. Biggest one I could find here is 18TB (seaget or WD which also claim their external usb hdd are compatible with windows and palle and not a single word about linux).
How could i achieve deduplication with an external HDD.
I was planing to attach this external hdd to pve. Is it possible to do deduplication in this way or shoud i mount it to pbs, add a new datastore (the external hdd) and then backup to it to achieve this ?
TIA.

 

[LnxBil]

One way to do it is to have ZFS as the backing store of PBS and doing zfs replication to the external disk. You will use the PBS-internal deduplication and may be able to store the data, yet you have to check with time if it really fits. This depends heavily on the data and no one can give you an estimate on YOUR environment.

Just also consinder: deduplication may not be what you want with a backup system: if that one copy of a block that is used in all of your VMs gets corrupt, your backup is becomes useless.

 

[Fathi]

One way to do it is to have ZFS as the backing store of PBS and doing zfs replication to the external disk. You will use the PBS-internal deduplication and may be able to store the data, yet you have to check with time if it really fits. This depends heavily on the data and no one can give you an estimate on YOUR environment.

Just also consinder: deduplication may not be what you want with a backup system: if that one copy of a block that is used in all of your VMs gets corrupt, your backup is becomes useless.

Thank you @LnxBil . Didn't think of the corrupt block on which depends several vm backups that could compromise them all. Thank you again for having pointed that out.