Proxmox & deduplication?

[ncage]

Right now my backups are not encrypted. I would assume deduplication works between different vm/containers? So if you have a file on multiple containers/vms it would only be stored once?

Which leads to my question....i'm thinking on encrypting my backups. If i use the same encryption key for all vm/containers will deduplication still be able to do this? Granted if there was a different encryption key i understand it wouldn't.

 

[aaron]

Yep, deduplication works within a datastore and also within the same encryption key. The same data, ecrypted with the same key, will result in the same chunk sent to the PBS.
Different key and the chunk sent to the PBS is different, as you expect

 

[Dunuin]

Right now my backups are not encrypted. I would assume deduplication works between different vm/containers? So if you have a file on multiple containers/vms it would only be stored once?

In theory yes, but in practice its not that easy. For VMs, PVE will chop that virtual disk in 4MB chunks, compress them, encrypt them and send them to the PBS. Only if every bit of those 4MB is identical it will be deduplicated. So won't help that much for small files. And then not all files are aligned to 4MB. Lets say you got a 5MB file "ABCDE" with some random data "OOO". Each letter A, B, ..., O is 1 MB of sectors:
VM1: "OABCDEOO" gets split into chunks "OABC" + "DEOO"
VM2: "OOABCDEO" gets split into chunks "OOAB" + "CDEO"

While both VMs store the same file, the resulting 4 chunks that get deduplicated will be all different and aren't deduplicatable.

So it's more useful in case you store multiple backups of the same VM or if you are working with clones created from a template.