ZFS deduplication for VM data

[scienta]

Hello,

I aim to enable ZFS dedup for my Nextcloud VM, as my users tend to duplicate a lot of data (not high data volumes, but ratio of about 1.4). The VM is stored on a ZFS pool that currently does not have dedup enabled and all Nextcloud data is stored in a single VM disk image. Now, would it be preferable to dedup the dataset on Proxmox for the VM disk image and format it with XFS/EXT4 inside, or should to use nested ZFS and enable dedup inside the VM?

The RAM requirement is of no concern. I have more than enough to spare for this amount of data. I'd rather know that the storage usage will not grow out of hand.

Thanks!

 

[shanreich]

The RAM usage is not the only concern with deduplication. Generally it requires additional work for every I/O operation that can have significant impact on the performance of your zpool.

In general, the downsides are not worth the space savings (particularly when the ratio is as low as 1.4). Storage is quite cheap at this point, it's usually preferable to invest in more disks rather than deal with the downsides of ZFS deduplication.

To quote an article from TrueNAS about ZFS [1]:

In case it wasn't obvious from the information above, enabling dedup in TrueNAS is strongly discouraged. There are almost always far more cost effective ways to store the additional, duplicated data (including just running more disks). Once you've enabled dedup on a dataset, it's not trivial to disable it: the data needs to be moved out of the dedup-enabled dataset and that dataset needs to be destroyed. If your workload needs even a moderate level of performance, dedup will likely result in unacceptably low performance. Because dedup "misses" are so expensive, the performance impact will increase as the achieved dedup ratio drops. iXsystems generally does not recommend even considering dedup unless the dedup ratio is expected to be greater than 5x.

[1] https://jro.io/truenas/openzfs/