ZFS dedup is not deduplicating the vm images

[jagan]

Hi All,

Please, can anyone help me out to set up ZFS dedup for my qemu VM images?

I am testing ZFS deduplication in PvE 6.0, I have a ZFS pool with a single (Raid0) 240GB SSD and mounted it as ZFS storage in web GUI.

I have installed a windows VM on this ZFS storage with a size of 30GB. When I clone this VM to the same pool the cloned VM also is consuming 30GB space.

As per deduplication, since both .raw files of 2 VMs are having same data it supposes to consume only 30GB right? but here it is consuming 30+30 60GB space in the ZFS pool.

In Proxmox the dedup works in VM images level ? or data level inside the guest?

 

[aaron]

By default dedup is not enabled because it needs a lot of resources.

Have you enabled it on the ZFS storage?

Also look into enabling compression. This way depending on what your data looks like you can still get some nice space savings without the big cost in resources needed for it.

 

[jagan]

Hi,

Dedup has been enabled please check the screenshot. It is showing 90GB used for 3 similar VMs (each config. with 30GB).




Do I have to config. zfs storage in the thin provision or thick provision to enable dedupe?

 

[aaron]

Ah, The used column shows the full size if you did not enable thin provision. The whole size of the disk is reserved in that case. The actual usage is in the Refer column.

Without any experience with dedup myself there are two things that I see. First the DEDUP column which shows that there are a lot of blocks shared / deduplicated.

Second: if you look at the zfs list output and there in the AVAIL column. For some reason the disks themselves have pretty much exactly the 30G more available. Now I don't know if that means anything because I don't have any experience with dedup nor do I know anyone who has. You can try to clone the VM a third time and see how it behaves. Also try to diverge the data of a VM by changing things so it's data cannot be deduplicated anymore.

TL;DR:
Dedup works but the reserved space for the not thin provisioned VM is still shown. Used column shows how much space is reserved. Refer column shows actual usage. Dedup is saving how much? I don't really know, not much experience in that regard.