ZFS dedup is not deduplicating the vm images

jagan

Member
Feb 11, 2015
88
0
6
india,chennai
Hi All,

Please, can anyone help me out to set up ZFS dedup for my qemu VM images?

I am testing ZFS deduplication in PvE 6.0, I have a ZFS pool with a single (Raid0) 240GB SSD and mounted it as ZFS storage in web GUI.

I have installed a windows VM on this ZFS storage with a size of 30GB. When I clone this VM to the same pool the cloned VM also is consuming 30GB space.

As per deduplication, since both .raw files of 2 VMs are having same data it supposes to consume only 30GB right? but here it is consuming 30+30 60GB space in the ZFS pool.

In Proxmox the dedup works in VM images level ? or data level inside the guest?
 
Last edited:

aaron

Proxmox Staff Member
Staff member
Jun 3, 2019
105
5
18
By default dedup is not enabled because it needs a lot of resources.

Have you enabled it on the ZFS storage?

Also look into enabling compression. This way depending on what your data looks like you can still get some nice space savings without the big cost in resources needed for it.
 

jagan

Member
Feb 11, 2015
88
0
6
india,chennai
Hi,

Dedup has been enabled please check the screenshot. It is showing 90GB used for 3 similar VMs (each config. with 30GB).

Dedup img.jpg


Do I have to config. zfs storage in the thin provision or thick provision to enable dedupe?
 

aaron

Proxmox Staff Member
Staff member
Jun 3, 2019
105
5
18
Ah, The used column shows the full size if you did not enable thin provision. The whole size of the disk is reserved in that case. The actual usage is in the Refer column.

Without any experience with dedup myself there are two things that I see. First the DEDUP column which shows that there are a lot of blocks shared / deduplicated.

Second: if you look at the zfs list output and there in the AVAIL column. For some reason the disks themselves have pretty much exactly the 30G more available. Now I don't know if that means anything because I don't have any experience with dedup nor do I know anyone who has. You can try to clone the VM a third time and see how it behaves. Also try to diverge the data of a VM by changing things so it's data cannot be deduplicated anymore.

TL;DR:
Dedup works but the reserved space for the not thin provisioned VM is still shown. Used column shows how much space is reserved. Refer column shows actual usage. Dedup is saving how much? I don't really know, not much experience in that regard.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!