ZFS dedup on a Proxmox host in 2026 — has fast dedup changed the recommendation?

[DeQuzzie]

Hi,

The standing advice for years has been to leave dedup=off on ZFS pools under Proxmox: the DDT lives in RAM, the write path suffers, and you can't turn it back off for data already written. OpenZFS 2.3 ships fast dedup, which is supposed to address most of that — quota'd DDT, pruning, dedup logs.

Has that changed anything on the Proxmox side? Concretely, for a pool that holds both guest disks and a large dataset of ordinary files with a lot of duplicates among them:

is dedup=on still a "don't", or is it now a defensible option for the file dataset only, with the guest disks on a dataset that has it off?
is there any RAM sizing guidance that isn't the old 5 GB per TB rule of thumb?
dedup_table_quota — does anyone have it in production on a PVE host, and what happens to the write path when the table hits the quota?
Everything I can find on this is either pre-2.3 or not Proxmox-specific.

 

[daanw]

From my experience, even the old ZFS dedup was doable as long as you had a nvme special vdev included.
The dedup memory myth is pretty much like the general ZFS memory myth. Adding that special vdev definitely had more impact than adding more RAM.

Actual numbers obviously depend on your workload and the resulting dedup factor whether it is worth the trouble and extra hardware at all.
With Proxmox in mind, if you have hundreds of virtual machines sharing 90% of their blocks and are limited on storage it might be interesting to check it out.

 

[IsThisThingOn]

I thinkg this is en excellent read on why it might could be considered now.

I stil think it is not a good idea. Why? Storage is cheap, performance is not. While it is faster, it still comes with a IMHO extreme performance penalty.

The other thing is that I don't think it would actually save that much.
But still, I think that in real life the dedublication would even be with 1000 VMs not that big.
And also working with ZFS snapshots (cloning a base image VM) will probably yield way better results without any performance implications. But to be fair, that saved storage will go down with usage and updates so it probably also depends on how long your VMs lifteime is.

 

[UdoB]

Your statement...

Storage is cheap, performance is not.

...seems to come from an age more than 10 months ago. Storage is expensive now. And CPU performance on my machines is plenty - all cores are nearly idle with 4 to 14 percent usage in this very moment.

That said... I am definitely not going to enable DDUP on any production machine. But I have a fitting test machine and it is tempting to try it out...

Edit: actual sefttest postponed - one would need to specify an adequate test-setup first and run reproducible tests before and after...

 

[DeQuzzie]

Thanks, both angles are useful.

On the special vdev: that matches what I'd expect for the DDT lookups, but the part I can't find numbers for is the write path — does putting the DDT on NVMe actually remove the penalty, or just move it? If anyone has before/after write latency at a fixed queue depth, that would settle it.

One thing I should have been clearer about in the first post: the duplicates I care about aren't blocks shared between VM images. It's a bulk dataset of ordinary files — archives, media, build outputs — where a good share of the files are byte-identical whole copies of each other. Linked clones from a base image don't help with that, and the dedup factor doesn't decay with guest updates either. So I'd be looking at dedup=on on that one dataset only, guest disks staying on a dataset with it off. Does anyone run it that way?

@UdoB on the test setup — for "would it even save anything", zdb -S <pool> gives the dedup histogram and the ratio you would have got, without enabling anything. It walks every block so it's slow and wants RAM, but it answers the question before you commit to a one-way setting. Beyond that I'd measure write latency with fio at a fixed queue depth before and after, and watch the DDT size against dedup_table_quota — which is the other thing nobody has mentioned yet: what actually happens to writes when the table hits the quota?

 

[Onslow]

FWIW:
"ZFS Fast Dedup for Proxmox VE 9.x: Technical Implementation Guide"
https://klarasystems.com/articles/zfs-fast-dedup-for-proxmox-ve-9x/

 

[IsThisThingOn]

Storage is expensive now. And CPU performance on my machines is plenty

I am not talking about CPU performance but storage performance
I would argue the statement "storage is cheap, performance is not" has become even more true in the last 10 months.
Even though storage has got more expensive.

 

[UdoB]

I am not talking about CPU performance but storage performance

Oh, a classic misunderstanding...

 

[DeQuzzie]

Thanks @Onslow — that one actually answers the quota question from my first post:

"Once the quota is reached, new writes for dedup-enabled datasets bypass deduplication and are written as unique blocks, preventing unbounded DDT growth."

So it degrades to plain writes rather than stalling. Good to have that in writing somewhere.

Two other useful things in there: it drops the old 5 GB/TB rule in favour of "DDT no more than 50% of max ARC, or 85% of the dedup vdev if you have one", and it puts a floor under the whole question — deploy above 3:1, don't bother below 2:1.

What it doesn't have is a single measurement. No benchmark, no before/after, no latency figure, nothing on recordsize or volblocksize, and — @daanw — nothing on whether putting the DDT on a fast special vdev removes the write penalty or merely relocates it. That one is still open and it's the number I'd most like to see.

The design pattern it recommends is the split I asked about: one dataset with dedup=on, general guest storage without it on zstd. So the shape is endorsed — my case just has ordinary files rather than images on the dedup side, and I still haven't heard from anyone actually running it that way.

@UdoB — this is why I keep coming back to zdb -S. It is the "adequate test setup" for the first half of your question: it gives the histogram and the ratio you would have got, on your real data, with nothing enabled and nothing irreversible. It walks every block so it's slow and wants RAM, but if it comes back at 1.2:1 then the second half of the test never needs to happen. Rob Norris does exactly this on his own laptop in the post @IsThisThingOn linked and gets 1.00x.

One gap worth naming: for a Feb 2026 article about saving space on ZFS 2.3, it never mentions block cloning once. Where the duplicates are whole identical files rather than blocks shared across images, cloning gets most of the win with no DDT, no RAM budget and no one-way switch — and by the article's own 2:1/3:1 gating that kind of data is a poor inline-dedup candidate anyway. Different mechanism for a different shape of duplication, and the guide skips it entirely.

 

[waltar]

If doing dedup at all there's an additional special dedup vdev available which should be considered to be configured too.

 

[daanw]

What it doesn't have is a single measurement. No benchmark, no before/after, no latency figure, nothing on recordsize or volblocksize, and — @daanw — nothing on whether putting the DDT on a fast special vdev removes the write penalty or merely relocates it. That one is still open and it's the number I'd most like to see.

My use case was hot storage of archive material with a modest 1.3 dedup ratio. RAIDZ3 pool of 12x10TB SAS disks with 128GB RAM, 1M recordsize/zstd dataset. Real world write speeds of about 750 MB/s on a non-dedup dataset. Dedup had a factor ~7 penalty compared to a non-dedup dataset with the DDT still being loaded from the harddisks, even causing kernel timeouts on large operations. Additionally, the system severely suffered from write throttles down to KB/s which we were able to relieve up to 10-20 MB/s by adding 128GB more RAM. In other words, unworkable.

By adding a mirrored 1TB nvme special vdev the dedup penalty was practically gone, staying close to the non-dedup write speeds, no timeouts and hardly any throttling.

Keep in mind that this was all in the old dedup ZFS 0.8 - 2.0 era.

If doing dedup at all there's an additional special dedup vdev available which should be considered to be configured too.

Only consider a separate Dedup VDEV if you have a specific need to isolate DDT I/O from metadata/small file I/O, or if the Special VDEV is too small to hold both metadata and the DDT.

 

[Onslow]

What it doesn't have is a single measurement. No benchmark, no before/after, no latency figure, nothing on recordsize or volblocksize, and — @daanw — nothing on whether putting the DDT on a fast special vdev removes the write penalty or merely relocates it. That one is still open and it's the number I'd most like to see.

Hi, @DeQuzzie

A measurement (a general one, not particular for Proxmox) is in the article linked by other poster.
Raw performance, throughput, performance delta, dedup ratio, write and read latency - in various workloads:
"Introducing OpenZFS Fast Dedup"
https://klarasystems.com/articles/introducing-openzfs-fast-dedup/

A small section about "Dedup VDEV" (probably not answering your questions ) is in the article
"Pool and VDEV Topology for Proxmox Workloads"
https://klarasystems.com/articles/pool-and-vdev-topology-for-proxmox-workloads/

"Dedup VDEV: Locates the dedup table on a specific device, ensuring it has all of the available IOPS from that device and does not contend with other workloads. The dedup table is memory-hungry and can be redirected to the dedup (or failing that, special) VDEVs, however Klara generally discourages dedup for generic VM stores because it is often not worth the small storage savings, compared to the cost in RAM and CPU unless there is a very strong, proven dedup use-case."

And an article not about deduplication, but may be worth mentioning in Proxmox context:
"Pool and VDEV Topology for Proxmox Workloads"
https://klarasystems.com/articles/pool-and-vdev-topology-for-proxmox-workloads/

I'm not affiliated with that site, just have read a few articles there