ZFS dedup on a Proxmox host in 2026 — has fast dedup changed the recommendation?

I setup ZFS fast dedup on 14x DRAID2 spinning disks pool, with a mirrored Special SSD vdev sometime last year.

No issues except for (1) unfixable metadata permanent error that basically meant I had to create a new just-big-enough RAID0 pool, + copy the data back and forth with rclone to fix it as zfs send would abort. After that, the issue has not reoccurred. Dual-booting proxmox and IIRC Linux Mint
 
On dedup_table_quota — I asked about this in the first post, so here's the measurement.

Test box, not the production pool: PVE 9.2.5, zfs-2.4.3-pve1, loopback pool, compression=off, recordsize=128K. Same 1.25 GiB corpus written twice — once with dedup=on and no quota, once with dedup=on and dedup_table_quota=256K.

no dedup, baseline 1285.8 MiB allocated
dedup=on, no quota 840.9 MiB dedupratio 1.53x DDT 6684 entries, 3.27M on disk
dedup=on, quota 256K 1287.2 MiB dedupratio 1.00x dedup_table_size 1.95M

Two things there are worth reading twice. The table grew to 1.95M against a 256K quota — it's a soft target, not a cap. And the capped run deduplicated nothing at all while still paying for that table, ending up marginally worse than never enabling dedup.

zpool status -D explains it: all 3604 entries sat at refcnt 1. The unique part of the corpus happened to be written first, filled the table, and by the time the duplicate files arrived no new entries could be created — so every duplicate was written as a fresh block.

Writes never errored, so the "degrades gracefully" part holds. But the practical consequence is stronger than the documentation suggests: the Klara PVE guide says new writes simply bypass dedup once the quota is reached, which is accurate about the mechanism and quiet about the outcome. With an undersized quota, whether you get any benefit at all depends on the order your data lands in. Unique data arriving first poisons the table and leaves you with the cost and none of the saving.

Sizing the quota from a zdb -S histogram before enabling anything looks like the only way to avoid that.
 
  • Like
Reactions: Onslow and waltar
" Doing it wrong " and posting "results" doesn't mean much. Nobody in real life is going to try to impose a 256KB quota on dedup, this isn't the 1980s.

32GB RAM on the server. I gave my ~30TB 4TBx14-disk DRAID2 fast-dedup a quota of 16GB + special device mirror. Even with the pool ~70% full it hasn't gotten anywhere near that. IIRC it's only using somewhere under 3GB for dedup table.

Had One-off issue, permanent error in metadata that was probably from dual-booting different ZFS versions. Scrub wasn't fixing it at all. Created new "just big enough" temporary RAID0 pool, copied data over with rclone in parallel, recreated DRAID pool, copied data back. Metadata error gone.
 
  • Like
Reactions: Johannes S