deduplication

There are a large amounts of improvements in ZFS 2.3.0 if you use NVME and deduplication: https://github.com/openzfs/zfs/releases/tag/zfs-2.3.0 When can we expect this release to become available?

Hey there, I've recently set up PBS to serve as a central backup server in my network, but I've run into quite an inconvenience: Whenever I make a backup of e.g. my home directory, it's taking quite a while since it has to transfer like 200+ gigs over the network. With about 120MBps (seemingly...

Hello, Proxmox community! I have a query that I've been unable to resolve for the past weeks. My deduplication ratio on Proxmox Backup Server (PBS) has significantly decreased. Previously, I know (though I can't access older logs to confirm) that the deduplication factor was above 12, but...

I have setup a remote server in a different city to which I ship all backups using a sync job. The remote PBS datastore however doens't seem to be doing deduplication. The local PBS. Usage : 91.02% (3.97 TB of 4.37 TB) Backup Count CT : 32 Groups, 380 Snapshots Host : 0 Groups, 0...

Right now my backups are not encrypted. I would assume deduplication works between different vm/containers? So if you have a file on multiple containers/vms it would only be stored once? Which leads to my question....i'm thinking on encrypting my backups. If i use the same encryption key for...

Hi! I some specific questions. I am testing out PBS for a specific workflow consisiting of backing up debian hosted NFS server nearly 1 PB total. i want to know if i have to invest a little or many money for that. What i see is that i only achieve nearly 1.2 ~ 1.3 deduplication factor for...

Hallo, ich habe noch bis vor kurzem ESXI und Veeam Backup genutzt, da ich immer mehr Open Source einsetze möchte PVE / PBS interessante Features bieten migriere ich gerade darauf. Bisher hatte ich die Backups vom Host mit Veeam ins Rechenzentrum gemacht über eine herkömmlich DSL Leitung (100...

Since all of my backup files are the same size, can someone explain how this works or is implemented? Does it need to be enabled somewhere?

Hallo zusammen, folgende Ausgangssituation: PVE 7.1-12, PBS 2.1-5, VM Windows Server 2022 mit aktivierter Deduplikation (Dedup-Rate derzeit 23 %) Wie schon im Titel erwähnt, sind die Dateien, die per File Restore wiederhergestellt werden, defekt. Die Sicherung läuft ohne Probleme und die...

Hello! We have 2 Proxmox host and we use the included vzdump tool to backup the VMs and containers that run on them. Our backup location is a ZFS server shared over NFS to both Proxmox hosts. This ZFS server has a RAID-10 like configuration with 4 disks, 40GB SSD cache partition, 20GB RAM and...

Hi, I'm looking to create a script that will create a backup of my current VM and stored them in a deduplication system. (I'm using BorgBackup with a remote repo for storing) I've tried to use vzdump which create a .vma file that doesn't seem to work well with deduplication. To by-pass that...

I have a ZFS pool where I put (only) containers which will have mostly identical data (95% or more). I am trying to save some disk space by using ZFS (as each container is 30GB and I will run a couple dozen of them) This is the current setup: # zfs get all | egrep 'dedup|checksum|compression'...

Hello everyone, I'm testing borgbackup to copy our backups to some off-site storage. With the normal tar file and following ones, the deduplication isnt as effective as it could be. When I extract the vzdump and inserting every file one by one its much more space efficient. My question is now...

I am planning to use several virtual machines that will all use the same Dropbox account, and I want to use the deduplication abilities of ZFS to minimize wasted disk space. I also want the drobox data to be on a separate physical device from the OSs so that they do not take up read/write...

I noticed QEMU backing images aren't supported in Proxmox. I tried setting up a backing image using qemu-img and ran the machine off the delta only to have it crash. Will there ever be support for this feature? The ability to use the backing images would allow deduplication of like operating...