PBS Sync Data Size Discrepancy - Need Help Understanding

Obsidian

New Member
Mar 13, 2024
2
1
1
I’m new to Proxmox Backup Server and have encountered a confusing issue with the sync process that I hope someone can explain.

My Setup:
• Server 1 (S1): Main Proxmox host + secondary PBS (pbs2)
• Server 2 (S2): Primary PBS (pbs1)
• Current workflow: S1 VMs → backup to pbs1 on S2

The Issue:
My primary PBS (pbs1) is working perfectly with 1.3TB of backup data. The deduplication is excellent - I have VMs that are 500GB each with 7 daily backups, but the total storage is only 1.3TB across all backups.

For redundancy, I created a secondary PBS (pbs2) on S1 and configured a sync job to pull the root datastore from pbs1. This is where things get strange:
The sync pulled 2.85TB of data - more than double what exists on the source PBS. Both the pbs1 UI and underlying filesystem (`df -h`) confirm only 1.3TB is stored on the primary server. The pbs2 dataset was newly created and confirmed empty.

After running garbage collection on pbs2, I get this output:

2025-09-14T11:14:23-04:00: Chunk cache: hits 444099, misses 319810 (hit ratio 58.14%)
2025-09-14T11:14:23-04:00: Removed garbage: 0 B
2025-09-14T11:14:23-04:00: Removed chunks: 0
2025-09-14T11:14:23-04:00: Pending removals: 53.276 GiB (in 13655 chunks)
2025-09-14T11:14:23-04:00: Original data usage: 2.914 TiB
2025-09-14T11:14:23-04:00: On-Disk usage: 1.142 TiB (39.20%)
2025-09-14T11:14:23-04:00: Deduplication factor: 2.55

Questions:
1. How is it possible for a 1.3TB source to generate 2.85TB of synced data? Why isn't it just a mirror of pbs1?
2. The garbage collection shows good deduplication (1.142 TiB on-disk), but why does the filesystem & pbs2 UI still show 2.85TB used?
3. Will the pending removals (53.276 GiB) resolve the size discrepancy once the grace period expires?

One other thing to note - I have encryption enabled on all backups.
 
If anyone’s curious, I figured out the problem: I was using ZFS with a dataset set to redundant_metadata=all and a 4K record size, which wasted about 1.3TB of space. By switching to redundant_metadata=none (since I’m on raidz2) and setting recordsize=1M, then copying the 2.9TB of data to a new dataset with Rsync, the used storage dropped to just 1.2TB. I hadn’t realized that an improper record size could waste so much space, but it made a massive difference.
 
  • Like
Reactions: Onslow