VM incremental backups much larger than expected on PBS.

R

RichardCNL

New Member
Dec 28, 2025
6
0
1
Hi all,


I’m running into an issue with Proxmox Backup Server (PBS) and wanted to see if anyone has experienced something similar.


Setup:


  • VM: DirectAdmin hosting server (~500 GB disk)
  • Filesystem: XFS
  • Database: MySQL (multiple WordPress sites, InnoDB tables, log tables)
  • PBS: backup every 4 hours
  • Observed logical changes on VM in last 4 hours: ~7 GB (measured via find -mmin -240)
  • PBS incremental backup size: ~40 GB per snapshot, every 4 hours.

What I’ve tried:


  • Checked /var/lib/mysql and /home for modified files → total changes ~7 GB
  • Considered chunk size adjustments (but for VM backups, chunk size is not configurable)
  • PBS data sync level: filesystem (to capture only changed blocks)
  • XFS metadata and MySQL small writes

Other relevant info:


  • ZFS replication is active between nodes (incremental snapshots)

Question:


  1. Has anyone experienced a similar discrepancy between actual modified data and PBS incremental size?
  2. Could XFS metadata churn, InnoDB small writes, or ZFS replication contribute to the high incremental size?
  3. Any tips to reduce incremental backup size for database-heavy VMs while keeping filesystem-level sync?

I’d really appreciate insights or examples from others who have run into this, I can’t find many posts on the web about this scenario.
 
Code: 
INFO: starting new backup job: vzdump --all 1 --notes-template '{{guestname}}' --fleecing 0 --mode snapshot --quiet 1 --mailto info@exampledomain.tld --mailnotification failure --notification-mode legacy-sendmail --storage pbs
INFO: skip external VMs: 101, 102, 104, 107, 113, 115, 309, 310, 311, 312, 314, 317
INFO: Starting Backup of VM 203 (qemu)
INFO: Backup started at 2025-12-28 12:00:01
INFO: status = running
INFO: VM Name: theems.exampledomain.tld
INFO: include disk 'scsi0' 'vm-storage:vm-203-disk-0' 550G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/203/2025-12-28T11:00:01Z'
INFO: enabling encryption
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task '00bae6eb-7a5a-4e3f-8206-1e3fdec8a3de'
INFO: resuming VM again
INFO: scsi0: dirty-bitmap status: OK (41.8 GiB of 550.0 GiB dirty)
INFO: using fast incremental mode (dirty-bitmap), 41.8 GiB dirty of 550.0 GiB total
INFO:   1% (560.0 MiB of 41.8 GiB) in 3s, read: 186.7 MiB/s, write: 178.7 MiB/s
INFO:   2% (1.2 GiB of 41.8 GiB) in 6s, read: 209.3 MiB/s, write: 206.7 MiB/s
INFO:   4% (1.8 GiB of 41.8 GiB) in 9s, read: 212.0 MiB/s, write: 210.7 MiB/s
INFO:   5% (2.3 GiB of 41.8 GiB) in 12s, read: 188.0 MiB/s, write: 188.0 MiB/s
INFO:   6% (2.8 GiB of 41.8 GiB) in 15s, read: 172.0 MiB/s, write: 172.0 MiB/s
INFO:   7% (3.3 GiB of 41.8 GiB) in 18s, read: 168.0 MiB/s, write: 168.0 MiB/s
INFO:   9% (3.8 GiB of 41.8 GiB) in 21s, read: 170.7 MiB/s, write: 169.3 MiB/s
INFO:  10% (4.3 GiB of 41.8 GiB) in 24s, read: 162.7 MiB/s, write: 161.3 MiB/s
INFO:  11% (4.8 GiB of 41.8 GiB) in 27s, read: 182.7 MiB/s, write: 181.3 MiB/s
INFO:  12% (5.4 GiB of 41.8 GiB) in 30s, read: 174.7 MiB/s, write: 172.0 MiB/s
INFO:  14% (5.9 GiB of 41.8 GiB) in 33s, read: 178.7 MiB/s, write: 169.3 MiB/s
INFO:  15% (6.4 GiB of 41.8 GiB) in 36s, read: 178.7 MiB/s, write: 173.3 MiB/s
INFO:  16% (6.9 GiB of 41.8 GiB) in 39s, read: 181.3 MiB/s, write: 178.7 MiB/s
INFO:  17% (7.4 GiB of 41.8 GiB) in 42s, read: 174.7 MiB/s, write: 173.3 MiB/s
INFO:  19% (8.0 GiB of 41.8 GiB) in 45s, read: 186.7 MiB/s, write: 186.7 MiB/s
INFO:  20% (8.5 GiB of 41.8 GiB) in 48s, read: 169.3 MiB/s, write: 168.0 MiB/s
INFO:  21% (9.0 GiB of 41.8 GiB) in 51s, read: 174.7 MiB/s, write: 162.7 MiB/s
INFO:  22% (9.5 GiB of 41.8 GiB) in 54s, read: 177.3 MiB/s, write: 173.3 MiB/s
INFO:  24% (10.0 GiB of 41.8 GiB) in 57s, read: 178.7 MiB/s, write: 170.7 MiB/s
INFO:  25% (10.5 GiB of 41.8 GiB) in 1m, read: 160.0 MiB/s, write: 152.0 MiB/s
INFO:  26% (11.1 GiB of 41.8 GiB) in 1m 3s, read: 193.3 MiB/s, write: 137.3 MiB/s
INFO:  27% (11.7 GiB of 41.8 GiB) in 1m 6s, read: 213.3 MiB/s, write: 117.3 MiB/s
INFO:  29% (12.2 GiB of 41.8 GiB) in 1m 9s, read: 180.0 MiB/s, write: 168.0 MiB/s
INFO:  30% (12.8 GiB of 41.8 GiB) in 1m 12s, read: 178.7 MiB/s, write: 178.7 MiB/s
INFO:  31% (13.2 GiB of 41.8 GiB) in 1m 15s, read: 169.3 MiB/s, write: 168.0 MiB/s
INFO:  32% (13.7 GiB of 41.8 GiB) in 1m 18s, read: 161.3 MiB/s, write: 161.3 MiB/s
INFO:  34% (14.2 GiB of 41.8 GiB) in 1m 21s, read: 170.7 MiB/s, write: 169.3 MiB/s
INFO:  35% (14.7 GiB of 41.8 GiB) in 1m 24s, read: 174.7 MiB/s, write: 174.7 MiB/s
INFO:  36% (15.2 GiB of 41.8 GiB) in 1m 27s, read: 160.0 MiB/s, write: 158.7 MiB/s
INFO:  37% (15.7 GiB of 41.8 GiB) in 1m 30s, read: 162.7 MiB/s, write: 152.0 MiB/s
INFO:  38% (16.2 GiB of 41.8 GiB) in 1m 33s, read: 166.7 MiB/s, write: 154.7 MiB/s
INFO:  39% (16.6 GiB of 41.8 GiB) in 1m 36s, read: 165.3 MiB/s, write: 165.3 MiB/s
INFO:  40% (17.1 GiB of 41.8 GiB) in 1m 39s, read: 160.0 MiB/s, write: 156.0 MiB/s
INFO:  42% (17.6 GiB of 41.8 GiB) in 1m 42s, read: 158.7 MiB/s, write: 158.7 MiB/s
INFO:  43% (18.1 GiB of 41.8 GiB) in 1m 45s, read: 162.7 MiB/s, write: 160.0 MiB/s
INFO:  44% (18.5 GiB of 41.8 GiB) in 1m 48s, read: 149.3 MiB/s, write: 148.0 MiB/s
INFO:  45% (18.9 GiB of 41.8 GiB) in 1m 51s, read: 154.7 MiB/s, write: 148.0 MiB/s
INFO:  46% (19.4 GiB of 41.8 GiB) in 1m 54s, read: 162.7 MiB/s, write: 154.7 MiB/s
INFO:  47% (19.9 GiB of 41.8 GiB) in 1m 57s, read: 176.0 MiB/s, write: 169.3 MiB/s
INFO:  48% (20.5 GiB of 41.8 GiB) in 2m, read: 180.0 MiB/s, write: 170.7 MiB/s
INFO:  50% (21.0 GiB of 41.8 GiB) in 2m 3s, read: 170.7 MiB/s, write: 170.7 MiB/s
INFO:  51% (21.5 GiB of 41.8 GiB) in 2m 6s, read: 168.0 MiB/s, write: 168.0 MiB/s
INFO:  52% (21.9 GiB of 41.8 GiB) in 2m 9s, read: 165.3 MiB/s, write: 164.0 MiB/s
INFO:  53% (22.4 GiB of 41.8 GiB) in 2m 12s, read: 169.3 MiB/s, write: 168.0 MiB/s
INFO:  54% (22.9 GiB of 41.8 GiB) in 2m 15s, read: 160.0 MiB/s, write: 157.3 MiB/s
INFO:  55% (23.4 GiB of 41.8 GiB) in 2m 18s, read: 164.0 MiB/s, write: 160.0 MiB/s
INFO:  57% (23.8 GiB of 41.8 GiB) in 2m 21s, read: 156.0 MiB/s, write: 156.0 MiB/s
INFO:  58% (24.3 GiB of 41.8 GiB) in 2m 24s, read: 170.7 MiB/s, write: 170.7 MiB/s
INFO:  59% (24.9 GiB of 41.8 GiB) in 2m 27s, read: 172.0 MiB/s, write: 172.0 MiB/s
INFO:  60% (25.4 GiB of 41.8 GiB) in 2m 30s, read: 173.3 MiB/s, write: 170.7 MiB/s
INFO:  61% (25.9 GiB of 41.8 GiB) in 2m 33s, read: 169.3 MiB/s, write: 160.0 MiB/s
INFO:  63% (26.4 GiB of 41.8 GiB) in 2m 36s, read: 170.7 MiB/s, write: 168.0 MiB/s
INFO:  64% (26.9 GiB of 41.8 GiB) in 2m 39s, read: 174.7 MiB/s, write: 170.7 MiB/s
INFO:  65% (27.4 GiB of 41.8 GiB) in 2m 42s, read: 172.0 MiB/s, write: 170.7 MiB/s
INFO:  66% (27.9 GiB of 41.8 GiB) in 2m 45s, read: 166.7 MiB/s, write: 149.3 MiB/s
INFO:  67% (28.3 GiB of 41.8 GiB) in 2m 48s, read: 165.3 MiB/s, write: 154.7 MiB/s
INFO:  68% (28.8 GiB of 41.8 GiB) in 2m 51s, read: 162.7 MiB/s, write: 157.3 MiB/s
INFO:  70% (29.3 GiB of 41.8 GiB) in 2m 54s, read: 172.0 MiB/s, write: 166.7 MiB/s
INFO:  71% (29.8 GiB of 41.8 GiB) in 2m 57s, read: 168.0 MiB/s, write: 168.0 MiB/s
INFO:  72% (30.3 GiB of 41.8 GiB) in 3m, read: 165.3 MiB/s, write: 162.7 MiB/s
INFO:  73% (30.8 GiB of 41.8 GiB) in 3m 3s, read: 166.7 MiB/s, write: 161.3 MiB/s
INFO:  74% (31.3 GiB of 41.8 GiB) in 3m 6s, read: 174.7 MiB/s, write: 164.0 MiB/s
INFO:  75% (31.8 GiB of 41.8 GiB) in 3m 9s, read: 156.0 MiB/s, write: 150.7 MiB/s
INFO:  77% (32.2 GiB of 41.8 GiB) in 3m 12s, read: 166.7 MiB/s, write: 160.0 MiB/s
INFO:  78% (32.8 GiB of 41.8 GiB) in 3m 15s, read: 173.3 MiB/s, write: 165.3 MiB/s
INFO:  79% (33.2 GiB of 41.8 GiB) in 3m 18s, read: 157.3 MiB/s, write: 152.0 MiB/s
INFO:  80% (33.7 GiB of 41.8 GiB) in 3m 21s, read: 166.7 MiB/s, write: 156.0 MiB/s
INFO:  81% (34.2 GiB of 41.8 GiB) in 3m 24s, read: 166.7 MiB/s, write: 165.3 MiB/s
INFO:  82% (34.7 GiB of 41.8 GiB) in 3m 27s, read: 165.3 MiB/s, write: 158.7 MiB/s
INFO:  84% (35.2 GiB of 41.8 GiB) in 3m 30s, read: 172.0 MiB/s, write: 169.3 MiB/s
INFO:  85% (35.7 GiB of 41.8 GiB) in 3m 33s, read: 174.7 MiB/s, write: 168.0 MiB/s
INFO:  86% (36.2 GiB of 41.8 GiB) in 3m 36s, read: 166.7 MiB/s, write: 160.0 MiB/s
INFO:  87% (36.7 GiB of 41.8 GiB) in 3m 39s, read: 177.3 MiB/s, write: 170.7 MiB/s
INFO:  89% (37.3 GiB of 41.8 GiB) in 3m 42s, read: 210.7 MiB/s, write: 210.7 MiB/s
INFO:  90% (37.9 GiB of 41.8 GiB) in 3m 45s, read: 190.7 MiB/s, write: 190.7 MiB/s
INFO:  92% (38.5 GiB of 41.8 GiB) in 3m 48s, read: 206.7 MiB/s, write: 206.7 MiB/s
INFO:  93% (39.0 GiB of 41.8 GiB) in 3m 51s, read: 180.0 MiB/s, write: 180.0 MiB/s
INFO:  94% (39.6 GiB of 41.8 GiB) in 3m 54s, read: 202.7 MiB/s, write: 202.7 MiB/s
INFO:  96% (40.3 GiB of 41.8 GiB) in 3m 57s, read: 225.3 MiB/s, write: 225.3 MiB/s
INFO:  97% (40.8 GiB of 41.8 GiB) in 4m, read: 181.3 MiB/s, write: 181.3 MiB/s
INFO:  98% (41.3 GiB of 41.8 GiB) in 4m 3s, read: 189.3 MiB/s, write: 189.3 MiB/s
INFO: 100% (41.8 GiB of 41.8 GiB) in 4m 6s, read: 156.0 MiB/s, write: 145.3 MiB/s
INFO: backup was done incrementally, reused 509.55 GiB (92%)
INFO: transferred 41.80 GiB in 246 seconds (174.0 MiB/s)
INFO: adding notes to backup
INFO: Finished Backup of VM 203 (00:04:08)
INFO: Backup finished at 2025-12-28 12:04:09
INFO: Starting Backup of VM 205 (qemu)
INFO: Backup started at 2025-12-28 12:04:09
INFO: status = running
INFO: VM Name: sftpgw.exampledomain.tld
INFO: include disk 'scsi0' 'vm-storage:vm-205-disk-0' 80G
INFO: exclude disk 'scsi1' 'nfs-synology-disks:205/vm-205-disk-0.qcow2' (backup=no)
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/205/2025-12-28T11:04:09Z'
INFO: enabling encryption
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task 'f0c1da98-3d8b-4f8d-a2a7-6637a7a9736e'
INFO: resuming VM again
INFO: scsi0: dirty-bitmap status: OK (204.0 MiB of 80.0 GiB dirty)
INFO: using fast incremental mode (dirty-bitmap), 204.0 MiB dirty of 80.0 GiB total
INFO: 100% (204.0 MiB of 204.0 MiB) in 2s, read: 102.0 MiB/s, write: 102.0 MiB/s
INFO: backup was done incrementally, reused 79.80 GiB (99%)
INFO: transferred 204.00 MiB in 2 seconds (102.0 MiB/s)
INFO: adding notes to backup
INFO: Finished Backup of VM 205 (00:00:03)
INFO: Backup finished at 2025-12-28 12:04:12
INFO: Starting Backup of VM 206 (qemu)
INFO: Backup started at 2025-12-28 12:04:12
INFO: status = running
INFO: VM Name: s1.exampledomain.tld
INFO: include disk 'scsi0' 'vm-storage:vm-206-disk-0' 500G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/206/2025-12-28T11:04:12Z'
INFO: enabling encryption
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task '2d6c804b-f4f6-4195-9902-12fa5d4aabb8'
INFO: resuming VM again
INFO: scsi0: dirty-bitmap status: OK (25.3 GiB of 500.0 GiB dirty)
INFO: using fast incremental mode (dirty-bitmap), 25.3 GiB dirty of 500.0 GiB total
INFO:   1% (516.0 MiB of 25.3 GiB) in 3s, read: 172.0 MiB/s, write: 172.0 MiB/s
INFO:   3% (1008.0 MiB of 25.3 GiB) in 6s, read: 164.0 MiB/s, write: 164.0 MiB/s
INFO:   5% (1.5 GiB of 25.3 GiB) in 9s, read: 173.3 MiB/s, write: 173.3 MiB/s
INFO:   7% (2.0 GiB of 25.3 GiB) in 12s, read: 174.7 MiB/s, write: 174.7 MiB/s
INFO:  10% (2.5 GiB of 25.3 GiB) in 15s, read: 180.0 MiB/s, write: 180.0 MiB/s
INFO:  11% (3.0 GiB of 25.3 GiB) in 18s, read: 169.3 MiB/s, write: 169.3 MiB/s
INFO:  14% (3.5 GiB of 25.3 GiB) in 21s, read: 176.0 MiB/s, write: 174.7 MiB/s
INFO:  16% (4.1 GiB of 25.3 GiB) in 24s, read: 188.0 MiB/s, write: 188.0 MiB/s
INFO:  18% (4.6 GiB of 25.3 GiB) in 27s, read: 184.0 MiB/s, write: 181.3 MiB/s
INFO:  20% (5.2 GiB of 25.3 GiB) in 30s, read: 177.3 MiB/s, write: 173.3 MiB/s
INFO:  22% (5.7 GiB of 25.3 GiB) in 33s, read: 184.0 MiB/s, write: 181.3 MiB/s
INFO:  24% (6.2 GiB of 25.3 GiB) in 36s, read: 186.7 MiB/s, write: 185.3 MiB/s
INFO:  26% (6.8 GiB of 25.3 GiB) in 39s, read: 193.3 MiB/s, write: 188.0 MiB/s
INFO:  29% (7.4 GiB of 25.3 GiB) in 42s, read: 192.0 MiB/s, write: 189.3 MiB/s
INFO:  31% (8.0 GiB of 25.3 GiB) in 45s, read: 221.3 MiB/s, write: 192.0 MiB/s
INFO:  33% (8.5 GiB of 25.3 GiB) in 48s, read: 176.0 MiB/s, write: 174.7 MiB/s
INFO:  35% (9.0 GiB of 25.3 GiB) in 51s, read: 156.0 MiB/s, write: 156.0 MiB/s
INFO:  37% (9.5 GiB of 25.3 GiB) in 54s, read: 173.3 MiB/s, write: 172.0 MiB/s
INFO:  39% (10.0 GiB of 25.3 GiB) in 57s, read: 177.3 MiB/s, write: 176.0 MiB/s
INFO:  41% (10.5 GiB of 25.3 GiB) in 1m, read: 169.3 MiB/s, write: 169.3 MiB/s
INFO:  43% (11.0 GiB of 25.3 GiB) in 1m 3s, read: 161.3 MiB/s, write: 161.3 MiB/s
INFO:  45% (11.5 GiB of 25.3 GiB) in 1m 6s, read: 182.7 MiB/s, write: 181.3 MiB/s
INFO:  47% (12.0 GiB of 25.3 GiB) in 1m 9s, read: 177.3 MiB/s, write: 177.3 MiB/s
INFO:  49% (12.5 GiB of 25.3 GiB) in 1m 12s, read: 166.7 MiB/s, write: 166.7 MiB/s
INFO:  51% (13.0 GiB of 25.3 GiB) in 1m 15s, read: 177.3 MiB/s, write: 177.3 MiB/s
INFO:  53% (13.6 GiB of 25.3 GiB) in 1m 18s, read: 180.0 MiB/s, write: 177.3 MiB/s
INFO:  55% (14.1 GiB of 25.3 GiB) in 1m 21s, read: 182.7 MiB/s, write: 177.3 MiB/s
INFO:  58% (14.7 GiB of 25.3 GiB) in 1m 24s, read: 204.0 MiB/s, write: 197.3 MiB/s
INFO:  60% (15.3 GiB of 25.3 GiB) in 1m 27s, read: 188.0 MiB/s, write: 188.0 MiB/s
INFO:  62% (15.8 GiB of 25.3 GiB) in 1m 30s, read: 169.3 MiB/s, write: 169.3 MiB/s
INFO:  64% (16.3 GiB of 25.3 GiB) in 1m 33s, read: 172.0 MiB/s, write: 172.0 MiB/s
INFO:  66% (16.8 GiB of 25.3 GiB) in 1m 36s, read: 180.0 MiB/s, write: 178.7 MiB/s
INFO:  68% (17.3 GiB of 25.3 GiB) in 1m 39s, read: 174.7 MiB/s, write: 170.7 MiB/s
INFO:  70% (17.8 GiB of 25.3 GiB) in 1m 42s, read: 168.0 MiB/s, write: 164.0 MiB/s
INFO:  72% (18.3 GiB of 25.3 GiB) in 1m 45s, read: 174.7 MiB/s, write: 174.7 MiB/s
INFO:  74% (18.9 GiB of 25.3 GiB) in 1m 48s, read: 188.0 MiB/s, write: 172.0 MiB/s
INFO:  76% (19.3 GiB of 25.3 GiB) in 1m 51s, read: 161.3 MiB/s, write: 161.3 MiB/s
INFO:  78% (19.9 GiB of 25.3 GiB) in 1m 54s, read: 186.7 MiB/s, write: 185.3 MiB/s
INFO:  80% (20.4 GiB of 25.3 GiB) in 1m 57s, read: 182.7 MiB/s, write: 178.7 MiB/s
INFO:  82% (20.9 GiB of 25.3 GiB) in 2m, read: 185.3 MiB/s, write: 172.0 MiB/s
INFO:  84% (21.4 GiB of 25.3 GiB) in 2m 3s, read: 169.3 MiB/s, write: 169.3 MiB/s
INFO:  86% (21.9 GiB of 25.3 GiB) in 2m 6s, read: 170.7 MiB/s, write: 170.7 MiB/s
INFO:  88% (22.4 GiB of 25.3 GiB) in 2m 9s, read: 172.0 MiB/s, write: 172.0 MiB/s
INFO:  90% (23.0 GiB of 25.3 GiB) in 2m 12s, read: 176.0 MiB/s, write: 176.0 MiB/s
INFO:  92% (23.5 GiB of 25.3 GiB) in 2m 15s, read: 181.3 MiB/s, write: 180.0 MiB/s
INFO:  95% (24.0 GiB of 25.3 GiB) in 2m 18s, read: 182.7 MiB/s, write: 181.3 MiB/s
INFO:  97% (24.6 GiB of 25.3 GiB) in 2m 21s, read: 178.7 MiB/s, write: 177.3 MiB/s
INFO:  99% (25.1 GiB of 25.3 GiB) in 2m 24s, read: 193.3 MiB/s, write: 193.3 MiB/s
INFO: 100% (25.3 GiB of 25.3 GiB) in 2m 26s, read: 82.0 MiB/s, write: 76.0 MiB/s
INFO: backup was done incrementally, reused 475.07 GiB (95%)
INFO: transferred 25.28 GiB in 146 seconds (177.3 MiB/s)
INFO: adding notes to backup
INFO: Finished Backup of VM 206 (00:02:27)
INFO: Backup finished at 2025-12-28 12:06:39
INFO: Starting Backup of VM 208 (qemu)
INFO: Backup started at 2025-12-28 12:06:39
INFO: status = running
INFO: VM Name: iem.exampledomain.tld
INFO: include disk 'scsi0' 'vm-storage:vm-208-disk-0' 306434M
INFO: include disk 'efidisk0' 'vm-storage:vm-208-disk-1' 528K
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/208/2025-12-28T11:06:39Z'
INFO: enabling encryption
INFO: started backup task 'bc0ef49e-3648-4505-b879-103253b217d4'
INFO: resuming VM again
INFO: efidisk0: dirty-bitmap status: OK (drive clean)
INFO: scsi0: dirty-bitmap status: OK (560.0 MiB of 299.3 GiB dirty)
INFO: using fast incremental mode (dirty-bitmap), 560.0 MiB dirty of 299.3 GiB total
INFO: 100% (560.0 MiB of 560.0 MiB) in 3s, read: 186.7 MiB/s, write: 178.7 MiB/s
INFO: backup was done incrementally, reused 298.73 GiB (99%)
INFO: transferred 560.00 MiB in 3 seconds (186.7 MiB/s)
INFO: adding notes to backup
INFO: Finished Backup of VM 208 (00:00:04)
INFO: Backup finished at 2025-12-28 12:06:43
INFO: Starting Backup of VM 216 (qemu)
INFO: Backup started at 2025-12-28 12:06:43
INFO: status = running
INFO: VM Name: vliet.exampledomain.tld
INFO: include disk 'scsi0' 'vm-storage:vm-216-disk-0' 80G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/216/2025-12-28T11:06:43Z'
INFO: enabling encryption
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task '75936215-3a2b-4c4f-9421-d35a2a89c8c1'
INFO: resuming VM again
INFO: scsi0: dirty-bitmap status: OK (2.5 GiB of 80.0 GiB dirty)
INFO: using fast incremental mode (dirty-bitmap), 2.5 GiB dirty of 80.0 GiB total
INFO:  21% (532.0 MiB of 2.5 GiB) in 3s, read: 177.3 MiB/s, write: 176.0 MiB/s
INFO:  39% (1004.0 MiB of 2.5 GiB) in 6s, read: 157.3 MiB/s, write: 152.0 MiB/s
INFO:  62% (1.5 GiB of 2.5 GiB) in 9s, read: 190.7 MiB/s, write: 190.7 MiB/s
INFO:  85% (2.1 GiB of 2.5 GiB) in 12s, read: 193.3 MiB/s, write: 192.0 MiB/s
INFO: 100% (2.5 GiB of 2.5 GiB) in 14s, read: 186.0 MiB/s, write: 186.0 MiB/s
INFO: backup was done incrementally, reused 77.55 GiB (96%)
INFO: transferred 2.47 GiB in 14 seconds (180.6 MiB/s)
INFO: adding notes to backup
INFO: Finished Backup of VM 216 (00:00:15)
INFO: Backup finished at 2025-12-28 12:06:58
INFO: Backup job finished successfully
TASK OK
 
Hi, I added a log but it is at review bij the Moderators.
The problem is this:
INFO: scsi0: dirty-bitmap status: OK (41.8 GiB of 550.0 GiB dirty)

It sees 41GB of dirty bitmap, even if we backup every 2 hours. This doenst make sense as the real change on the VM is only max 8GB.
 
Hi,

From the QEMU doc :
Bitmaps are bit vectors where each ‘1’ bit in the vector indicates a modified(“dirty”) segment of the corresponding block device. The size of the segment that is tracked is the granularity of the bitmap. If the granularity of a bitmap is 64K, each ‘1’ bit means that a 64K region as a whole may have changed in some way, possibly by as little as one byte.
Click to expand...
Cf : https://qemu-project.gitlab.io/qemu/interop/bitmaps.html#overview

If I ask one of my VMs I got this :
"dirty-bitmaps": [
{
"name": "pbs-incremental-dirty-bitmap",
"recording": true,
"persistent": false,
"busy": false,
"granularity": 4194304,
"count": 96468992
}
],
Click to expand...
So PBS seems to use 4MiB for granularity.

You said that your system use XFS, by default XFS seems to use a 4KiB block size (from what I have seek), and can add another write (inode & "XFS journal").

So for example you use "touch mytestfile.txt" I see 3 options :
  1. You are lucky, the data block/inode/"xfs journal" touch the same chunk for the dirty bitmap => your dirty-bitmap will be increased by 4MiB
  2. You are a bit lucky, the data block/inode touch the same chunk for the dirty bitmap, but the "xfs journal" touch another chunk => your dirty-bitmap will be increased by 8MiB
  3. You are very unlucky and all touch a distinct chunck => your dirty-bitmap will be increased by 12MiB
So in theory and if I understand correctly how the dirty-bitmap work, in the worst scenario case with a 7GiB changed inside the VMs, the dirty-bitmap should be around 21GiB, but that is only correct if we consider the number are correct (because you should consider all changes inside your VM, and i don't think "find" is the right tool for that).

Best regards,
 
Hi Janus,

We do have aprox 6TB production data and setup a PBS node with 16TB (14x 1.6TB RAIDZ3) of SSD disks, as we thought this would be more than enough.
Now it appears to be enough for a few backups per VM as deduplication is not near what we expected from other vendors like Nakivo and VEEAM.
So by my understanding PBS gives a really bad dedup ratio with XFS, around 4 or 5 and there is nothing we can do about it?
 
Hi,

I don't know how you make the connection between a 40GB dirty bitmap and bad deduplication.

Can you explain more and/or give more details, because it's seem you seek for a specific answer without telling the true question.

Best regards,
 
Hi,

Well, the answer looking for is, how build a bigger retention of lets say 90 days without running out of disk space.
We expected to get a better result as we also use BORG to backup our complete cluster which is using only 8TB backup storage to backup 6TB of production data for 90 days.
And now with PBS we backup 2TB of production data (So small portion of what is also backupped with BORG) and already hitting 10TB of SSD disk usage.
So this makes no sense, except if PBS is really that bad with handling XFS disks.
 
> for modified files → total changes ~7 GB

This sounds like you found the "new" usage found at the end of the 4 hours, not the total amount written to disk during that time?

If one rewrote the entire disk with the same bits, per my understanding, I'd expect it would be 100% new and somewhere between zero and 100% new usage depending on where the bits were actually written to disk.

PBS compares "this 4 MB chunk I found" to "all the chunks in PBS already" and either saves the new one or knows it doesn't need to save it.

We saw around 100x dedupe early on in our cluster, IIRC, until we added several Windows VMs and several more servers, and between the new OS and usage PBS is somewhere around 55x, without looking today. So dedupe is all relative to how much actual data is changing.
 
Hi Steve,

We used a simple find command to check for all changed data on that VM. But this are al business Wordpress websites and some small Woocommerce shops, nothing special. Also our other backup system BORG is not noticing really big changes (Around 2 -3 percent per day max).
 
You must log in or register to reply here.
Top Bottom