Proxmox is backing up my entire physical disk rather than just used space

[tailsofautonomy]

I am new to Proxmox but so far am very impressed with how intuitive it is and how much of it just works without additional configuration.

I am however running into an issue with my Proxmox Backup Server and my OMV7 VM running on PVE. I am trying to back the VM (64GB) up alongside it's 1TB data drive (LUKS Encrypted) that has been passed through to the VM. The issue is PBS is trying to backup the entire 1TB drive despite the fact only 600GB is in use. As my backup drive is only 1TB in size, this leads to the backup failing. I have tried to following but still running into the same issue:

• Enabled Discard and SSD Emulation
• Enabled fstrim support within OMV7 and confirmed it is working via fstrim -av and made it persidant via cryptsetup --allow-discards --persistent refresh /dev/mapper/sdb-crypt
• Tried backing up just the pass through 1TB data drive without the VM being backed up.
• Tried changing my backup mode from Snapshot to Suspend.

Here is an example log of what I mean:

INFO: starting new backup job: vzdump 100 --remove 0 --node redacted --mode suspend --notes-template '{{guestname}}' --notification-mode auto --storage proxmox-pbs
INFO: Starting Backup of VM 100 (qemu)
INFO: Backup started at 2025-04-14 19:12:45
INFO: status = running
INFO: backup mode: suspend
INFO: ionice priority: 7
INFO: VM Name: OMV
INFO: include disk 'scsi0' 'local-lvm:vm-100-disk-0' 64G
INFO: include disk 'scsi5' '/dev/disk/by-id/ata-Samsung_SSD_860_QVO_1TB_S4CZNF0N592583L' 976762584K
INFO: suspending guest
INFO: skip unused drive 'local-lvm:vm-100-disk-1' (not included into backup)
INFO: pending configuration changes found (not included into backup)
INFO: creating Proxmox Backup Server archive 'vm/100/2025-04-14T18:12:45Z'
INFO: enabling encryption
INFO: started backup task '443d0a84-8972-4c4f-9088-af6d612d916e'
INFO: resuming VM again after 1 seconds
INFO: scsi0: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: scsi5: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: 0% (324.0 MiB of 995.5 GiB) in 3s, read: 108.0 MiB/s, write: 108.0 MiB/s

It then proceeds to fill up around 900GB's or so and error out. Is there a way with my configuration to get PBS to only backup the used space rather than the empty space of the drive?

Edit: I should add I am using EXT4 rather than ZFS as my filesystem.

 

[jlauro]

I didn't think you could backup pass through disks and could only backup virtual disks? Wonder if that's new?

Is vm100 your PBS server? If so, it can't backup itself.

 

[UdoB]

data drive (LUKS Encrypted)

Are you sure that LUKS does "fstrim" the underlying physical device for empty blocks?

In my (possibly wrong!) understanding the whole disk (or partition, whatever you gave to LUKS) should look like randomized(!) data. This means a) it is not clear which sector is used at all and b) it is not compressible. I do expect a backup of a LUKS device to be exactly nearly as large as the source, independent of the backup mechanism.

Just my generic two €¢...

 

[tailsofautonomy]

I didn't think you could backup pass through disks and could only backup virtual disks? Wonder if that's new?

Is vm100 your PBS server? If so, it can't backup itself.

VM100 is my OMV VM, my Proxmox Backup Server is running on bare metal and exists on a separate machine.

I used the command qm set 100 -scsi5 /dev/disk/by-id/ata-Samsung_SSD_860_QVO_1TB_S4CZNF0N592583L to get my SSD passed through to the VM and then just ticked the 'backup' box under the Hardware section and as can be seen from my log, it appears to be backing up the drive.

 

[tailsofautonomy]

Are you sure that LUKS does "fstrim" the underlying physical device for empty blocks?

In my (possibly wrong!) understanding the whole disk (or partition, whatever you gave to LUKS) should look like randomized(!) data. This means a) it is not clear which sector is used at all and b) it is not compressible. I do expect a backup of a LUKS device to be exactly nearly as large as the source, independent of the backup mechanism.

Just my generic two €¢...

That was also my initial suspicion as well but googling the issue it appeared to suggest that while LUKS disables fstrim support by default as it weakens drive encryption, it can be enabled and run if set with the correct flags (--allow discard). I tested this by running fstrim and it removed several hundred GB's from my SSD within my VM. So it appears to be working within my OMV. It just seems that Proxmox does not see this and treats the drive as if it is still full.

I also decided to confirm this issue by passing through an unencrypted hard drive I had in use on another system, running fstrim on it within OMV and running a PBS job again and it still attempted to backup the entire disk. Here is that output of that job:

INFO: starting new backup job: vzdump 100 --storage proxmox-pbs --notification-mode auto --node max --mode suspend --notes-template '{{guestname}}' --remove 0
INFO: Starting Backup of VM 100 (qemu)
INFO: Backup started at 2025-04-15 12:33:07
INFO: status = running
INFO: backup mode: suspend
INFO: ionice priority: 7
INFO: VM Name: OMV
INFO: include disk 'scsi0' 'local-lvm:vm-100-disk-0' 64G
INFO: exclude disk 'scsi1' 'local-lvm:vm-100-disk-1' (backup=no)
INFO: exclude disk 'scsi5' '/dev/disk/by-id/ata-Samsung_SSD_860_QVO_1TB_S4CZNF0N592583L' (backup=no)
INFO: include disk 'scsi6' '/dev/disk/by-id/ata-TOSHIBA_MQ04ABF100_99I4P3Q2T' 976762584K
INFO: suspending guest
INFO: creating Proxmox Backup Server archive 'vm/100/2025-04-15T11:33:07Z'
INFO: enabling encryption
INFO: started backup task 'ccaedbb0-2e48-4993-bbf2-50635af057d2'
INFO: resuming VM again after 1 seconds
INFO: scsi0: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: scsi6: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: 0% (404.0 MiB of 995.5 GiB) in 3s, read: 134.7 MiB/s, write: 125.3 MiB/s
INFO: 0% (592.0 MiB of 995.5 GiB) in 5s, read: 94.0 MiB/s, write: 76.0 MiB/s

My vm-100-disk-0, which is my OMV VM I have my drives attached to is thin provisioned, I wonder if this is causing a conflict or changing the way PBS treats my physical drive's (EXT4 formatted) even if they are not virtual drives as in the case of my VM.

 

[Taomyn]

As I understand the backups on PBS are basically disk images, and the log will always show backing up the entire disk e.g. 404.0 MiB of 995 GiB, as all my backups are the same:

INFO: 0% (680.0 MiB of 250.0 GiB) in 3s, read: 226.7 MiB/s, write: 192.0 MiB/s
INFO: 1% (2.6 GiB of 250.0 GiB) in 24s, read: 94.1 MiB/s, write: 83.2 MiB/s
INFO: 2% (5.0 GiB of 250.0 GiB) in 55s, read: 80.8 MiB/s, write: 75.0 MiB/s
INFO: 3% (7.6 GiB of 250.0 GiB) in 1m 25s, read: 88.7 MiB/s, write: 78.7 MiB/s
INFO: 4% (10.2 GiB of 250.0 GiB) in 1m 51s, read: 99.1 MiB/s, write: 86.2 MiB/s
INFO: 5% (12.6 GiB of 250.0 GiB) in 2m 22s, read: 80.1 MiB/s, write: 64.0 MiB/s
INFO: 6% (15.1 GiB of 250.0 GiB) in 2m 57s, read: 73.1 MiB/s, write: 56.0 MiB/s
INFO: 7% (17.6 GiB of 250.0 GiB) in 3m 20s, read: 113.4 MiB/s, write: 74.4 MiB/s
INFO: 8% (20.1 GiB of 250.0 GiB) in 3m 40s, read: 125.0 MiB/s, write: 79.8 MiB/s
INFO: 9% (23.0 GiB of 250.0 GiB) in 4m 3s, read: 131.8 MiB/s, write: 59.0 MiB/s
INFO: 10% (25.3 GiB of 250.0 GiB) in 4m 10s, read: 326.9 MiB/s, write: 110.3 MiB/s
INFO: 11% (27.6 GiB of 250.0 GiB) in 4m 24s, read: 167.7 MiB/s, write: 66.6 MiB/s
INFO: 12% (30.5 GiB of 250.0 GiB) in 4m 39s, read: 203.2 MiB/s, write: 94.4 MiB/s
INFO: 13% (32.5 GiB of 250.0 GiB) in 4m 56s, read: 118.8 MiB/s, write: 63.1 MiB/s
INFO: 14% (35.1 GiB of 250.0 GiB) in 5m 11s, read: 180.3 MiB/s, write: 80.3 MiB/s
INFO: 15% (38.2 GiB of 250.0 GiB) in 5m 24s, read: 239.7 MiB/s, write: 68.3 MiB/s
INFO: 16% (40.4 GiB of 250.0 GiB) in 5m 39s, read: 150.4 MiB/s, write: 58.1 MiB/s
INFO: 17% (43.3 GiB of 250.0 GiB) in 5m 49s, read: 298.0 MiB/s, write: 51.2 MiB/s
INFO: 18% (45.5 GiB of 250.0 GiB) in 5m 58s, read: 253.8 MiB/s, write: 66.2 MiB/s
INFO: 19% (47.7 GiB of 250.0 GiB) in 6m 6s, read: 283.0 MiB/s, write: 74.0 MiB/s
INFO: 20% (50.7 GiB of 250.0 GiB) in 6m 15s, read: 340.0 MiB/s, write: 50.7 MiB/s
INFO: 21% (52.7 GiB of 250.0 GiB) in 6m 21s, read: 336.0 MiB/s, write: 84.0 MiB/s
INFO: 22% (55.5 GiB of 250.0 GiB) in 6m 31s, read: 285.2 MiB/s, write: 55.6 MiB/s
INFO: 23% (57.9 GiB of 250.0 GiB) in 6m 47s, read: 151.5 MiB/s, write: 45.5 MiB/s
INFO: 24% (60.2 GiB of 250.0 GiB) in 6m 58s, read: 217.5 MiB/s, write: 68.0 MiB/s
INFO: 25% (63.0 GiB of 250.0 GiB) in 7m 9s, read: 258.9 MiB/s, write: 15.3 MiB/s
INFO: 26% (65.0 GiB of 250.0 GiB) in 7m 12s, read: 701.3 MiB/s, write: 0 B/s
INFO: 27% (68.0 GiB of 250.0 GiB) in 7m 15s, read: 1022.7 MiB/s, write: 172.0 MiB/s
INFO: 28% (70.0 GiB of 250.0 GiB) in 7m 27s, read: 169.3 MiB/s, write: 53.3 MiB/s
INFO: 29% (73.1 GiB of 250.0 GiB) in 7m 33s, read: 526.7 MiB/s, write: 67.3 MiB/s
INFO: 30% (75.1 GiB of 250.0 GiB) in 7m 41s, read: 262.0 MiB/s, write: 35.0 MiB/s
INFO: 31% (77.6 GiB of 250.0 GiB) in 7m 56s, read: 166.9 MiB/s, write: 44.8 MiB/s
INFO: 32% (80.8 GiB of 250.0 GiB) in 7m 59s, read: 1.1 GiB/s, write: 80.0 MiB/s
INFO: 33% (82.6 GiB of 250.0 GiB) in 8m 8s, read: 208.9 MiB/s, write: 31.6 MiB/s
INFO: 34% (85.0 GiB of 250.0 GiB) in 8m 16s, read: 309.0 MiB/s, write: 83.0 MiB/s
INFO: 35% (87.7 GiB of 250.0 GiB) in 8m 40s, read: 114.7 MiB/s, write: 68.8 MiB/s
INFO: 36% (90.2 GiB of 250.0 GiB) in 8m 51s, read: 236.0 MiB/s, write: 18.5 MiB/s
INFO: 37% (92.6 GiB of 250.0 GiB) in 9m 5s, read: 171.4 MiB/s, write: 50.6 MiB/s
INFO: 38% (95.6 GiB of 250.0 GiB) in 9m 39s, read: 90.8 MiB/s, write: 49.5 MiB/s
INFO: 39% (97.5 GiB of 250.0 GiB) in 10m, read: 93.7 MiB/s, write: 38.7 MiB/s
INFO: 40% (100.1 GiB of 250.0 GiB) in 10m 11s, read: 237.5 MiB/s, write: 83.3 MiB/s
INFO: 41% (102.6 GiB of 250.0 GiB) in 10m 23s, read: 213.7 MiB/s, write: 121.7 MiB/s
INFO: 44% (111.7 GiB of 250.0 GiB) in 10m 26s, read: 3.1 GiB/s, write: 0 B/s
INFO: 49% (122.7 GiB of 250.0 GiB) in 10m 29s, read: 3.7 GiB/s, write: 0 B/s
INFO: 53% (133.4 GiB of 250.0 GiB) in 10m 32s, read: 3.6 GiB/s, write: 0 B/s
INFO: 57% (144.6 GiB of 250.0 GiB) in 10m 35s, read: 3.7 GiB/s, write: 0 B/s
INFO: 61% (154.5 GiB of 250.0 GiB) in 10m 38s, read: 3.3 GiB/s, write: 0 B/s
INFO: 66% (165.7 GiB of 250.0 GiB) in 10m 41s, read: 3.7 GiB/s, write: 0 B/s
INFO: 70% (176.5 GiB of 250.0 GiB) in 10m 44s, read: 3.6 GiB/s, write: 0 B/s
INFO: 74% (187.3 GiB of 250.0 GiB) in 10m 47s, read: 3.6 GiB/s, write: 0 B/s
INFO: 79% (198.6 GiB of 250.0 GiB) in 10m 50s, read: 3.8 GiB/s, write: 0 B/s
INFO: 83% (209.4 GiB of 250.0 GiB) in 10m 53s, read: 3.6 GiB/s, write: 0 B/s
INFO: 88% (220.6 GiB of 250.0 GiB) in 10m 56s, read: 3.7 GiB/s, write: 0 B/s
INFO: 92% (231.7 GiB of 250.0 GiB) in 10m 59s, read: 3.7 GiB/s, write: 0 B/s
INFO: 97% (242.6 GiB of 250.0 GiB) in 11m 2s, read: 3.6 GiB/s, write: 0 B/s
INFO: 100% (250.0 GiB of 250.0 GiB) in 11m 5s, read: 2.5 GiB/s, write: 5.5 MiB/s

But I then look at the end of each line and see that the write value is what is actually being backed up. I'm pretty sure if it was always writing the entire disk I'd have issues with my PBS storage being unable to hold all the backups. It's difficult to tell if you see the same as you have truncated your example logs.

When I've restored disks I see a similar log and I some lines take time as the data is read then written, but many where they go much quicker presumably because there is no actual data to restore for that part of the disk.

Of course, I could be completely wrong, but your thread peaked my interest.

 

[SteveITS]

@tailsofautonomy
Just to clarify how (I understand) PBS works it will see and show the entire disk but the empty chunks are deduplicated down to one. Every backup is a full backup.

Despite having “several TB” of disk in our cluster, our PBS is using under 350 GB so far.

So if we ignore the disk size shown what is the actual problem? PBS is out of space? Can you show that failure?

 

[UdoB]

It just seems that Proxmox does not see this and treats the drive as if it is still full.

Okay.

Probably my post regarding "fstrim" was completely stupid - that's one level lower than backup handles data. Sorry for the confusion.

@Taomyn 's #6 makes more sense

 

[jimbojeff]

Hello everyone. I have a similar problem. I'm currently backing up my OMV VM with over 12 TB of storage space. Apparently, Proxmox is backing up the entire storage space (including the free space). My hard drive is almost empty (only 250 GB used):





dose know sombody why?

 

[SteveITS]

This is normal and the space used by the backup is much less even though it will show 12.8 TB used for every backup.

Note PBS will deduplicate across backups. And VMs.

 

[jimbojeff]

This is normal and the space used by the backup is much less even though it will show 12.8 TB used for every backup.

Note PBS will deduplicate across backups. And VMs.

Thanks. But this take to long. Is taking to long. Is that normal? Is there another way to make a backup, so that it doesn't take so long?

 

[Onslow]

Unfortunately you haven't given us all the details - the screenshot doesn't show all the options.
You'd better posted the textual form of the data (in the CODE tags - clicked < / > in the menu above and pasted the text).

Anyway, you seem to backup in mode "stop". Is there any specific reason that the VM is stopped during the backup?
And: is this the first backup of this VM, or a subsequent?

 

[jimbojeff]

Unfortunately you haven't given us all the details - the screenshot doesn't show all the options.
You'd better posted the textual form of the data (in the CODE tags - clicked < / > in the menu above and pasted the text).

Anyway, you seem to backup in mode "stop". Is there any specific reason that the VM is stopped during the backup?
And: is this the first backup of this VM, or a subsequent?

Header
Proxmox
Virtual Environment 9.0.11
Search
Virtual Machine 102 (omv) on node 'proxmox' (backup)
No Tags
Type 'help' for help.
# /
unknown command: ''
Server View
Logs
()
INFO: starting new backup job: vzdump 102 --storage proxmox.bu --node proxmox --remove 0 --notification-mode notification-system --notes-template '{{guestname}}' --compress zstd --mode stop
INFO: Starting Backup of VM 102 (qemu)
INFO: Backup started at 2025-11-02 15:03:08
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: VM Name: omv
INFO: include disk 'scsi0' 'local-lvm:vm-102-disk-0' 64G
INFO: include disk 'scsi1' '/dev/disk/by-id/ata-ST14000NM001G-2KJ103_ZL26NCJS' 13039G
INFO: creating vzdump archive '/mnt/pve/proxmox.bu/dump/vzdump-qemu-102-2025_11_02-15_02_48.vma.zst'
INFO: starting kvm to execute backup task
INFO: started backup task 'de7f7acc-5fb8-4cd2-8c36-ccdffe5f4c70'
INFO:   0% (735.9 MiB of 12.8 TiB) in 3s, read: 245.3 MiB/s, write: 184.1 MiB/s
INFO:   1% (131.0 GiB of 12.8 TiB) in 21m 1s, read: 106.1 MiB/s, write: 99.9 MiB/s
INFO:   2% (262.2 GiB of 12.8 TiB) in 43m 52s, read: 97.9 MiB/s, write: 88.9 MiB/s
INFO:   3% (393.1 GiB of 12.8 TiB) in 1h 6m 7s, read: 100.4 MiB/s, write: 80.5 MiB/s
INFO:   4% (524.2 GiB of 12.8 TiB) in 1h 20m 28s, read: 155.9 MiB/s, write: 89.5 MiB/s
INFO:   5% (655.2 GiB of 12.8 TiB) in 1h 44m 35s, read: 92.7 MiB/s, write: 84.0 MiB/s
INFO:   6% (786.2 GiB of 12.8 TiB) in 2h 6m 24s, read: 102.4 MiB/s, write: 93.7 MiB/s
INFO:   7% (917.2 GiB of 12.8 TiB) in 2h 31m 3s, read: 90.7 MiB/s, write: 84.5 MiB/s
INFO:   8% (1.0 TiB of 12.8 TiB) in 2h 55m 42s, read: 90.8 MiB/s, write: 81.0 MiB/s
INFO:   9% (1.2 TiB of 12.8 TiB) in 3h 10m 44s, read: 148.9 MiB/s, write: 66.4 MiB/s
INFO:  10% (1.3 TiB of 12.8 TiB) in 3h 31m 18s, read: 108.6 MiB/s, write: 82.1 MiB/s
INFO:  11% (1.4 TiB of 12.8 TiB) in 3h 53m 37s, read: 100.2 MiB/s, write: 92.6 MiB/s
INFO:  12% (1.5 TiB of 12.8 TiB) in 4h 11m 59s, read: 121.8 MiB/s, write: 112.8 MiB/s
INFO:  13% (1.7 TiB of 12.8 TiB) in 4h 33m 18s, read: 104.8 MiB/s, write: 98.5 MiB/s
INFO:  14% (1.8 TiB of 12.8 TiB) in 4h 53m 17s, read: 111.9 MiB/s, write: 102.5 MiB/s
INFO:  15% (1.9 TiB of 12.8 TiB) in 5h 10m 46s, read: 127.9 MiB/s, write: 118.3 MiB/s

 

[jimbojeff]

yes this is my first backup. Could I create a snapshot? Would that be sufficient if I wanted to restore my VM (completely but without my data on HDD "scsi1")?

 

[Onslow]

yes this is my first backup.

I asked because I've understood from the thread context that you too are using Proxmox Backup Server (now I'm not so sure).
And usually subsequent backups of running VMs, executed in "snapshot" mode, last substantially shorter. At least backups to PBS (I don't observe this speedup when creating backups to local storage).
This speedup is thanks to "dirty-bitmap" because of it the backup process knows which data have changed since the previous backup.
So, paradoxically, backup of a running VM can be much faster than of a stopped one.

I'll give an example from my server.
Creating a subsequent daily backup of a running VM takes 2 minutes.
Creating a subsequent daily backup of the stopped clone of the same VM takes more than 5 minutes. Despite of the fact, that the data in the running VM is changing all the time and the stopped clone is stopped for many days, so in the summary this is clearly written:
"reused xxxx GiB (100%). And all "write: " entries are "0 B/s" - because there were no writes. But the process had to read all the data, having no dirty-bitmap to help it.

Could I create a snapshot?

Yes.

Would that be sufficient if I wanted to restore my VM (completely but without my data on HDD "scsi1")?

No.

"Snapshot mode" in backup is about the way a backup is made.
This is not the same thing as snapshot.

Edit: returning to your last question. The precise answer could be "it depends". On what happened before restoring.
Anyway, I'm not sure if a snapshot without "scsi1" would work.

The secure approach is taking into consideration the fact that "a snapshot is not a backup!"

 

[jimbojeff]

I asked because I've understood from the thread context that you too are using Proxmox Backup Server (now I'm not so sure).
And usually subsequent backups of running VMs, executed in "snapshot" mode, last substantially shorter. At least backups to PBS (I don't observe this speedup when creating backups to local storage).
This speedup is thanks to "dirty-bitmap" because of it the backup process knows which data have changed since the previous backup.
So, paradoxically, backup of a running VM can be much faster than of a stopped one.

I'll give an example from my server.
Creating a subsequent daily backup of a running VM takes 2 minutes.
Creating a subsequent daily backup of the stopped clone of the same VM takes more than 5 minutes. Despite of the fact, that the data in the running VM is changing all the time and the stopped clone is stopped for many days, so in the summary this is clearly written:
"reused xxxx GiB (100%). And all "write: " entries are "0 B/s" - because there were no writes. But the process had to read all the data, having no dirty-bitmap to help it.


Yes.

No.

"Snapshot mode" in backup is about the way a backup is made.
This is not the same thing as snapshot.

Edit: returning to your last question. The precise answer could be "it depends". On what happened before restoring.
Anyway, I'm not sure if a snapshot without "scsi1" would work.

The secure approach is taking into consideration the fact that "a snapshot is not a backup!"

Thank you very much. I don't actually use a Proxmox backup server. I use a local directory on my PC as the backup directory in Proxmox via an SMB service.