Any Hints why my incremental Backups are way too large ?

adoII

Active Member
Jan 28, 2010
164
13
38
We are still evaluating pbs Server and begin to like it more and more.
One problem ist that on some Servers e.g. mailserver or lamp server the incremental size ist unbevlievable high. An incremental after 24 hours is about 50% of the initial Backup size. But with a find-command run i find only few changed files.
I know, dirty bitmaps don't lie but what in hell is changing on the disks ? I am sure there is no trim job running and the mounts are noatime.
Do you have any hints what i can do on the virtual machines to reduce number of changed blocks ? Maybe there are mount options for ext4 or maybe ext4 as a copy on write filsystem is ineffective for incremental Backups ? Or Disks which are more than 90% full tend to change more blocks than emptier disks ?
Any Ideas what I can try and where I can search ?
 

mira

Proxmox Staff Member
Staff member
Aug 1, 2018
1,623
154
68
Please provide the output of pveversion -v from your PVE node where the VM is running, the output of proxmox-backup-manager versions --verbose and the complete task log of the backup.
 

adoII

Active Member
Jan 28, 2010
164
13
38
I attach the output as a file.
Okay I see it is 123GB changed and 543GB unchanged. For me this still seems way to large,
 

Attachments

  • upload.txt
    11 KB · Views: 17

mira

Proxmox Staff Member
Staff member
Aug 1, 2018
1,623
154
68
So the last 3 disks have (close to) no changes. I guess the first disk is your root?
What's running in the VM?

The way the dirty bitmaps work, is by marking a whole block dirty even if just a single bit in that block has changed. So if the writes are small but in a lot of different blocks, they all will be marked dirty.
 

adoII

Active Member
Jan 28, 2010
164
13
38
Hmm, let's look at scsi0 80 GB Disk.
it constantly only has 20 GB in use and 60 GB free.
So it looks strange that 40GB change within a day
It is an ext3 filesystem.
I checked all cronjobs and logs an i have no idea what si happening ...
 

adoII

Active Member
Jan 28, 2010
164
13
38
I think the opposite.
If i run fstrim on an 80GB drive with only 20GB of data it will lead to 60GB changed data.
If I run fstri again it will lead to another 60GB of trimmed data.
 
Feb 20, 2021
70
5
8
https://pve.proxmox.com/wiki/Shrink_Qcow2_Disk_Files

yes, it will change blocks, but it will mark the blocks as empty: this should reduce the backup size.

maybe "iotop" is your friend, for more details use "fatrace". disk writes heavily depend on the services running on it. ext4 uses journaling as most databases do, too which very likely causes a lot of changes to disk.

does a swap partion on this disk exist?
is a database running which does a lot of updates/changes?
is php running and creating (+deleting) a lot of session files?
are lvm snapshots or filesystem snapshots beeing used?
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!