Proxmox Backup Client extremely slow on large Mailservers ?

[adoII]

I have a Mailserver with 500GB of E-Mails in 40 Mio Files in Maildir Format.
When I backup the Mailserver with traditional tar/amanda it takes around 35 Minutes to do an incremental file backup the Server
When I backup the server with proxmox backup client it takes hours to backup the mailserver. After 4 hours the backup is till running. It seems even that the differential backup takes a lot more time than the original initial backup.
Are there any ideas how I could speed up file backups on Maildirectories with proxmox-backup-client ?

 

[DerDanilo]

Same problem here. We have about 1TB of mails and millions of files. It takes about 4h.

Source is ceph nvme, target PBS with NFS share. The Pbs client, the Pbs server and the NFS storage, as well as ceph are bored. There must be a way to speed things up and use available resources.

Any ideas?

 

[scosol]

"The Pbs client, the Pbs server and the NFS storage, as well as ceph are bored" When you say they are "bored" what does that mean exactly? My observation is that the client seems to be single-threaded (at least in its compress/encrypt operations) and so one CPU core is my bottleneck. I am (vary naively) assuming that the client is first performing a full scan of the source mount to determine what has changed. This equates to a full read of all that data, and since its a mailserver, many MANY small read and node-level ops. There isn't really any way to "speed this up" unless that requirement becomes... no longer a requirement. You are essentially trying to backup a db, and you're treating it like a filesystem. Since it's a (presumably) active mailstore, why are you performing an incremental at all?
...
@adoll you say you've got an existing "tar+amanda" workflow, that is what you both should be doing (then use the pbs-client to backup those tars).
Remember, your mailstore is essentially a constantly active-db that doesn't require global atomicity in its state- (with no other fs-level "snapshot" capabilities available) my off-the-cuff approach would be to tar+gz up groups of users' mail at a time on an interval to create larger chunks in to a tmpstore, and then run the backup client against that tmpstore at some other interval. ... Prune as needed, and "user x's mail should be in one of your backups" is achieved.

 

[DerDanilo]

Maybe this makes sense, somehow. But with many thousand of mail domains this is not really a practical solution to split things into TARs if it comes to a full backup for desaster recovery and maybe also selective restoration of mails for a specific domain.

The PBS client could also support parallel processing (one backup task) to allow reading way faster from scale out storage systems (ceph e.g.). By default it could use a single core/thread, but it should support to utilized more system resources if explicitely requested.
On some systems the "old" NFS share is beeing used for PBS as storage backend. That's why I mentioned it.

 

[Audifilm]

You could do a full backup every week or month using proxmox-backup-client, and combine it with a rsync proces that creates a separate folder with a copy of "changed files in the last X days", and made then also a backup of that using proxmox-backup-client.
So you will have 2 backups and select the most apropiate.
Then you can prune or delete differential (rsync created) copies, when a new full backup is done.

You can use something like that, to create a folder with a copy of files modified in the last day, that then you will backup much faster:
https://unix.stackexchange.com/questions/173958/find-files-newer-than-a-day-and-copy

export DAYS=1
export BASE_DIR=/mnt/data (sample, folder with lot of files and TBs that I full copied previously, and I want to copy incrementally)
export DATE_DIR=PBS_$(date "+%Y-%m-%d")
mkdir -p "$BASE_DIR/$DATE_DIR" (new folder that will contain only a copy of new files)
cd $BASE_DIR
find . -type f -mtime -$DAYS -not \( -path ./$DATE_DIR\* -prune \) -exec rsync --relative {} $BASE_DIR/$DATE_DIR \;
proxmox-backup-client backup sample.pxar:$BASE_DIR/$DATE_DIR --backup-id 'sample-incremental-1day'