Slow backups from NFS share

[decibel83]

Hi,
I'm using Proxmox Backup Client on a Debian virtual machine on Proxmox VE to backup a quite large NFS share (~ 10 Tb) mounted on the VM.

The backup works, but it's very slow.
First backup took a very long time (~ 12 days) and it's expected, but even next backups are taking so long: at the moment the second run is running until 11 days:

2021-06-07T10:54:46+02:00: starting new backup on datastore 'dc-backup': "host/backup/2021-06-07T08:54:45Z"
2021-06-07T10:54:46+02:00: download 'index.json.blob' from previous backup.
2021-06-07T10:54:46+02:00: created new dynamic index 1 ("host/backup/2021-06-07T08:54:45Z/catalog.pcat1.didx")
2021-06-07T10:54:46+02:00: register chunks in 'storage1-nfsshare.pxar.didx' from previous backup.
2021-06-07T10:54:46+02:00: download 'storage1-nfsshare.pxar.didx' from previous backup.
2021-06-07T10:54:54+02:00: created new dynamic index 2 ("host/backup/2021-06-07T08:54:45Z/storage1-nfsshare.pxar.didx")

I'm expecting an incremental/differential backup, but I don't see so much time differences between the first and the following backups.

This is the command I used on Proxmox Backup Client to backup the NFS mounted share:

root@backup:/mnt# PBS_PASSWORD=password proxmox-backup-client backup storage1-nfsshare.pxar:/mnt/nfsshare --repository dc-backup@pbs@pbs-server
:8007:dc-backup --entries-max 40000000
Starting backup: host/backup/2021-06-07T08:54:45Z
Client name: backup
Starting backup protocol: Mon Jun  7 10:54:45 2021
Downloading previous manifest (Thu May 13 09:36:53 2021)
Upload directory '/mnt/nfsshare' to 'dc-backup@pbs@nfsshare:8007:dc-backup' as storage1-nfsshare.pxar.didx

Are I am doing something wrong or this is expected?

Is there any way to speed up the backup?

Thank you very much!
Bye

 

[dietmar]

I'm expecting an incremental/differential backup, but I don't see so much time differences between the first and the following backups.

Proxmox backup always need to read all container data, so that is expected behavior.

 

[decibel83]

Proxmox backup always need to read all container data, so that is expected behavior.

11 days and this is expected?
Really??

if there is no way to speed up the backup, this means that Proxmox Backup Server is not usable for other things than VMs :-(

 

[dietmar]

11 days and this is expected?

Well, your setup is probably the worst I can imagine for proxmox backup. You send data over the network multiple times, introducing high io delays. Maybe you can run proxmox-backup-client directly on the NFS server?

Also, what network cards/speed to you use (10GB)?

 

[decibel83]

Well, your setup is probably the worst I can imagine for proxmox backup. You send data over the network multiple times, introducing high io delays. Maybe you can run proxmox-backup-client directly on the NFS server?

Also, what network cards/speed to you use (10GB)?

Yes, it's a slow network because for now I'm backing up these data on a remove PBS Server, so the uplink is 200 Mbit/sec.

I know this is not a good situation, but I'm wondering why PBS should transfer the whole data everytime.
These data are incremental and does not change so much, so an incremental or differential backup should be good.
Uploading VM images are incremental, so I was thinking that data from Proxmox Backup Client is incremental too, but I understand it is not.

Am I wrong?
Thank you!

 

[dcsapak]

Uploading VM images are incremental, so I was thinking that data from Proxmox Backup Client is incremental too, but I understand it is not.

the upload of chunks is incremental, but not the reading of the files. for a file-based backup all files must be read in whole