Guidance Needed to Improve Sync Performance on 10 Mbps Link

T

totae

Member
May 27, 2023
22
1
8
Hello,

I am experiencing a network bottleneck issue.

The network bandwidth between our Main site and DR site is limited to 10 Mbps. I understand this is quite slow, but we are unable to increase the bandwidth at this time due to cost constraints.

As a result, the synchronization jobs take a very long time to complete. I have already tried separating the sync tasks, but it did not improve the performance.

Could you please suggest any methods or best practices to improve synchronization speed under these limitations?

Example : seeding function on Veeam
https://helpcenter.veeam.com/docs/vbr/userguide/replica_seeding.html?ver=13

Thank you for your support.

Code: 
proxmox-backup: 4.0.0 (running kernel: 6.14.11-4-pve)
proxmox-backup-server: 4.0.16-1 (running version: 4.0.16)
proxmox-kernel-helper: 9.0.4
proxmox-kernel-6.14.11-4-pve-signed: 6.14.11-4
proxmox-kernel-6.14: 6.14.11-4
proxmox-kernel-6.14.8-2-pve-signed: 6.14.8-2
ifupdown2: 3.3.0-1+pmx10
libjs-extjs: 7.0.0-5
proxmox-backup-docs: 4.0.16-1
proxmox-backup-client: 4.0.16-1
proxmox-mail-forward: 1.0.2
proxmox-mini-journalreader: 1.6
proxmox-offline-mirror-helper: 0.7.2
proxmox-widget-toolkit: 5.0.6
pve-xtermjs: 5.5.0-2
smartmontools: 7.4-pve1
zfsutils-linux: 2.3.4-pve1
 
Is it about the first initial sync or about later syncs?
 
Okay. For the initial sync, you could have either used a "Removable Datastore", sync to it, move it to the remote location, add it there as well and then sync from it to the local remote datastore. https://pbs.proxmox.com/docs/storage.html#removable-datastores

Or, the harder way, manually copy the full datastore directory (including hidden files and directories, starting with a ., e.g. the .chunks dir) to an external disk and seed the remote location that way.

But once the initial sync is done, there is not much more to optimize since only new chunks and the backup metadata of new backup snapshots will be transferred.
 
Thank you for the update.

Regarding the Removable Datastore method, this may not be suitable for our infrastructure because our storage is local (different storage on each site) and we perform synchronization daily.

If we synchronize the /. and /.chunks directories, will this synchronize all Groups/VMs?
Currently, we use filtered sync for specific Groups/VMs to reduce the amount of data transferred.

Best Regards,
 
You must log in or register to reply here.
Top Bottom