Remotes & sync: pre-seeding remote datastore?

[rainyJune]

We are establishing a remote with sync jobs to our existing PBS instance which has some history of snapshots & approx 10TB of data on disk. Our remote site has only gigabit IP connectivity some of which is reserved during business hours.

Rather than waiting days for our first sync to complete, is it possible to pre-seed the remote datastore by copying the contents of the datastore directory onto the remote filesystem rather than transferring it over the network? (by installing all drives within a single machine)

 

[fabian]

yes. one way to do it would be to create the datastore on the new disks, then copy the chunk store contents into its chunk store, transfer the new disks to the new server, manually create the datastore.cfg entry and then start the sync. just be careful to wait for a full sync to finish before setting up garbage collection the sync should be rather fast, since it would only transfer the index files/manifest/logs (snapshot metadata) and any new backup snapshot made in the meantime.

 

[rainyJune]

fantastic, thanks Fabian. We'll go ahead with that.

 

[fabian]

just be sure that the directory structure of the original and new datastore match - the chunks are in subdirs of .chunks in the datastore, and need to follow that exact scheme for them to be found and re-used!

 

[rainyJune]

will do. is there support for a dry-run on the sync job, printing delta sizes before we kick off the job for real as there is with proxmox-backup-client?

very happy customer here, PBS is a great tool even independent of PVE (and especially with PVE).

 

[fabian]

no there isn't (yet - it would be possible to implement though).

if you attach both disks to the same system (or within a well connected, local network) you could do a real sync as well (e.g. by setting up a remote pointing to localhost), if you repeatedly run the sync it will give you an idea of the delta. obviously syncing has more overhead than a plain file copy, so you still might want to do a manual chunk preseed in that case.