How to build a new datastore storage, and push existing backups back and forth

[sherminator]

Hi everybody,

we must build a new storage for our backups (see https://forum.proxmox.com/threads/zfs-raidz1-expanding-not-possible.127480), so I would like to copy our backups (about 30 TB) to another storage, build a new ZFS pool, and put the backups back.

I successfully rsync'd /mnt/datastore to another storage with

rsync -avP <source> <target> --delete

. My idea which I would like to bring up for discussion is:

1. stop PBS services (which services do I need to stop? How do PBS clients behave when there's no server for a couple of hours?)
2. run a (further) rsync
3. backup the current datastore configuration (do I have to care for namespace configuration?)
3. remove the current datastore
4. destroy the ZFS pool
5. create a new ZFS pool
6. restore the datastore configuration
7. rsync all backups to the new datastore
8. start PBS services

Thanks and greets
Stephan

 

[Scynk159]

If you have the opportunity, in my opinion the simplest option would be:

1. Setup the second PBS server
2. Use Remote-Sync to transfer backups/configuracion/namespace
https://pbs.proxmox.com/docs/managing-remotes.html

This way you can set vm filters to sync it partially (e.x. few vm per day/night), or set network bandwith limit, to not overload production network. And the main PBS stay online in this time.

You can also check maintenance mode on datastore (it might be helpful in other scenarios):
https://pbs.proxmox.com/docs/maintenance.html#maintenance-mode

As far as I know, if PBS is unavailable and PVE uses it as storage, nothing bad will happen. Storage will be unavailable and that's it. It automatically reactivates when PBS go online.

 

[sherminator]

Hey,

thanks for your reply, and sorry for my late reaction!

1. Setup the second PBS server
2. Use Remote-Sync to transfer backups/configuracion/namespace

I tried this, but unfortunately the second machine has just a bunch of HDDs, and so it's too slow for PBS (syncing at about 50 MB/s). With rsync I get about 300 MB/s, so I would like to stick at this path.

So again: What's the clean way to stop PBS?
What I successfully tried in the meantime: All you need to "transfer" a datastore manually is the content of the datastore itself, and /etc/proxmox-backup/datastore.cfg - so cool!

 

[krikey]

Sherminator,

Is the reason PBS remote-sync is slow because it's handling a lot of comparison data on the HDDs before any real data is actually sent? I ask as I have a relatively slow lab based PBS test I'm currently running.

 

[sherminator]

Is the reason PBS remote-sync is slow because it's handling a lot of comparison data on the HDDs before any real data is actually sent?

I don't know for sure, but yes, I assume that there is a lot of "small block size" reading before data is transfered,