Can two PBS instances share same datastore?

[trmg]

Hello!

I have two locations that are geographically distant from one another where PVE and PBS (to back up the local PVE environment) is running. I have the opposing PBS instances configured with a separate datastore with a sync job so that the backups for location A periodically sync to location B and vice versa. This has been working really well, but I am wondering if this is necessary?

Is it possible for the two PBS instances to share the same datastore and sync across? Since as far as I understand deduplication is across the entire datastore, this could potentially reduce my total backup storage footprint.

I'm not sure if the way the datastore is structured compared to how the sync function works supports a "bi-directional" sync. My initial thought is no (and makes sense), but thought, other than looking silly, it wouldn't hurt to ask.

Thanks!

 

[Dunuin]

I'm not sure if the way the datastore is structured compared to how the sync function works supports a "bi-directional" sync.

A sync task is always pulling.

Is it possible for the two PBS instances to share the same datastore and sync across? Since as far as I understand deduplication is across the entire datastore, this could potentially reduce my total backup storage footprint.

For both PBS instances to share the same datastore you would need some kind of shared storage (like a NFS share) but this would be a terrible idea as PBS needs IOPS peformance so you want local storage and not the big latency over the internet.
Whats possible is to use a single datastore per PBS and then split that by using two namespaces and sync one namespace from A to B and the other one from B to A.
This is how I do it here. And it's a good idea to store everything twice. That way you got a offsite backup, fast restore from the local PBS and in case a chunk corrupts (making all backup snapshots of a VM invalid) you can repair that chunk with the healty offsite copy.
And as both namespaces share the same datastore it will be covered by duduplication which is indeed per datastore.

 

[trmg]

A sync task is always pulling.

Yeah, which is where I was thinking if they were cross-pulling into the same data-store things could go wrong.

For both PBS instances to share the same datastore you would need some kind of shared storage (like a NFS share) but this would be a terrible idea as PBS needs IOPS peformance so you want local storage and not the big latency over the internet.

Yep yep this is precisely why I have a local PBS instance handle backups for the local environment then sync across to the remote PBS instance!

Whats possible is to use a single datastore per PBS and then split that by using two namespaces and sync one namespace from A to B and the other one from B to A.
This is how I do it here. And it's a good idea to store everything twice. That way you got a offsite backup, fast restore from the local PBS and in ase a chunk corrupts (making all backup snapshots of a VM invalid) you can repair that chunk with the healty offsite copy.
And as both namespaces share the same datastore it will be covered by duduplication which is indeed per datastore.

Oooooooh I will definitely need to give this a try. Currently at both sites backups are simply to the root namespace. Just so I'm understanding correctly, here's what I'm thinking:

Handwave over the technical details, but basically....

Site A will store all its backups under the "Site A" namespace.
Site B will store all its backups under the "Site B" namespace.

For Site A PBS, the sync job will sync the Site B namespace to the *same* datastore in which the Site A namespace resides.
For Site B PBS, the sync job will sync the Site A namespace to the *same* datastore in which the Site B namespace resides.

I'll need to look into the best method of moving the backups from the root namespace. Skimming this thread it seems either a sync job on the local instance or creating a directory within the datastore, making sure it's owned by the correct user and group and has the correct permissions, and then moving the backup directory structure (groups) into the newly created directory.

 

[Dunuin]

Site A will store all its backups under the "Site A" namespace.
Site B will store all its backups under the "Site B" namespace.

For Site A PBS, the sync job will sync the Site B namespace to the *same* datastore in which the Site A namespace resides.
For Site B PBS, the sync job will sync the Site A namespace to the *same* datastore in which the Site B namespace resides.

Yes.

So:
PBS A
--- Datastore A
------- Namespace A1
------- Namespace A2

PBS B
--- Datastore B
------- Namespace B1
------- Namespace B2

PVE A backups to namespace A1 on PBS A.
PVE B backups to namespace B2 on PBS B.
PBS A pulls from namespace B2 to namespace A2.
PBS B pulls from namespace A1 to namespace B1.

I would also suggest to use different users. A restore-only users that is only allowed to restore from all 4 namespaces but isn't allowed to prune or create backups. And another user that is allowed to restore from all 4 namespaces and to create backups to namesapce A1 and B2. That way no PVE is allowed to delete or overwrite the synced namespaces (B1 and A2) so you got ransomware protection when you do the pruning on the PBS itself instead pruning via backup job and you don't set the "removed vanished" checkbox of the sync job. Then there is no way to alter the synced namespaces unless you do that via the PBS webUI/API.

I'll need to look into the best method of moving the backups from the root namespace. Skimming this thread it seems either a sync job on the local instance or creating a directory within the datastore, making sure it's owned by the correct user and group and has the correct permissions, and then moving the backup directory structure (groups) into the newly created directory.

Yes, proper way would be to set localhost as a remote and then use that to do a local sync.

 

[trmg]

Cool, that matches my understanding which is good as I already started juggling things around, heh.

The amount of free disk space on one of the hosts was looking to be a bit tight if I did things the most proper way and did a sync, assuming a sync would result in a complete second copy of the datastore. So what I did was I created the namespaces via the PBS web UI then then via the CLI moved the ct and vm directories to within the appropriate namespace. Then on the PVE hosts added another PBS storage device that pointed to the appropriate namespace on its local PBS instance and adjusted the backup jobs appropriately. I also adjusted the sync jobs on the PBS instances and they seem happy as well.

It seems to have worked as I can browse backups, sync jobs seemed to be happy, I can perform file level restores and full restores, and can perform backups.

As for user permissions, there is definitely room for improvement in my setup here. I did create a user for use with the PBS storage connectivity on the PVE side so that it only has permission to the needed namespace using the built-in role of DatastoreBackup, but that can likely be further improved. I for sure need to lock down the sync jobs better though which is on my list of todos.