[SOLVED] Need some help with remotes and sync jobs

[Ovidiu]

Hi there,

I have 1 PVE and 2 PBS instances. PBS-A is installed on the same host as PVE while PBS-B is on its own host (remote).
I am doing backups to PBS-A and keep them only for a short term.
PBS-B is meant to add PBS-A as a remote and then sync backups and keep them long term.

What works:
- PVE backs up to PBS-A
- created an API token on PBS-A
- gave API token RemoteSyncOperator permissions on /datastore/local
- added PBS-A as remote on PBS-B by using "user@pam!token-name" as "Auth ID" and the secret that was generated when creating the API token

Where the first problem happens
- on PBS-B trying to list the remotes I get this error:

proxmox-backup-manager remote list
unable to format result: property userid does not exist in schema.

What exactly is this error trying to tell me?

 

[oversite]

Hi there,

I have 1 PVE and 2 PBS instances. PBS-A is installed on the same host as PVE while PBS-B is on its own host (remote).
I am doing backups to PBS-A and keep them only for a short term.
PBS-B is meant to add PBS-A as a remote and then sync backups and keep them long term.

What works:
- PVE backs up to PBS-A
- created an API token on PBS-A
- gave API token RemoteSyncOperator permissions on /datastore/local
- added PBS-A as remote on PBS-B by using "user@pam!token-name" as "Auth ID" and the secret that was generated when creating the API token

Where the first problem happens
- on PBS-B trying to list the remotes I get this error:

proxmox-backup-manager remote list
unable to format result: property userid does not exist in schema.

What exactly is this error trying to tell me?

something is wrong with that command, it looks like a bug to me, it should give you the list of added remotes

 

[Ovidiu]

well, let say the command has a bug, I am still having the problem that after adding the remote I am trying to setup a sync job.

So I add the job on PBS-B, select the source remote and then when I try to select the source datastore and click the drop-down, instead of it showing me the remote datastores available on PBS-A I see an empty drop-down.

Next I tried doing it via cli as explained here: http://my-pbs:8007/docs/managing-remotes.html#syncjobs and I could create the sync job but when trying to run it I get:

2021-01-17T23:05:10+01:00: TASK ERROR: permission check failed

There clearly is something wrong with some permission.

 

[fabian]

you misunderstood the permissions. you need

token@PBS-A: read permission to source datastore on PBS-A
setup@PBS-B: write permission to remote config, write permission to target datastore on PBS-B
owner@PBS-B: operator permission on remote/datastore combination, write permission to target datastore on PBS-B

on PBS-A, your token needs to have some sort of read access to the datastore, not RemoteSyncOperator. keep in mind that with the token you can only sync what the token can read.

on PBS-B, you need to setup a sync job. the user setting that up and the owner that is configured in the sync job need to have access to the remote/datastore combination (e.g., RemoteSyncOperator for the owner, more for the account doing the setup). they also need write access to the target datastore on PBS-B (since the sync will create new backup groups/snapshots, the same roles used for making regular backups are used here). if you want to set the remove_vanished option, those users also need to have the prune privilege. for your use case, you problably don't want to set remove_vanished but control pruning on PBS-B independently from PBS-A.

the token on PBS-A and the owner and setup users on PBS-B are not related in any way. you can have many sync jobs using a single remote.cfg entry.

 

[Ovidiu]

thanks @fabian that sounds good but I am still confused. For one, where does the token on PBS-A get used? I mean if I set up the token on PBS-A that token must somehow be used on PBS-B as PBS-B is the one where the sync job runs.

So when adding PBS-A on PBS-B as remote, what credentials do I supply? I assumed that I would have to use the token from PBS-A?

Any chance you have a link to a how-to or more detailed step by step explanation?

Meanwhile I'll be searching this forum to see if I can find any useful threads.

 

[fabian]

thanks @fabian that sounds good but I am still confused. For one, where does the token on PBS-A get used? I mean if I set up the token on PBS-A that token must somehow be used on PBS-B as PBS-B is the one where the sync job runs.

So when adding PBS-A on PBS-B as remote, what credentials do I supply? I assumed that I would have to use the token from PBS-A?

Any chance you have a link to a how-to or more detailed step by step explanation?

Meanwhile I'll be searching this forum to see if I can find any useful threads.

I'll give you a short step-by-step.

on source instance (PBS-A in your case) as user with admin privileges:
- setup user (+ optionally token) if it doesn't already exist => let's call this sync@pbs
- give sync@pbs user (+ token) read permission on source datastore (either DatastoreBackup to only sync owned backups, or DatastoreReader to sync everything)

on target instance (PBS-B in your case) as user with admin privileges:
- setup remote entry with sync@pbs user/token configured on PBS-A
- setup user/token that should own the synced backups (if it does not already exist) => let's call this "owner@pbs"
- give owner@pbs user/token RemoteSyncOperator on remote+datastore
- give owner@pbs user/token DatastoreBackup (or DatastorePowerUser if you want to use remove_if_vanished) on tartget datastore
- setup sync job with remote+source datastore, target datastore and owner@pbs user/token from previous steps

as you can see, "sync@pbs" is only used on PBS-B to configure the remote entry (PBS-B will then log in as "sync@pbs" on PBS-A to query and download backup groups/snapshots). "owner@pbs" is only used on PBS-B, PBS-A never sees it.

 

[Ovidiu]

@fabian

thanks for your step-by-step - it seems to be working now and the sync job is running. All that needed changing was this:

- give sync@pbs user (+ token) read permission on source datastore (either DatastoreBackup to only sync owned backups, or DatastoreReader to sync everything)

I had used RemoteSyncOperator and changing it to DatastoreBackup seems to have solved the problem.

English isn't my first language I guess it would have clicked earlier had I posted into the German Proxmox Forum