I'm trying to set up a push-direction sync job between two PBS instances and I'm stuck on a permission check that fails no matter what privileges I grant. I've worked through the checks in the docs and gone well past them, so I suspect I'm misunderstanding which check is actually firing.
Setup
Source: PBS 4.2.5-1 - datastore local-nfs (NFS mount, receives nightly backups from its PVE host)
Target: PBS 4.0.11-2 - datastore nfs-backup (NFS mount)
Both sites behind NAT gateways. The target is reachable from the source via DDNS and a port forward to 8007. The source is not externally reachable, which is why this is push rather than pull.
Config
/etc/proxmox-backup/remote.cfg on the source:
remote: AIT-server
auth-id amc@pbs
host ourpbsserver.ddns.net
fingerprint 1c:de:23:...:e0:3d
password <redacted>
The fingerprint matches proxmox-backup-manager cert info on the target.
/etc/proxmox-backup/sync.cfg on the source:
sync: ait-push
comment push offsite
owner admin@pbs
rate-out 2 MiB
remote AIT-server
remote-store nfs-backup
schedule Wed 17:40
store local-nfs
sync-direction push
transfer-last 1
ACLs on the source:
admin@pbs /datastore/local-nfs 1 Admin
admin@pbs /remote/AIT-server 1 RemoteSyncPushOperator
ACLs on the target:
amc@pbs /datastore/nfs-backup 1 DatastoreAudit
amc@pbs /datastore/nfs-backup 1 DatastoreBackup
As far as I can tell that satisfies the documented requirements for push direction: Remote.Audit on /remote/{remote} and Remote.DatastoreBackup on /remote/{remote}/{remote-store} via RemoteSyncPushOperator, Datastore.Read and Datastore.Audit on the local source datastore, and Datastore.Backup on the target.
Symptoms
# proxmox-backup-manager sync-job run ait-push
TASK ERROR: permission check failed.
Error: task failed (status permission check failed.)
The task log contains nothing beyond that line.
Creating a sync job through the GUI and selecting the target remote gives:
Internal server error 500
failed to scan remote 'AIT-server' - remote connection to 'ourpbsserver.ddns.net'
failed - permission check failed
So the datastore scan against the remote fails the same way.
What I've ruled out
Credentials, network path, port and certificate are all fine. From the source, using the same user as in remote.cfg:
# proxmox-backup-client namespace list \
--repository 'amc@pbs@ourpbsserver.ddns.net:8007:nfs-backup'
This succeeds and returns cleanly (empty, as the target datastore has no content yet). No connection error, no fingerprint prompt, no permission error. So amc@pbs can authenticate to the target and enumerate namespaces inside nfs-backup — but a scan of the same remote from the sync machinery fails.
It is not a privilege the source-side job owner lacks. I granted admin@pbs the Admin role at / on the source. Sync still failed identically.
It is not the job owner at all. I set --owner root@pam on the job. Still failed identically.
It is not a privilege the target-side remote user lacks. I granted amc@pbs the Admin role on /datastore/nfs-backup, and separately the Audit role at / on the target. Both still failed identically.
So the check fails with root-equivalent privileges on the source and read-all plus datastore-admin on the target, which suggests to me it isn't really a privilege lookup that's failing — or it's checking a path I'm not thinking of.
Logs
proxmox-backup-proxy on the source shows only the TASK ERROR line at the time of the failure, no detail.
proxmox-backup-proxy on the target shows no authentication attempt at the corresponding timestamp — only unrelated periodic messages about removable datastores that aren't currently mounted.
Questions
Which specific privilege and ACL path does the remote scan operation check, and is it checked on the source, the target, or both?
Is there a version constraint on push sync between <SOURCE VERSION> and <TARGET VERSION> that would present as a permission error rather than a version error?
Is there a way to get the failing check to log the privilege and path it tested? Anything I can raise the log level on would help.
Possibly relevant
The source host has a clock problem — proxmox-backup-proxy logs rrd update failed: time in past every ten seconds with stored timestamps far in the future, so the system clock jumped at some point and poisoned the RRD files. I'm fixing that separately, but flagging it in case ticket or token timestamp validation is implicated. Note that proxmox-backup-client against the same remote from the same host works fine, so general authentication doesn't appear to be affected.
Happy to provide any further output. Thanks.
Setup
Source: PBS 4.2.5-1 - datastore local-nfs (NFS mount, receives nightly backups from its PVE host)
Target: PBS 4.0.11-2 - datastore nfs-backup (NFS mount)
Both sites behind NAT gateways. The target is reachable from the source via DDNS and a port forward to 8007. The source is not externally reachable, which is why this is push rather than pull.
Config
/etc/proxmox-backup/remote.cfg on the source:
remote: AIT-server
auth-id amc@pbs
host ourpbsserver.ddns.net
fingerprint 1c:de:23:...:e0:3d
password <redacted>
The fingerprint matches proxmox-backup-manager cert info on the target.
/etc/proxmox-backup/sync.cfg on the source:
sync: ait-push
comment push offsite
owner admin@pbs
rate-out 2 MiB
remote AIT-server
remote-store nfs-backup
schedule Wed 17:40
store local-nfs
sync-direction push
transfer-last 1
ACLs on the source:
admin@pbs /datastore/local-nfs 1 Admin
admin@pbs /remote/AIT-server 1 RemoteSyncPushOperator
ACLs on the target:
amc@pbs /datastore/nfs-backup 1 DatastoreAudit
amc@pbs /datastore/nfs-backup 1 DatastoreBackup
As far as I can tell that satisfies the documented requirements for push direction: Remote.Audit on /remote/{remote} and Remote.DatastoreBackup on /remote/{remote}/{remote-store} via RemoteSyncPushOperator, Datastore.Read and Datastore.Audit on the local source datastore, and Datastore.Backup on the target.
Symptoms
# proxmox-backup-manager sync-job run ait-push
TASK ERROR: permission check failed.
Error: task failed (status permission check failed.)
The task log contains nothing beyond that line.
Creating a sync job through the GUI and selecting the target remote gives:
Internal server error 500
failed to scan remote 'AIT-server' - remote connection to 'ourpbsserver.ddns.net'
failed - permission check failed
So the datastore scan against the remote fails the same way.
What I've ruled out
Credentials, network path, port and certificate are all fine. From the source, using the same user as in remote.cfg:
# proxmox-backup-client namespace list \
--repository 'amc@pbs@ourpbsserver.ddns.net:8007:nfs-backup'
This succeeds and returns cleanly (empty, as the target datastore has no content yet). No connection error, no fingerprint prompt, no permission error. So amc@pbs can authenticate to the target and enumerate namespaces inside nfs-backup — but a scan of the same remote from the sync machinery fails.
It is not a privilege the source-side job owner lacks. I granted admin@pbs the Admin role at / on the source. Sync still failed identically.
It is not the job owner at all. I set --owner root@pam on the job. Still failed identically.
It is not a privilege the target-side remote user lacks. I granted amc@pbs the Admin role on /datastore/nfs-backup, and separately the Audit role at / on the target. Both still failed identically.
So the check fails with root-equivalent privileges on the source and read-all plus datastore-admin on the target, which suggests to me it isn't really a privilege lookup that's failing — or it's checking a path I'm not thinking of.
Logs
proxmox-backup-proxy on the source shows only the TASK ERROR line at the time of the failure, no detail.
proxmox-backup-proxy on the target shows no authentication attempt at the corresponding timestamp — only unrelated periodic messages about removable datastores that aren't currently mounted.
Questions
Which specific privilege and ACL path does the remote scan operation check, and is it checked on the source, the target, or both?
Is there a version constraint on push sync between <SOURCE VERSION> and <TARGET VERSION> that would present as a permission error rather than a version error?
Is there a way to get the failing check to log the privilege and path it tested? Anything I can raise the log level on would help.
Possibly relevant
The source host has a clock problem — proxmox-backup-proxy logs rrd update failed: time in past every ten seconds with stored timestamps far in the future, so the system clock jumped at some point and poisoned the RRD files. I'm fixing that separately, but flagging it in case ticket or token timestamp validation is implicated. Note that proxmox-backup-client against the same remote from the same host works fine, so general authentication doesn't appear to be affected.
Happy to provide any further output. Thanks.
