Posting this as a note for anyone scripting PBS setup, since the symptom looks identical to several well-documented problems but has a different cause.
Setup: PBS 4.2.5 in an unprivileged LXC, three standalone PVE 9.2.6 hosts, one per-host API token,
I scripted adding PBS as storage on all three hosts. All three failed with:
That message already has several known causes — tokens not inheriting user permissions, the
Isolating it: create the token, wait 10s so the token itself is definitely settled, then grant the ACL and query immediately.
Same token, same ACL, nothing changed but time. So it is the ACL that needs a moment, not the token — and the API returns an empty list rather than a 403, which is presumably intended permission-filtering. PVE's error is accurate from its side: the list really was empty.
What I changed: my provisioning script now sleeps/retries between
Suggestion: it would be worth a line in the ACL section of the docs — something like "ACL changes may take up to a second to become effective for API requests; scripts granting a role and immediately using it should retry." Anyone hitting this while automating will otherwise search the error string and find three other causes, none of which apply.
Only one instance here, so if anyone can confirm or contradict the ~1s window I would be glad to hear it before this goes any further.
DISCLAIMER: this post was created with help from Claude Opus 5
Setup: PBS 4.2.5 in an unprivileged LXC, three standalone PVE 9.2.6 hosts, one per-host API token,
Datastore.Backup on the datastore path.I scripted adding PBS as storage on all three hosts. All three failed with:
Code:
create storage failed: pbs1: Cannot find datastore 'estate', check permissions and existence!That message already has several known causes — tokens not inheriting user permissions, the
! being eaten by bash in the auth header, the wrong datastore name. It was none of those. Each host succeeded on a retry where the only change was a short delay.Isolating it: create the token, wait 10s so the token itself is definitely settled, then grant the ACL and query immediately.
Code:
t+0s {"data":[]}
+1s {"data":[{"backend-type":"filesystem","store":"estate", ...}]}Same token, same ACL, nothing changed but time. So it is the ACL that needs a moment, not the token — and the API returns an empty list rather than a 403, which is presumably intended permission-filtering. PVE's error is accurate from its side: the list really was empty.
What I changed: my provisioning script now sleeps/retries between
proxmox-backup-manager acl update and the first use of that grant. Trivial fix once you know, but it cost three failed runs that looked exactly like a permissions misconfiguration — and sent me down the wrong path, since I briefly concluded DatastoreBackup was insufficient and added DatastoreAudit. It is not required; DatastoreBackup alone is enough. I had changed two things at once and credited the wrong one.Suggestion: it would be worth a line in the ACL section of the docs — something like "ACL changes may take up to a second to become effective for API requests; scripts granting a role and immediately using it should retry." Anyone hitting this while automating will otherwise search the error string and find three other causes, none of which apply.
Only one instance here, so if anyone can confirm or contradict the ~1s window I would be glad to hear it before this goes any further.
DISCLAIMER: this post was created with help from Claude Opus 5
