"Connection Error - Server Offline?"

R

rackem

New Member
Jun 23, 2025
6
0
1
Newb user. Having issues connecting PBS to PVE. Every attempt to add it immediately returns “Connection error - server offline?”. This is my second try. The first time, I had PBS installed on the same box as PVE just to test. Same errors. Since that’s not best practice, I spun up a VM in Synology VMM instead, but no love. Thanks in advance for any help.

Setup:
  • PVE (Proxmox VE 8.4.0) is on a dedicated physical server.
  • PBS (Proxmox Backup Server 3.4.1-1) is a VM hosted in Synology Virtual Machine Manager.
  • Both are fresh installs installed the last few days.
  • Both have static IPs on the same subnet (192.168.64.0/24):
  • PVE5: 192.168.64.5
  • PBS3: 192.168.64.3
  • GUI is reachable on both
  • Ping between hosts is successful from their respective consoles.
  • NFS datastore (ds5-backup) is mounted and added successfully in PBS.
  • No firewall installed / active on the OS of PBS or PVE, or on the Synology
What I’ve Tried:
  • Tried user/password authentication with a new, dedicated user
  • User: backup-user@pbs
  • Verified: creds, permissions, fingerprint (self-signed), datastore name, PBS VM IP address
  • Permissions: DatastoreAdmin (later added Audit as part of troubleshooting)
  • Confirmed backup-user@pbs credentials work via CLI.
Other Things I’ve Tried:
  • Tried root@pam with password and same permissions — same result.
  • Tried API Token Auth (backup-user@pbs!pve5) — same result.
  • Tested the PBS API using curl, authenticating with the same backup-user@pbs credentials that were used in the Proxmox VE GUI during the attempt to add PBS. Authentication succeeded, and a valid ticket was returned. However, running pvesm status returns ds5-backup: error fetching datastores - 401 Unauthorized, indicating that PVE itself is still unable to authenticate with PBS.
  • On PBS, running journalctl -u proxmox-backup-proxy -f during the connection attempt showed no activity—no incoming request or log entry was recorded when trying to add the backup server from PVE.
  • On PVE, running journalctl -u pveproxy -u pvedaemon -u pvestatd -f during the connection attempt showed no new activity—aside from the expected 401 Unauthorized response, no additional log entries appeared when trying to add the backup server.
Edit for clarity.
 
Last edited:
Hi,
rackem said:
Confirmed backup-user@pbs credentials work via CLI.
Click to expand...
How exactly did you verify this and from where? On the PVE host, PBS host?
rackem said:
Tested the PBS API using curl, authenticating with the same backup-user@pbs credentials that were used in the Proxmox VE GUI during the attempt to add PBS. Authentication succeeded, and a valid ticket was returned. However, running pvesm status returns ds5-backup: error fetching datastores - 401 Unauthorized, indicating that PVE itself is still unable to authenticate with PBS.
Click to expand...
Are you able to list the datastore's content when using the proxmox-backup-client on the PVE? E.g. proxmox-backup-client list --repository <your-repos> should list you the groups in the root namespace. The repository must be provided as described in https://pbs.proxmox.com/docs/backup-client.html#backup-repository-locations.

If that does not work, than it is most likely a permission issue with your user. Note that if you use api tokens, you must also define the permissions for the token, as they are independent from the user's. See also https://pbs.proxmox.com/docs/user-management.html#api-tokens
 
Chris said:
Hi,

How exactly did you verify this and from where? On the PVE host, PBS host?

Are you able to list the datastore's content when using the proxmox-backup-client on the PVE? E.g. proxmox-backup-client list --repository <your-repos> should list you the groups in the root namespace. The repository must be provided as described in https://pbs.proxmox.com/docs/backup-client.html#backup-repository-locations.

If that does not work, than it is most likely a permission issue with your user. Note that if you use api tokens, you must also define the permissions for the token, as they are independent from the user's. See also https://pbs.proxmox.com/docs/user-management.html#api-tokens
Click to expand...
Thanks for getting back to me. Some answers:

The CLI test was made with this command from pve5 to pbs3 after I set the backup-user password as an environmental variable:

proxmox-backup-client login --repository backup-user@pbs@192.168.64.3:ds5-backup.

Then followed it up with:

proxmox-backup-client list --repository backup-user@pbs@192.168.64.3:ds5-backup. No errors for either. I

Also, I was able to run the command for the datastore's contents with this commend after logging in from pve5:

proxmox-backup-client list --repository backup-user@pbs@192.168.64.3:ds5-backup.

No errors (no content either, since I've yet to add pbs to pve).

Regarding the api tokens, I did define specific permissions (datastore admin) for the pve5 token as part of testing.

Thanks again.

Edit for formatting.
 
Last edited:
rackem said:
proxmox-backup-client list --repository backup-user@pbs@192.168.64.3:ds5-backup
Click to expand...
Can you do the same for the api token, executing on the pve host.. you will have to provide the api token secret as password, easiest by passing the storages secret file path via the PBS_PASSWORD_FILE environment variable if you have already set it up as storage with token, e.g /etc/pve/priv/storage/<pbs-storage-name>.
 
Chris said:
Can you do the same for the api token, executing on the pve host.. you will have to provide the api token secret as password, easiest by passing the storages secret file path via the PBS_PASSWORD_FILE environment variable if you have already set it up as storage with token, e.g /etc/pve/priv/storage/<pbs-storage-name>.
Click to expand...

Done from pve5 console:

proxmox-backup-client login --repository 'backup-user@pbs!pve5@192.168.64.3:ds5-backup'

No errors.
 
Ah... sorry. You did say list. Just ran it:

proxmox-backup-client list --repository 'backup-user@pbs!pve5@192.168.64.3:ds5-backup'

It gave me this:

connecting to 192.168.64.3:8007
connected to 192.168.64.3:8007
flushed 260 bytes
parsed 2 headers
incoming body is content-length (43 bytes)
incoming body completed
pooling idle connection for ("https", 192.168.64.3:8007)
Error: authentication failed - invalid credentials

So... yeah, something's not right regardless of what I think i'm doing in the respective GUIs. I did create a new API token, "pve5-new", in case I got the secret key wrong for pve5, but got he same response when testing from CLI.

Next I went into LLM Land for help in troubleshooting. Kind of went down the rabbit hole, so apologies in advance. Here's the summary. I did review it for accuracy, so hope it's helpful and not a hallucination.

---
PVE to PBS "Permission Check Failed" Troubleshooting Summary (Comprehensive - Updated)

Issue: proxmox-backup-client list from PVE (192.168.64.5) to PBS (192.168.64.3) consistently fails with "Error: permission check failed."

Client-Side (PVE 192.168.64.5) Observations & Testing:

  • proxmox-backup-client output consistently shows:

    connecting to 192.168.64.3:8007<br>connected to 192.168.64.3:8007<br>...<br>Error: permission check failed.<br>
  • The initial client error message often included a (certificate fingerprint mismatch!) note, even though the connection seemed to establish.
  • Attempted to clear cached PBS fingerprints on PVE (rm -f /etc/proxmox-backup/*.fingerprint), but the issue persists, and the client does not prompt to accept a new fingerprint after this.
  • Direct Console Testing with Environment Variables: The proxmox-backup-client list command was also tested successfully using environment variables (PBS_URL, PBS_REPO, PBS_TOKEN, PBS_PASSWORD) directly from the PVE console, replicating the exact "permission check failed" error, confirming the issue is independent of vzdump's call method.
Server-Side (PBS 192.168.64.3) Observations & Checks:

  • Network Layer Verification: tcpdump on PBS confirmed that the full TLS handshake from PVE completes, and application data (the proxmox-backup-client request) is successfully received by the PBS VM on port 8007.
  • Certificate Fingerprint: PBS's active TLS certificate fingerprint was verified via openssl s_client, confirming it matches the expected value (DA:84:C0:0C:58:3B:CD:DE:86:16:32:04:19:96:A8:14:0A:73:EA:44:8D:18:66:F1:C8:38:62:30:16:0F:7D:6B).
  • API Token Configuration: The specific API token backup-user@pbs!pve5-new is configured with Datastore.Admin role for the /datastore/ds5-backup path, which is the target datastore.
  • Time Synchronization: Time was confirmed to be synchronized between PVE and PBS.
  • PBS Logging Verbosity: Multiple attempts were made to increase the proxmox-backup-proxy.service logging level to debug (Environment=RUST_LOG=debug via systemctl edit and direct /etc/default/proxmox-backup-proxy modification). However, journalctl -u proxmox-backup-proxy.service -f does not show any new debug output related to incoming client connections or permission checks, even when the PVE client connects and receives the "permission check failed" error. Only service restart logs are visible.
Current Conclusion: The issue is occurring at the application layer on the PBS side. Despite confirmed correct API token permissions, successful network/TLS connection, and client-side testing replicating the error, PBS debug logging is inexplicably not providing further details on the reason for this rejection.
 
Please share the output of cat /etc/proxmox-backup/acl.cfg as well as ss -tulpen from the Proxmox Backup Server. Further, check and post the output of tail -n 100 /var/log/proxmox-backup/api/access.log. Do you have any proxies in between the PVE and the PBS?
 
No proxies at all.

Tulpen and log output txt files attached.

For the acl.cfg:
cat /etc/proxmox-backup/acl.cfg. == >
acl:1:/datastore/ds5-backup:backup-user@pbs,backup-user@pbs!pve5,backup-user@pbs!pve5-new:DatastoreAdmin
 

Attachments

From the access logs one sees:
::ffff:192.168.64.5 - - [24/06/2025:18:44:37 -0400] "GET /api2/json/admin/datastore/ds5-backup/groups" 401 43 proxmox-backup-client/1.0
Click to expand...
and on the client side you get a
rackem said:
authentication failed - invalid credentials
Click to expand...
which according to a quick glance at the code is thrown when the provided password does not match.

So please do double check the token secret you provide... Maybe you have copied it over and it contains some unintended characters?
 
Hey Chris-

Thanks for the reply. I get that's what the logs are saying, which has made me triple check everything. I copy and paste the secret from the token creation secret prompt directly to the PVE connection pane for PBS. Same for just using a password. I copy directly from my password safe and paste (and I've typed manually, same as I do when connecting via iterm2). Maybe it's some corruption on the copy/paste but that seems implausible. I will go another round and see what happens, then report back. Thanks again.
 
You must log in or register to reply here.
Top Bottom