Cannot add PBS storage accessible via Cloudflare and Zero Trust

5

5k7

Member
Sep 28, 2022
20
0
6
Hello,

I have a problem connecting to PBS, which runs behind a Cloudflare tunnel and is accessed through Zero Trust.

My setup:
  1. The PBS server is tunneled to a private domain, restricted by Zero trust – pbs.mydomain.com
  2. A Bypass Policy is added for the PVE host – by IP address
  3. In PVE, I'm adding the storage using login, password, host:443, and datastore, no fingerprint
Every time I get the following error message:
create storage failed: nexus-pbs: error fetching datastores - 403 Forbidden (500)


What I’ve checked and ruled out so far:

  1. Insufficient permissions in PBS: I assigned the user an admin role and root namespace to make sure nothing was missing. Additionally, I was able to add the storage from another PVE host in the same network without any issues. So we can fully rule out any misconfiguration or permission problems on the PBS side.
  2. Cloudflare correctly passing traffic: I thoroughly verified that the Bypass works and that the server can communicate with PBS – there are no issues here. I also checked the flow from external PVE server by hitting two endpoints in sequence:
Bash: 
read TICKET TOKEN <<<$(curl -sk \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  --data 'username=backup@pbs&password=mypw' \
  https://pbs.mydomain.com/api2/json/access/ticket \
  | jq -r '.data.ticket,.data.CSRFPreventionToken')

Returned HTTP 200

Bash: 
curl -ik \
  -H "Cookie: PBSAuthCookie=$TICKET" \
  -H "CSRFPreventionToken: $TOKEN" \
  https://pbs.mydomain.com/api2/json/admin/datastore

Returned HTTP 200 with body:

JSON: 
{"data":[{"comment":null,"mount-status":"nonremovable","store":"pbs"}]}

2. Adding storage using CLI:

Bash: 
pvesm add pbs nexus-pbs \
    --server pbs.mydomain.com \
    --port 443 \
    --username backup@pbs \
    --password 'mypw' \
    --datastore pbs \
    --content backup

create storage failed: nexus-pbs: error fetching datastores - 403 Forbidden

I believe I’ve checked everything thoroughly enough that the only remaining idea I have is a possible bug in PVE when PBS is running on a non-default port.

Has anyone encountered this issue or has any other ideas?
 
It seems that during connection PVE is sending RST and closing connection. May it be due to additional headers provided by CF?

Any other suggestions/ideas?
 
Have you tried to get a list of available Proxmox Backup Server datastores with (as shown here):
Code: 
pvesm scan pbs <server> <username> [--password <string>]
so in your case maybe:
Code: 
pvesm scan pbs \
    --server pbs.mydomain.com \
    --port 443 \
    --username backup@pbs \
    --password 'mypw' \
 
  • Like
Reactions: UdoB
No, I didn’t, because I don’t believe this issue is related to an invalid configuration or permissions. I have PVE running on a local network, and using the same commands, I can connect without any problems.


I tried what you suggested, but I got the same error:
error fetching datastores - 403 Forbidden


I also noticed that even when I provide incorrect credentials, the response remains the same. That made me think it might be related to IPv6/IPv4. The connection is actually being made over IPv6, and that might be where the issue lies. I’ll need to check the IPv6 setup locally, which I plan to do later.
 
As I mentioned at the beginning, I’ve only tunneled port 8007 to port 443 on my domain using a Cloudflare Tunnel with Zero Trust policies. So, ICMP is not enabled — only HTTPS is allowed.


I also shared a curl request from PVE to PBS, and it’s working — it even returns the datastore data from PBS.
 
Have you managed to get it working? i have the same issue here, my second best alternative is opening a vpn between the two servers
 
You must log in or register to reply here.
Top Bottom