Datastore was on CIFS, remounted on NFS, rolling back -> Permission denied (os error 13)

sgw

Active Member
Apr 5, 2024
181
34
33
www.oops.co.at
At a customer we run PBS-4.1.6, patched up to date.

For some years we have a datastore with this mount-point:

Bash:
//192.168.1.36/BackupProxmoxSSD /mnt/backupssd cifs uid=34,noforceuid,gid=34,noforcegid,credentials=/etc/samba/.creds,defaults 0 0

Code:
datastore: SynologySSD

    comment

    gc-schedule sat 20:00

    notification-mode notification-system

    path /mnt/backupssd

That worked. We saw occasional hickups, related to CIFS maybe.
So I decided to try to mount the same share via NFS (Synology NAS gives me access through CIFS and NFS, to the same data ..)

For NFS I had to do some chown-magic, as it doesn't allow to use "uid=,gid=" in fstab. Didn't work, though. Some special char issue ... (I can link to that issue later).

Decided to roll back to fix the datastore for tonight.

And now I don't get access to that datastore anymore.


Code:
Apr 09 17:59:06 pbs proxmox-backup-proxy[649]: GET /api2/json/admin/datastore/SynologySSD/status: 400 Bad Request: [client [::ffff:192.168.1.202]:45438] Permission denied (os error 13)

As far as I understand my mount-options should somehow make chown useless: everything is mapped to user:group 34:34 ...

I rebooted PBS several times, restarted services etc


Code:
# ls -la /mnt/backupssd

total 8

drwxr-xr-x 2 backup backup    0 Apr  4 23:55 .

drwxr-xr-x 7 root   root   4096 Apr  9 14:56 ..

drwxr-xr-x 2 backup backup    0 Aug 14  2024 .chunks

drwxr-xr-x 2 backup backup    0 Aug 14  2024 ct

-rwxr-xr-x 1 backup backup  384 Apr  4 23:55 .gc-status

-rwxr-xr-x 1 backup backup    0 Aug 14  2024 .lock

drwxr-xr-x 2 backup backup    0 Mar 23 11:51 vm

perms look the same as in another working datastore

Any ideas, anyone??
 
I am trying stuff like this in parallel:

# find /mnt/backupssd/ -not -user backup -print | xargs -n 100 -P 4 chown backup:backup

Maybe some meta-data (database?) has gotten out of sync with the content of the actual directory.
Although "permission denied" sounds different.

There are ~ 7TB backups in there, so it would be great to get access again.
 
Hi @sgw,
Having dealt with MultiProtocol access in prior life with multiple enterprise storage products I can tell you things are not simple there. Each vendor implements permissions mappings differently. Some store additional metadata in the filesystem, others change the model based on the internal options. I doubt that Synology is one of those since they use standard FS.

To be frank I don't know anything about Synology implementation, my recommendation would be to reach out to their support. Also thoroughly document your steps - that may help them to untangle the issue. At the very least post the details on their forum.

There is a good chance you should be able to restore access but I don't have specific steps that would help. You may also want to consult one of the AI models.

Good luck


Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
@bbgeek17 thanks! You point at the right issue, and I had solved it a few minutes before you wrote ;-)

I disabled the NFS-sharing and looked into the permissions on the NAS, for the CIFS-users there.

The write-permission for our user "usr_proxmox" was gone! not set. Changed that ... *bam* ... datastore browseable again!

What a waste of my time this afternoon ... A huge loop with no improvements.

Thanks anyway!
 
  • Like
Reactions: bbgeek17