Permission denied by adding a new datastore on pbs

[Toormser]

After adding the HiDrive storage (https://forum.proxmox.com/threads/c...d-use-it-for-backup-e-g-strato-hidrive.94131/) to my pbs I've been trying to add a datastore with the mountpoint from hidrive to my pbs.

I've got the following error:

root@pbs:~# proxmox-backup-manager datastore create syncstore /mnt/hidrive
TASK ERROR: EACCES: Permission denied
root@pbs:~#

root@pbs:~# chown -R backup:backup /mnt/hidrive/
chown: changing ownership of '/mnt/hidrive/': Permission denied
root@pbs:~# chmod -R 600 /mnt/hidrive/
root@pbs:~#

In the gui as well



The storage is not in use from another host or client. This folder "sync" is freshly created on my hidrive storage.

 

[floh8]

ich versuchs heute mal nachzustellen. is komisch,ne? was mir auffällt, du haust immer die hosts durcheinander. wenn du dich hier auf den pbs beziehst ist es klarer, was du treibst. Deine Threadüberschrift lautet auf pve.

 

[Toormser]

Now the head title is correct. Was my bad. This problem here occours on pbs.

 

[floh8]

what was wrong?

 

[Toormser]

what was wrong?

I mean the head title is now correct.

 

[floh8]

oh, what a miss. I read your post totally wrong ..shame on me

 

[floh8]

wegen deinem test. ich denke, du findest einige hinweise hier im forum oder im www. versuch es mal. ein pbs problem ist es nicht. ich könnte mir auch vorstellen, die rechte von hidrive müssen angepasst, vielleicht auch ein neuer benutzer angelegt werden.

 

[Toormser]

I don't think there is a permission problem on the side from Strato HiDrive as you can see in the console output. Btw, this is a international thread, let's talk in english here for everybody.

 

[floh8]

of course. if u switch between the german and english forum so sometimes u get crazy.
try to create firstly a subdirectory and use this as datastore folder.

 

[floh8]

if u create it locally, then the process works with user root and change the uid and gid of the directory to "backup". so this can also be a problem, u have to handle

 

[Toormser]

Okay. New Information.

We're missunderstood the thing "Add Datastore". The "Permisson Denied" is legit because the folder "/mnt/hidrive" is already exists. If you click on "Add Datastore" then you can't use a mounted folder because the "Add Datastore" Method will be create a datastore (folder) for you.

Now it makes senese, why the error message comes up.

Try to create a datastore with a free folder name like /mnt/exmpalefolder ... this will work, cauz the folder not exists.

This is bad, becaus it's not solving my main problem: How can I add a remote storage like HiDrive for sync jobs


And I can't create a datastore like /mnt/hidrive/storesync as well.

unable to create chunk store 'hidrs21' at "/mnt/hidrive/storesync" - EACCES: Permission denied

A little bit confusing because the folder is created at remote storage

 

[Toormser]

Maybe a staff member can help us tomorrow.

 

[floh8]

us? you ignore my posts. u can do much more testing.

 

[floh8]

Okay. New Information.

We're missunderstood the thing "Add Datastore". The "Permisson Denied" is legit because the folder "/mnt/hidrive" is already exists. If you click on "Add Datastore" then you can't use a mounted folder because the "Add Datastore" Method will be create a datastore (folder) for you.

Now it makes senese, why the error message comes up.

Try to create a datastore with a free folder name like /mnt/exmpalefolder ... this will work, cauz the folder not exists.

This is bad, becaus it's not solving my main problem: How can I add a remote storage like HiDrive for sync jobs


And I can't create a datastore like /mnt/hidrive/storesync as well.

unable to create chunk store 'hidrs21' at "/mnt/hidrive/storesync" - EACCES: Permission denied

A little bit confusing because the folder is created at remote storage

thats wrong. pbs do not create a directory for its self. i told u to create a subdir and take this path for the web gui. clear?
#change#: ok thats right, if the subfolder not exist

But u ignore my advice
what about your sshfs parameters uid and gid? u can create the subfolder for pbs like i mentioned.

 

[fabian]

to create a datastore, it must be possible to create directories and files owned by the user and group 'backup'. it seems that does not work for the drive/disk you have mounted.

 

[Toormser]

to create a datastore, it must be possible to create directories and files owned by the user and group 'backup'. it seems that does not work for the drive/disk you have mounted.

Thanks for your answer. In my case it's not fit.

E.g. 1

1. I want to create the datastore in gui at /mnt/datastore/stores/ named "syncers"
2. The folder "syncers" will be created by the process, after that I became the error message "perission denied"
3. For that reason the permissions to create folders are there

in another exmpale I can create a folder in cli as the user backup as well.

E.g. 2

1. sudo -u backup mkdir /mnt/datastore/stores/syncers

   ('syncers' which was created from the gui before from e.g. 1)

Works fine.



I'm mounting the HiDrive Storage with a user whith the highest privileges and I have only this one.

sshfs#harddisk-9999@sftp.hidrive.strato.com:/users/harddisk-9999/proxmox /mnt/datastore/stores fuse defaults,uid=backup,gid=backup,umask=0,allow_other,_netdev 0 0

All is in one line of /etc/fstab as I can see, the mounting process works well at booting.

Maybe this is a case which need more investigation. I've Proxmox need more information / logs from my side, let me know and much thanks for your assistance and time in this file.

 

[fabian]

sshfs has a long list of restrictions / features not working, some of them depending on the remote server. I am not sure it's a good fit and possible to get running - but you can of course play around with the various options (see man sshfs). performance will not be good in any case

 

[lynx]

Seems like it is possible to get sshfs working but with some caveats:

• even if complete sshfs mount contents are owned by backup:backup, it somehow still wants to change some ownerships when initializing a datastore and EACCES: Permission denied is thrown (it turns out that sshfs does not support chown even to the same user/group, trying it manually results in permission denied)
• using a workaround bindfs --chown-ignore --chgrp-ignore /mnt/my-sshfs /mnt/my-sshfs-bindfs and creating a datastore through that mount workarounds the initial datastore initialization
• it seems like switch back to original fuse mounted sshfs folder (once the datastore is already initialized through bindfs) works fine, no more EACCES error
• performance for over the internet mounted sshfs is not good
• unfortunately there remains some undetermiinstic "flaky" behavior, sometimes there is an error Bad Request (400) unable to detect file type when opening store Contents tab and a similar error unable to detect file type when performing a Sync job. Will check out available sshfs options if any of those will help ...

 

[lynx]

• unfortunately there remains some undetermiinstic "flaky" behavior, sometimes there is an error Bad Request (400) unable to detect file type when opening store Contents tab and a similar error unable to detect file type when performing a Sync job. Will check out available sshfs options if any of those will help ...

Adding fuse mount options sync_readdir,dir_cache=no,no_readahead,sshfs_sync,max_conns=1 solves that last issue. Did not investigate which one(s) of them.

Sshfs shorage over the internet is slow, but seems good enough when having a decent ping time, high deduplication factor, backups size limited to a few dozens of GB and when instant restore time is not a requirement. I ONLY tested this with a sync job puling data from another server.

Edit: Staff says it is not supported

 

[dekiesel]

Since this helped me a lot in getting my own workaround going (bindfs didn't work in my setup) I'll post it here in hopes of it helping other people in the future.

What this does is create a "normal" local folder, then create a datastore (in the gui) using that folder, push the data in that folder into the sshfs folder and then remount that folder so that it is where PBS created the datastore in the filesystem:

• Create datastore folder (e.g. /pbsdatastore)
• Create temporary datastore folder (e.g. /pbsdatastore_tmp)
• Mount remote ssh folder to /pbsdatatstore_tmp (yes, this way around, for the ol' switcheroo)
• Create datastore in pbs guii (use /pbsdatastore)
• Stop pbs: systemctl stop proxmox-backup-proxy.service proxmox-backup.service
• Copy the content of /pbsdatastore to /pbsdatastore_tmp (check that all files where copied, some are "hidden")
• Unmount /pbsdatastore_tmp: umount /pbsdatastore_tmp
• Move /pbsdatastore to e.g. /pbsdatastore_old
• Create datastore-folder: mkdir /pbsdatastore
• Mount remote ssh folder to /pbsdatatstore
• Start pbs: systemctl start proxmox-backup-proxy.service proxmox-backup.service
• If everything worked delete /pbsdatastore_old.

I hope this workaround will one day not be necessary anymore, my guess is that a lot of people are ok with slow (nightly) backups as long as they are remote.

Performance: I backed up my biggest CT (25g) in 45m, with an upload of 34Mb/s. A subsequent backup took 2m.