proxmxo managed cephfs mount on external computer (non porxmox) cant access file content

I

ilia987

Active Member
Sep 9, 2019
275
13
38
37
i have usefully integrated ceph(proxmox based) in all the lxc containers,
now i want to integrate it outside of proxmox for some user for read only access , to replace the current nfs share,
what do i need to do ? what params to put in /etc/fstab
 
ilia987 said:
i followed the gide above,

and when i mount i get
Code: 
mount error 13 = Permission denied

i executed the command:
Code: 
mount -t ceph ceph-mon1.storage:6789:/ /mnt/mycephfs -o name=cephx,secretfile=/etc/ceph/ceph.client.cephx.keyring
Click to expand...
working only when setting only / in path when creating authorization


i could not mount sub dir, only root



works:
Code: 
mount -t ceph ceph-mon1.storage:6789,ceph-mon2.storage:6789,ceph-mon3.storage:6789:/ /mnt/mycephfs -o name=cephx,secretfile=/etc/ceph/ceph.client.cephx.keyring

does not work fstab
Code: 
ceph ceph-mon1.storage:6789,ceph-mon2.storage:6789,ceph-mon3.storage:6789:/ /mnt/ftd ceph name=cephx,secretfile=/etc/ceph/ceph.client.cephx.keyring,noatime 0 0
 
Last edited:
update :have some progress i can mount it and see the files but i dont have permission to read the files :(
Code: 
ceph-mon1.storage:6789:/data /mnt/ftd ceph name=cephx,secretfile=/etc/ceph/ceph.client.cephx.keyring,ro,_netdev,noatime    0   0
 
Did you create a new user for accessing the cephfs?
or do you use the default root what is created by Proxmox VE?
 
wolfgang said:
Did you create a new user for accessing the cephfs?
or do you use the default root what is created by Proxmox VE?
Click to expand...
what i did:
Code: 
#on client, outside of proxmox (desktop)
sudo su
apt-get install ceph-common

mkdir /etc/ceph

ssh  ceph-mon1.storage
ssh root@ceph-mon1.storage "ceph config generate-minimal-conf" | tee /etc/ceph/ceph.conf
chmod 644 /etc/ceph/ceph.conf

ssh root@ceph-mon1.storage " ceph fs authorize cephfs-data client.cephx / rw" | tee /etc/ceph/ceph.client.cephx.keyring

chmod 600 /etc/ceph/ceph.client.cephx.keyring
#edit ceph.client.cephx.keyring to contain only key



mkdir /mnt/mycephfs
#edit fstab
ceph-mon1.storage:6789,ceph-mon2.storage:6789,ceph-mon3.storage:6789:/data /mnt/ftd ceph name=cephx,secretfile=/etc/ceph/ceph.client.cephx.keyring,ro,_netdev,noatime    0   0

mounts created using proxmox (gui) works inside lxc conainers,

but when i try to create cephfs mount outside the proxmox ( local desktop) i can only see the files , i cannot access theirs contents.
 
Do you use the same ceph version at the server and client?
 
Can you please post the ceph auth list (w/o key ;)) of the user?
 
Alwin said:
Can you please post the ceph auth list (w/o key ;)) of the user?
Click to expand...
all keys are removed
Code: 
mds.pve-srv2                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mon]    allow    profile    mds       
    caps:    [osd]    allow    rwx           
mds.pve-srv3                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mon]    allow    profile    mds       
    caps:    [osd]    allow    rwx           
mds.pve-srv4                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mon]    allow    profile    mds       
    caps:    [osd]    allow    rwx           
osd.0                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.1                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.10                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.11                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.2                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.3                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.4                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.5                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.6                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.7                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.8                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.9                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
client.admin                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mgr]    allow    *           
    caps:    [mon]    allow    *           
    caps:    [osd]    allow    *           
client.bootstrap-mds                           
    key:                       
    caps:    [mon]    allow    profile    bootstrap-mds       
client.bootstrap-mgr                           
    key:                       
    caps:    [mon]    allow    profile    bootstrap-mgr       
client.bootstrap-osd                           
    key:                       
    caps:    [mon]    allow    profile    bootstrap-osd       
client.bootstrap-rbd                           
    key:                       
    caps:    [mon]    allow    profile    bootstrap-rbd       
client.bootstrap-rbd-mirror                           
    key:                       
    caps:    [mon]    allow    profile    bootstrap-rbd-mirror       
client.bootstrap-rgw                           
    key:                       
    caps:    [mon]    allow    profile    bootstrap-rgw       
client.cephx                           
    key:                       
    caps:    [mds]    allow    rw           
    caps:    [mon]    allow    r           
    caps:    [osd]    allow    rw    tag    cephfs    data=cephfs-data
mgr.pve-srv2                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mon]    allow    profile    mgr       
    caps:    [osd]    allow    *           
mgr.pve-srv3                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mon]    allow    profile    mgr       
    caps:    [osd]    allow    *           
mgr.pve-srv4                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mon]    allow    profile    mgr       
    caps:    [osd]    allow    *
 
issue still exist, still looking for help\guidance

i am creating the mount under root user (sudo mount -a)
and trying to access it as another user (local\DC user) not as cephx (this user not really exist, it is exist only to create the share)

i can see the file folder and files are mounted using

mount command inside fstab
Code: 
ceph-mon1.storage:6789,ceph-mon2.storage:6789,ceph-mon3.storage:6789:/data /mnt/ftd ceph name=cephx,secretfile=/etc/ceph/ceph.client.cephx.keyring,ro,_netdev,noatime    0   0

ls
Code: 
-rwxr-xr-x 1 user2_ldap domain^users 219873796 Jan 1 18:20 /mnt/ftd/data/file.bin
 
This has more to do with the permissions set on the directories and files, then the ceph auth.
 
Alwin said:
This has more to do with the permissions set on the directories and files, then the ceph auth.
Click to expand...
i can access from lcx inside proxmox with user user2_ldap,
but from desktop (outside of proxmox) with mount using fstab with the same user(user2_ldap) i can go in and out to internal directories but i cannot access its files contents
 
logs under strace:
Code: 
openat(AT_FDCWD, "/mnt/ftd/data/file.bin", O_RDONLY) = 3
read(3, 0x7ffd679877b0, 8192)           = -1 EPERM (Operation not permitted)
 
i tried mounting under admin (with admin key) and it works,
but i dont think this is smart. still looking for better solution
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom