proxmxo managed cephfs mount on external computer (non porxmox) cant access file content

I

ilia987

Well-Known Member
Sep 9, 2019
281
14
58
38
i have usefully integrated ceph(proxmox based) in all the lxc containers,
now i want to integrate it outside of proxmox for some user for read only access , to replace the current nfs share,
what do i need to do ? what params to put in /etc/fstab
 
ilia987 said:
i followed the gide above,

and when i mount i get
Code: 
mount error 13 = Permission denied

i executed the command:
Code: 
mount -t ceph ceph-mon1.storage:6789:/ /mnt/mycephfs -o name=cephx,secretfile=/etc/ceph/ceph.client.cephx.keyring
Click to expand...
working only when setting only / in path when creating authorization


i could not mount sub dir, only root



works:
Code: 
mount -t ceph ceph-mon1.storage:6789,ceph-mon2.storage:6789,ceph-mon3.storage:6789:/ /mnt/mycephfs -o name=cephx,secretfile=/etc/ceph/ceph.client.cephx.keyring

does not work fstab
Code: 
ceph ceph-mon1.storage:6789,ceph-mon2.storage:6789,ceph-mon3.storage:6789:/ /mnt/ftd ceph name=cephx,secretfile=/etc/ceph/ceph.client.cephx.keyring,noatime 0 0
 
Last edited:
update :have some progress i can mount it and see the files but i dont have permission to read the files :(
Code: 
ceph-mon1.storage:6789:/data /mnt/ftd ceph name=cephx,secretfile=/etc/ceph/ceph.client.cephx.keyring,ro,_netdev,noatime    0   0
 
Did you create a new user for accessing the cephfs?
or do you use the default root what is created by Proxmox VE?
 
wolfgang said:
Did you create a new user for accessing the cephfs?
or do you use the default root what is created by Proxmox VE?
Click to expand...
what i did:
Code: 
#on client, outside of proxmox (desktop)
sudo su
apt-get install ceph-common

mkdir /etc/ceph

ssh  ceph-mon1.storage
ssh root@ceph-mon1.storage "ceph config generate-minimal-conf" | tee /etc/ceph/ceph.conf
chmod 644 /etc/ceph/ceph.conf

ssh root@ceph-mon1.storage " ceph fs authorize cephfs-data client.cephx / rw" | tee /etc/ceph/ceph.client.cephx.keyring

chmod 600 /etc/ceph/ceph.client.cephx.keyring
#edit ceph.client.cephx.keyring to contain only key



mkdir /mnt/mycephfs
#edit fstab
ceph-mon1.storage:6789,ceph-mon2.storage:6789,ceph-mon3.storage:6789:/data /mnt/ftd ceph name=cephx,secretfile=/etc/ceph/ceph.client.cephx.keyring,ro,_netdev,noatime    0   0

mounts created using proxmox (gui) works inside lxc conainers,

but when i try to create cephfs mount outside the proxmox ( local desktop) i can only see the files , i cannot access theirs contents.
 
Do you use the same ceph version at the server and client?
 
Can you please post the ceph auth list (w/o key ;)) of the user?
 
Alwin said:
Can you please post the ceph auth list (w/o key ;)) of the user?
Click to expand...
all keys are removed
Code: 
mds.pve-srv2                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mon]    allow    profile    mds       
    caps:    [osd]    allow    rwx           
mds.pve-srv3                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mon]    allow    profile    mds       
    caps:    [osd]    allow    rwx           
mds.pve-srv4                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mon]    allow    profile    mds       
    caps:    [osd]    allow    rwx           
osd.0                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.1                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.10                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.11                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.2                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.3                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.4                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.5                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.6                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.7                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.8                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
osd.9                           
    key:                       
    caps:    [mgr]    allow    profile    osd       
    caps:    [mon]    allow    profile    osd       
    caps:    [osd]    allow    *           
client.admin                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mgr]    allow    *           
    caps:    [mon]    allow    *           
    caps:    [osd]    allow    *           
client.bootstrap-mds                           
    key:                       
    caps:    [mon]    allow    profile    bootstrap-mds       
client.bootstrap-mgr                           
    key:                       
    caps:    [mon]    allow    profile    bootstrap-mgr       
client.bootstrap-osd                           
    key:                       
    caps:    [mon]    allow    profile    bootstrap-osd       
client.bootstrap-rbd                           
    key:                       
    caps:    [mon]    allow    profile    bootstrap-rbd       
client.bootstrap-rbd-mirror                           
    key:                       
    caps:    [mon]    allow    profile    bootstrap-rbd-mirror       
client.bootstrap-rgw                           
    key:                       
    caps:    [mon]    allow    profile    bootstrap-rgw       
client.cephx                           
    key:                       
    caps:    [mds]    allow    rw           
    caps:    [mon]    allow    r           
    caps:    [osd]    allow    rw    tag    cephfs    data=cephfs-data
mgr.pve-srv2                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mon]    allow    profile    mgr       
    caps:    [osd]    allow    *           
mgr.pve-srv3                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mon]    allow    profile    mgr       
    caps:    [osd]    allow    *           
mgr.pve-srv4                           
    key:                       
    caps:    [mds]    allow    *           
    caps:    [mon]    allow    profile    mgr       
    caps:    [osd]    allow    *
 
issue still exist, still looking for help\guidance

i am creating the mount under root user (sudo mount -a)
and trying to access it as another user (local\DC user) not as cephx (this user not really exist, it is exist only to create the share)

i can see the file folder and files are mounted using

mount command inside fstab
Code: 
ceph-mon1.storage:6789,ceph-mon2.storage:6789,ceph-mon3.storage:6789:/data /mnt/ftd ceph name=cephx,secretfile=/etc/ceph/ceph.client.cephx.keyring,ro,_netdev,noatime    0   0

ls
Code: 
-rwxr-xr-x 1 user2_ldap domain^users 219873796 Jan 1 18:20 /mnt/ftd/data/file.bin
 
This has more to do with the permissions set on the directories and files, then the ceph auth.
 
Alwin said:
This has more to do with the permissions set on the directories and files, then the ceph auth.
Click to expand...
i can access from lcx inside proxmox with user user2_ldap,
but from desktop (outside of proxmox) with mount using fstab with the same user(user2_ldap) i can go in and out to internal directories but i cannot access its files contents
 
logs under strace:
Code: 
openat(AT_FDCWD, "/mnt/ftd/data/file.bin", O_RDONLY) = 3
read(3, 0x7ffd679877b0, 8192)           = -1 EPERM (Operation not permitted)
 
i tried mounting under admin (with admin key) and it works,
but i dont think this is smart. still looking for better solution
 
You must log in or register to reply here.
Top Bottom