Proxmox and RBD Ceph Mountpoint with pct set?

[oops404]

Hello all,

i have a ceph storage and created a ceph pool on my cluster with the name ceph-replicate.

Now I want that multiple LXC linux container can access the same storage. I thought maybe I can do this with the pct set command. Could this work?
pct set <container_id> --mp0 cephool_name,/mnt/ceph

I tried:

pct set 107 --mp1 ceph:ceph-replicate,/mnt/files

I get:
---------
400 Parameter verification failed.
mp1: invalid format - duplicate key in comma-separated list property: volume

pct set <vmid> [OPTIONS]
---------

Not sure if the syntax is wrong or that this is not working and I need to create a container which have samba and nfs share to provide the storage to all other container.

Best

 

[oops404]

I have "fixed" it now that I have additionally configure cephFS ^^

 

[aaron]

Probably only if they are on the same node!

The command should look more like pct set 107 --mp1 ceph:ceph-replicate,mp=/mnt/files.
But I don't think this is a good idea because the image is formatted with ext4 and mounting that multiple times will surely cause issues.

Using CephFS from within the containers is probably the nicer option. Give them access to the Ceph network and install the cephfs packages and add authorization for it. Or, you can use bindmounts to /mnt/pve/{your cephfs} to utilize the CephFS mounts that PVE will do automatically if the storage is defined.

Then it could be for example pct set 107 --mp1 /mnt/pve/cephfs/subfolder1,mp=/mnt/files

 

[oops404]

Using CephFS from within the containers is probably the nicer option. Give them access to the Ceph network and install the cephfs packages and add authorization for it. Or, you can use bindmounts to /mnt/pve/{your cephfs} to utilize the CephFS mounts that PVE will do automatically if the storage is defined.

Then it could be for example pct set 107 --mp1 /mnt/pve/cephfs/subfolder1,mp=/mnt/files

I have tried to use bindmounts and I can create them but then the container can't be moved. I used:
pct set 105 --mp0 /mnt/pve/cephfs,mp=/mnt/files

The error:

"2023-06-26 17:24:04 ERROR: migration aborted (duration 00:00:00): cannot migrate local bind mount point 'mp0'
TASK ERROR: migration aborted"

I have changed the mp0 to this but then the container is not longer starting so I reverted (mp0: cephfs,mp=/mnt/files)

I tried to bring it in CephFS but seems not to work
"
root@ct-seafile:/etc/ceph# mount -t ceph 10.0.200.28,10.0.200.29,10.0.200.30:/ /mnt/files -o name=ctseafile,secretfile=/etc/ceph/ceph.client.ctseafile.keyring
modprobe: FATAL: Module ceph not found in directory /lib/modules/6.2.16-3-pve
failed to load ceph kernel module (1)
secret is not valid base64: Invalid argument.
adding ceph secret key to kernel failed: Invalid argument
couldn't append secret option: -22
root@ct-seafile:/etc/ceph#
"

Have no idea how to get this working