[SOLVED] Mount RBD pool in LXC CT?

[Gardouille]

I try to map and mount a RBD pool (from a ceph cluster) into a LXC container without success …:

/usr/bin/rbd map --pool rbd test --id test --keyring /etc/ceph/ceph.client.test.keyring
rbd: sysfs write failed
rbd: map failed: (30) Read-only file system

The Ceph configuration file seems good:

/etc/ceph/ceph.conf
[global]
auth_service_required = cephx
mon_initial_members = mon1,mon2,mon3
fsid = 618bacef-6194-964f-1649-fabcde186390
keyring = /etc/ceph/$cluster.$name.keyring
auth_supported = cephx
auth_cluster_required = cephx
mon_host = 192.168.0.90:6789,192.168.0.91:6789,192.168.0.92:6789
auth_client_required = cephx

If i look the mounted fs, i got one line for /sys with read-write and another one with read-only:

mount | grep sysfs
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (ro,nosuid,nodev,noexec,relatime)
sysfs on /sys/devices/virtual/net type sysfs (rw,relatime)
sysfs on /sys/devices/virtual/net type sysfs (rw)osuid odev oexec,relatime)

I guess i have to change some configurations into the CT's configuration file (/etc/pve/lxc/xxx.conf), some lxc.aa_profile or lxc.mount.auto …

But i wonder, is LXC the best way to mount and use a RBD pool? As much as possible i want to use LXC CT but without adding security risks …

 

[dietmar]

Why don't you configure RBD on the proxmox host, and then simply our mountpoint feature?

 

[Gardouille]

Why don't you configure RBD on the proxmox host, and then simply our mountpoint feature?

The CT is on a Proxmox cluster, this means i would have to map RBD on the 3 hosts. But RBD pool will allow concurrent map from 3 hosts?

 

[dietmar]

The CT is on a Proxmox cluster, this means i would have to map RBD on the 3 hosts.

Why? The CT only runs on one host. Besides, I don't think it is a problem.

 

[Gardouille]

Hi!

Sorry for the delay

1. So on the host:

mkdir /mnt/ct/199121/test
/usr/bin/rbd map --pool rbd test --id test --keyring /etc/ceph/ceph.client.test.keyring
mount /dev/rbd/rbd/test /mnt/ct/199121/test

2. And on the container:

mkdir /mnt/rbdtest

3. And according to the wiki (lxc bind mounts) i add a line to the CT's config file (/etc/pve/node1/lxc/199121.conf):

mp0: /mnt/ct/199121/test,mp=/mnt/rbdtest

And it works on the first host (node1).

But if i reproduce the first step on a second node, then i shutdown the CT and try to migrate it, i got an error:
Feb 03 11:49:10 ERROR: migration aborted (duration 00:00:00): can't determine assigned storage for mountpoint 'mp0'
TASK ERROR: migration aborted

So i try the "manual method", still in the CT's config file (/etc/pve/node1/lxc/199121.conf):

lxc.mount.entry: /mnt/ct/199121/test mnt/rbdtest none bind,create=dir,optional 0 0

Then i can successfully migrate the CT

Seems good for me, unless you got any other tips