Moving Docker Volumes from NFS Mounts to Ceph FS mounts inside LXC. Suggestions?

[juanjax]

Hi Folks.

I'm relatively new to ProxMox. I have a PVE which consist of three nodes which are configured in HA; sharing data via NFS.

NFS is used for PVEs Volumes, ISOs, Templates as well as shared volumes to Docker containers (data) which are hosted of ProxMox LXCs. Everything is working great. However, I've been wanting to eliminate the Single Point of Failure introduced by the NFS. So I configured Ceph with 4tb SSD; one in each node (3 total). I moved all the data within proxmox environment off the NFS and unto the Ceph Storage environment.

However, now I need to migrate or move the docker volumes to Ceph. I will like to move it using Ceph FS. The logic here, is to streamline the process by just having to change the existing NFS mount points to reference the new CephFs mounts. (as opposed to having to modify all docker volumes manually; which would be necessary if I was to use just the Ceph storage via the LXCs.)

I'm just not to clear of how to go about:

1. Mounting Ceph FS on Nodes so it becomes available to LXCs.
2. Best way to move (rsync?) the NFS docker content unto the Ceph FS mount.

Any assistance with this is greatly appreciated. Tons of video on youtube about ProxMox Ceph and storage in general; but nothing on ProxMox CephFS.

Thank you in advance!!!!

 

[SatchK]

Hi mate,

I'm in process doing a very similar thing trying to set up a docker swarm on LXCs with CephFS for volume storage.

Mounting in the LXC is really straight forward, from the PVE host with the LXC:

pct set <lxc ID> -mp0 /mnt/pve/<cephfs name>,mp=/mnt/cephfs,shared=1

so eg:

pct set 105 -mp0 /mnt/pve/cephfs,mp=/mnt/cephfs,shared=1

 

[SatchK]

Keep in mind that the LXC conf file also needs to be updated if you're using unprivileged containers with this:

https://pve.proxmox.com/wiki/Unprivileged_LXC_containers

 

[SatchK]

And now another resource:

https://gist.github.com/Drallas/e03eb5a4f68bb526f920a423455bc0c9