Question on Usage of Ceph in VMs as a Common Storage Device

[Seed]

Hello,

I have a ceph pool that I want to be able to mount to virtual machines sharing this mount to a few VMs as a shared storing, similar to NFS. How can i do this? Do I need to use cephfs in the VM? How can I create a disk that is shared to be able to mount to VMs across nodes?

Thanks!

 

[Alwin]

If the VMs have access to the Ceph cluster, than they can use RBD or CephFS, as the Proxmox VE nodes do. But maybe a more classical/flexibel approach is to use a NFS/SMB server that has its disks on RBD.

 

[Seed]

But maybe a more classical/flexibel approach is to use a NFS/SMB server that has its disks on RBD.

Yeah I’m trying to avoid using NFS if at all possible. Too many issues with file ownership in the past across hosts.

 

[Alwin]

Too many issues with file ownership in the past across hosts.

But that won't go away. CephFS is a POSIX compatible filesystem, hence has also user permissions.

 

[Seed]

But that won't go away. CephFS is a POSIX compatible filesystem, hence has also user permissions

Hmm. That sucks....

so is the idea then that I create a virtual disk on a vm then share that out to other vms?

maybe an alternative would be to create a disk on the ceph pool and use that for a few containers on the one node, and if that node takes a dump I should be able to remount that disk to another vm on a different node?

 

[Alwin]

Either use a NFS/SMB server (as VM) or a filesystem that is global lock aware (eg. OCFS2).
https://en.wikipedia.org/wiki/Clustered_file_system#SHARED-DISK