Cephfs MDS only reachable in cluster network

luigima

New Member
Aug 13, 2018
7
1
3
28
Hey,

I am a little bit confused or maybe I am just missing a small detail.
My plan is to deploy and mount a cephfs directory on a samba VM.
Thus I created a MDS in the proxmox cephfs GUI but the address of the MDS is from my cluster network where the OSDs are communicating.
Well, I tried to mount the cephfs directory and it appears that it only works if I provide the cluster ip address of the MDS instance.
According to the docs this should not be the case shouldn't it?

The MDS instances should be reachable from the public network, see here: http://docs.ceph.com/docs/mimic/_images/ditaa-2452ee22ef7d825a489a08e0b935453f2b06b0e6.png

Which leads me to the question what I am doing wrong? Is this intended? Do the samba VM need to be in the cluster network? Can I somehow change the ip address of the MDS instance?


Best Regards
luigima
 

Alwin

Proxmox Retired Staff
Retired Staff
Aug 1, 2017
4,617
453
88
If you used our default setup 'pveceph init' then the public and the cluster network do have the same IP range. You can separate them by editing the '/etc/pve/ceph.conf' and restart all Ceph services (may have a short disconnect).
 

luigima

New Member
Aug 13, 2018
7
1
3
28
This seems to be the right answer. I missed the configuration detail. Indeed, the cluster and public network are the same. However, I couldn't manage to migrate the network yet. Changing the ceph.conf file and then restarting all ceph services one by one does not seem to work. Results in timeouts..
 

Alwin

Proxmox Retired Staff
Retired Staff
Aug 1, 2017
4,617
453
88
It very much depends on which network you migrated, but all ceph services on all nodes need to be restarted to get a healthy cluster.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!