CephFS Mount Question

[gdi2k]

I would like to get CephFS working (I have already been using RBD for quite some time with great success).

I've followed the guidance in the wiki to set up CephFS, and things seem fine.

I then tried to mount it on a Ubuntu 18.04 client VM, like:

root@server2:~# mount -t ceph 10.32.32.3:6789:/ /media/cephfs -o name=cephfs,secret=XXXXXXXXXXXXXXXXXXXXXX
mount error 110 = Connection timed out

dmesg shows:

[162518.366245] libceph: loaded (mon/osd proto 15/24)
[162518.378000] FS-Cache: Netfs 'ceph' registered for caching
[162518.378011] ceph: loaded (mds proto 32)
[162518.396567] libceph: mon0 10.32.32.1:6789 socket error on read
[162519.157343] libceph: mon0 10.32.32.1:6789 socket error on read
[162520.149407] libceph: mon0 10.32.32.1:6789 socket error on read
[162522.165390] libceph: mon0 10.32.32.1:6789 socket error on write
[162522.201367] libceph: mon0 10.32.32.1:6789 socket error on write
[162523.157475] libceph: mon0 10.32.32.1:6789 socket error on write
[162524.149132] libceph: mon0 10.32.32.1:6789 socket error on read
[162525.269416] libceph: mon0 10.32.32.1:6789 socket error on read
[162526.168917] libceph: mon0 10.32.32.1:6789 socket error on read
[162527.157306] libceph: mon0 10.32.32.1:6789 socket closed (con state CONNECTING)
[162528.341128] libceph: mon0 10.32.32.1:6789 socket error on write
[162529.173108] libceph: mon0 10.32.32.1:6789 socket error on write
[162530.165140] libceph: mon0 10.32.32.1:6789 socket error on read
[162531.157371] libceph: mon0 10.32.32.1:6789 socket error on write
[162532.149305] libceph: mon0 10.32.32.1:6789 socket error on write
[162533.173255] libceph: mon0 10.32.32.1:6789 socket error on read
[162534.229119] libceph: mon0 10.32.32.1:6789 socket closed (con state CONNECTING)
[162535.157144] libceph: mon0 10.32.32.1:6789 socket error on write
[162536.149148] libceph: mon0 10.32.32.1:6789 socket closed (con state CONNECTING)
[162537.301261] libceph: mon0 10.32.32.1:6789 socket error on read
[162538.165318] libceph: mon0 10.32.32.1:6789 socket error on read
[162539.157079] libceph: mon0 10.32.32.1:6789 socket error on read
[162540.373401] libceph: mon0 10.32.32.1:6789 socket error on write

Any idea what could be going wrong?

 

[Alwin]

162518.396567] libceph: mon0 10.32.32.1:6789 socket error on read

The VM doesn't seem to have access to the Ceph cluster.

 

[gdi2k]

You're absolutely right. It took me a couple of days to remember that ceph is set up to use a completely different network to the rest of proxmox. Makes total sense now. Will test tomorrow...

Thanks!