Proxmox 5 and ceph luminous: can't create monitor

[BCandler]

I'm getting the same trouble with creating the ceph monitor. Latest debian stretch, latest pve updates, three older machines running smoothly except the mentioned RocksDB-error.

CPUs:
Core 2 Quad Q6600 @ 2.4 GHz
Core 2 Duo E6550 @ 2.33 GHz
Core 2 Duo E6750 @ 2.66 GHz

I've exchanged the Core2Quad with a Xeon 3050 @ 2.13 GHz, but it didn't work either.
This information, logs and command lines are already uploaded to the bugtracker.

Later this weekend I will try to use other machines with a not-so-old Core i7 and maybe an AMD FX 8950 too.

 

[LEX]

Who can try the solution Daniel from ceph bugtracker No.20529?

 

[Norman Uittenbogaart]

ah so thats the errors I'm having with ceph.
I can't seem to create osd's, now I see this is related to my old Xeon processor.......

 

[spirit]

It's not yet solved, ceph devs are still working on it. (but luminous should be released before it's fixed)

 

[Norman Uittenbogaart]

It's not yet solved, ceph devs are still working on it. (but luminous should be released before it's fixed)

Really?
They are going to release a final version with such a big failure in it?

 

[tom]

Really?
They are going to release a final version with such a big failure in it?

AFAIK this bugs affects ONLY very old hardware.

 

[LEX]

Who can try the new Daniel's fix?

 

[spirit]

Who can try the new Daniel's fix?

I'm going to test Daniel's fix from today. (previous fix was not working for me)

 

[Norman Uittenbogaart]

I'm going to test Daniel's fix from today. (previous fix was not working for me)

did it work?

 

[spirit]

I'm currently in contact with Daniel, trying to debug that on my test platform. I'll report results in ceph tracker.

 

[spirit]

Hi,
I have made a successfull build with a different patch.
Working on my old xeon and also newer cpus.

can you test ?

http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/

 

[Norman Uittenbogaart]

Hi,
I have made a successfull build with a different patch.
Working on my old xeon and also newer cpus.

can you test ?

http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/

I will check, revert asap

 

[Norman Uittenbogaart]

It works with your patches!

 

[LEX]

can you build this path for proxmox repo?

 

[fabian]

once a final fix is applied upstream, it will either be part of the stable release anyway or we can try to backport it.

 

[spirit]

Yes, It need to be tested by ceph team, to see if they are no regression for new processor with sse4.2 support.
(It's mainly to have faster crc32, which is used by bluestore/rocksdb)

 

[LEX]

maybe for non-subscription repository for test, because I'm newbie and can't install the patch differently.

 

[Norman Uittenbogaart]

Hi Lex,

Do the following,

nano /root/ceph_install.sh

Paste the following in there,

#!/bin/bash

mkdir /root/ceph_install
cd /root/ceph_install

wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/ceph-mon_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/ceph-base_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/ceph-common_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/ceph-fuse_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/ceph-mgr_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/ceph-mds_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/ceph-osd_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/libcephfs2_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/python-ceph_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/python-cephfs_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/librados2_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/librbd1_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/python-rados_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/python-rbd_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/python-rgw_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/libradosstriper1_12.1.4-1_amd64.deb
wget http://odisoweb1.odiso.net/ceph12.1.4-runtimesse4/librgw2_12.1.4-1_amd64.deb

dpkg -i *.deb

save the file and enter

bash /root/ceph_install.sh

and it will install the patched files......

 

[LEX]

I did it. then tried to create a monitor using the web interface:

monmaptool: monmap file /tmp/monmap
monmaptool: generated fsid 7cd82a04-117b-4916-a83c-6903922af1c8
epoch 0
fsid 7cd82a04-117b-4916-a83c-6903922af1c8
last_changed 2017-08-22 18:47:34.858032
created 2017-08-22 18:47:34.858032
0: 192.168.1.70:6789/0 mon.node0
monmaptool: writing epoch 0 to /tmp/monmap (1 monitors)
TASK ERROR: unable to open file '/etc/pve/ceph.conf.tmp.2165' - Permission denied

 

[LEX]

tried again:

TASK ERROR: monitor filesystem '/var/lib/ceph/mon/ceph-node0' already exist.