Ceph add monitor any valid prefix is expected rather than "192.168.0.0/24, 192.168.1.0/24".

TheMrg

Active Member
Aug 1, 2019
118
4
38
42
pveceph mon create
Error: any valid prefix is expected rather than "192.168.0.0/24, 192.168.1.0/24".
command '/sbin/ip address show to '192.168.0.0/24, 192.168.1.0/24' up' failed: exit code 1

what is the reason?
proxmox 6.1-8
ceph 14.2.8

ceph.conf:
[global]
<------> auth_client_required = cephx
<------> auth_cluster_required = cephx
<------> auth_service_required = cephx
<------> cluster_network = 192.168.0.0/24
<------> fsid = 00cf7097-2cb5-47f6-b342-e55094bf839e
<------> mon_host = 192.168.0.20 192.168.0.2 192.168.0.1 192.168.0.3
<------> mon_initial_members = storage3 storage2
<------> public_network = 192.168.0.0/24, 192.168.1.0/24

[client]
<------> keyring = /etc/pve/priv/$cluster.$name.keyring

PS:
Create new OSDs on storage4 works without any issue.
We create 4 new OSDs successfully.

Thanks
 
Last edited:
thanks.
Ok, is there an other workaround? f.e. create mon with ceph not with proxmox?
or how are the steps? should i edit the ceph.conf ?
we need the two networks.

Thanks so much. system is productive, so our ceph cluster have to stay online.
 
ok we edit manually the ceph config. add monitor and add the other network again to ceph.conf.
But with monitor we get:

Apr 14 12:43:37 storage4 ceph-mon[125140]: 2020-04-14 12:43:37.908 7f68e236e700 -1 mon.storage4@-1(probing) e0 handle_auth_bad_method hmm, they didn't like 2 result (13) Permission denied

manually edit the keyring to the key ring of other mons:
/var/lib/ceph/mon/ceph-storage4
mon is up, but in log:
Apr 14 13:08:03 storage4 ceph-mon[168545]: 2020-04-14 13:08:03.531 7f4f6446e700 -1 mon.storage4@4(electing) e9 failed to get devid for : fallback method has serial ''but no model

and is it a good idea to manually change the /var/lib/ceph/mon/ceph-storage4/keyring file?

Thanks so much
 
Last edited:
Both networks need to be routed between each other.

and is it a good idea to manually change the /var/lib/ceph/mon/ceph-storage4/keyring file?
If the keyring or the MON exists on the cluster, then yes.
 
Both networks need to be routed between each other.


If the keyring or the MON exists on the cluster, then yes.

Thanks for answer. What do you mean with: "Both networks need to be routed between each other." ?

ok now we have 4 mon nodes with all the same /var/lib/ceph/mon/ceph-..../keyring file. hope this is ok. storage1-3 hase the same file (key). add new monitor on storage4, /var/lib/ceph/mon/ceph-storage4/keyring get an other key. we delete the key and added the key from
/var/lib/ceph/mon/ceph-storage1/keyring.

now all 4 /var/lib/ceph/mon/ceph-storageN/keyring files are the same.
 
Thanks for answer. What do you mean with: "Both networks need to be routed between each other." ?
Traffic from one network needs to be able to pass into the other network. Form all Ceph nodes.

ok now we have 4 mon nodes with all the same /var/lib/ceph/mon/ceph-..../keyring file. hope this is ok. storage1-3 hase the same file (key). add new monitor on storage4, /var/lib/ceph/mon/ceph-storage4/keyring get an other key. we delete the key and added the key from
/var/lib/ceph/mon/ceph-storage1/keyring.
Everything should be done automatically by the pveceph mon create. Once the multiple networks work with our tooling, the manual edit of the ceph.conf shouldn't be necessary anymore.
 
Thanks, but with pveceph mon create we get:
mon.storage4@-1(probing) e0 handle_auth_bad_method hmm, they didn't like 2 result (13) Permission denied

we edit the /var/lib/ceph/mon/ceph-storage4/keyring to the key all other mon nodes uses,
now all is fine
hope this is ok. sometimes we get:
(electing) e9 failed to get devid for : fallback method has serial ''but no model
 
(electing) e9 failed to get devid for : fallback method has serial ''but no model
This message is not related to that issue. It doesn't hinder running Ceph.
 
thanks very much.
ok, it's from ceph monitor:

Apr 14 13:19:13 storage4 ceph-mon[168545]: 2020-04-14 13:19:13.760 7f4f6446e700 -1 mon.storage4@4(electing) e9 failed to get devid for : fallback method has serial ''but no model
Apr 14 13:19:13 storage4 ceph-mon[168545]: 2020-04-14 13:19:13.760 7f4f6446e700 -1 mon.storage4@4(electing) e9 failed to get devid for : fallback method has serial ''but no model
Apr 14 13:19:13 storage4 ceph-mon[168545]: 2020-04-14 13:19:13.760 7f4f6446e700 -1 mon.storage4@4(electing) e9 failed to get devid for : fallback method has serial ''but no model
Apr 14 13:20:12 storage4 ceph-mon[168545]: 2020-04-14 13:20:12.707 7f4f6446e700 -1 mon.storage4@3(electing) e10 failed to get devid for : fallback method has serial ''but no model
Apr 14 13:20:12 storage4 ceph-mon[168545]: 2020-04-14 13:20:12.707 7f4f6446e700 -1 mon.storage4@3(electing) e10 failed to get devid for : fallback method has serial ''but no model
Apr 14 13:20:17 storage4 ceph-mon[168545]: 2020-04-14 13:20:17.507 7f4f6446e700 -1 mon.storage4@3(electing) e10 failed to get devid for : fallback method has serial ''but no model
Apr 15 07:59:23 storage4 ceph-mon[168545]: 2020-04-15 07:59:23.343 7f4f6446e700 -1 mon.storage4@3(electing) e10 failed to get devid for : fallback method has serial ''but no model
Apr 15 07:59:23 storage4 ceph-mon[168545]: 2020-04-15 07:59:23.343 7f4f6446e700 -1 mon.storage4@3(electing) e10 failed to get devid for : fallback method has serial ''but no model
Apr 15 07:59:23 storage4 ceph-mon[168545]: 2020-04-15 07:59:23.343 7f4f6446e700 -1 mon.storage4@3(electing) e10 failed to get devid for : fallback method has serial ''but no model
Apr 15 07:59:50 storage4 ceph-mon[168545]: 2020-04-15 07:59:50.832 7f4f6446e700 -1 mon.storage4@2(electing) e11 failed to get devid for : fallback method has serial ''but no model
Apr 15 07:59:50 storage4 ceph-mon[168545]: 2020-04-15 07:59:50.832 7f4f6446e700 -1 mon.storage4@2(electing) e11 failed to get devid for : fallback method has serial ''but no model
Apr 15 07:59:52 storage4 ceph-mon[168545]: 2020-04-15 07:59:52.840 7f4f6446e700 -1 mon.storage4@2(electing) e11 failed to get devid for : fallback method has serial ''but no model
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!