Web UI cannot create CEPH monitor when multiple pulbic nets are defined

[Whatever]

According to CEPH docs (https://docs.ceph.com/docs/master/rados/configuration/network-config-ref/#id1) several public nets could be defined (useful in case of rdb mirroring when slave CEPH cluster is located in separate location or/and monitors need to be created on different network interfaces/addresses)

public_network = 10.10.10.0/24, 10.71.200.0/24

However when trying to create mon from Web UI I get an error:

command '/sbin/ip address show to '10.10.10.0/24, 10.71.200.0/24' up' failed: exit code 1 (500)

Could anyone suggest how to properly create mon in such setup?
Thanks in advance

 

[Alwin]

Could anyone suggest how to properly create mon in such setup?

This seems not to be handled, yet.

Try on the CLI, pveceph mon create -mon-address <IP>.

 

[t.lamprecht]

FYI tracked under https://bugzilla.proxmox.com/show_bug.cgi?id=2422 (from this threads OP, it seems )

 

[Whatever]

FYI tracked under https://bugzilla.proxmox.com/show_bug.cgi?id=2422 (from this threads OP, it seems )

Well, this report is mine)
This post mainly about asking for any advice to workaround

 

[Whatever]

This seems not to be handled, yet.

Try on the CLI, pveceph mon create -mon-address <IP>.

Will give a try. Thank!

 

[kifeo]

Hello,

I'm in the same situation, the CLI command gives :
root@proxmox1:~# pveceph osd create /dev/nvme0n1
Error: any valid prefix is expected rather than "192.168.1.10/24, 10.0.5.0/24".
command '/sbin/ip address show to '192.168.1.10/24, 10.0.5.0/24' up' failed: exit code 1

any other hint for a workaround ?

 

[kifeo]

OK, I could workaround the situation by creating the VLAN interface in the other network 192.168.1.0/24.
I removed the 10.0.5.0/24 entry in the ceph.conf

Thanks !

 

[Alwin]

If you are on Ceph Nautilus, the mon_host line needs to exist, then you may remove the public_network to let Ceph auto-detect the networks. But you will need to test, if this work for a service restart.

 

[kifeo]

Hi, me again.
Is the patch available in a certain pve version ?
I'm in 6.3-3 and the issue is still present.

EDIT: Looking at the code it seems it is only for mon, and not osd ?

 

[Alwin]

Is the patch available in a certain pve version ?

not yet available but on the todo list for a v2.

 

[kifeo]

Hello,
I wanted also to create a monitor with the command 'pveceph mon create -mon-address 10.0.15.12'

I still have the issue in 6.3-6

Error: any valid prefix is expected rather than "192.168.1.10/24, 10.0.10.0/24".
command '/sbin/ip address show to '192.168.1.10/24, 10.0.10.0/24' up' failed: exit code 1

I thought the fix #2422 did fix it for the monitors ?

 

[kifeo]

as for reference, I added it manually :
https://docs.ceph.com/en/latest/rados/operations/add-or-rm-mons/

 

[kifeo]

I created a script for the workaround : https://github.com/kifeo/deploy-ceph-monitor

 

[Casual]

Same problem, can't add new OSD when using few networks, PVE 8.1.4

 

[Casual]

```
root@pve03:~# pveceph osd create /dev/sdf
Error: any valid prefix is expected rather than "10.10.10.10/24 10.10.0.100/24".
command '/sbin/ip address show to '10.10.1.10/24 10.10.0.100/24' up' failed: exit code 1

```
Same in GUI.

 

[fiona]

Hi,

```
root@pve03:~# pveceph osd create /dev/sdf
Error: any valid prefix is expected rather than "10.10.10.10/24 10.10.0.100/24".
command '/sbin/ip address show to '10.10.1.10/24 10.10.0.100/24' up' failed: exit code 1

```
Same in GUI.

do you maybe have multiple entries for the cluster_network in the Ceph configuration? That is currently not properly supported in Proxmox VE, only having multiple entries for public_network is. Feel free to open a bug report so we can keep track of the issue: https://bugzilla.proxmox.com/