[SOLVED] Ceph server - Full Mesh Network Method 2 - Config Problem

[fxandrei]

So i have 3 servers with proxmox installed, and want to install ceph on them.

Besides the 4 network ports for cluster comminications, public network, etc, i also have 2 dedicated ports that i want to use for ceph.

So i have this tutorial : https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server#Method_2
So in this example you can see that you have 2 ports configured with the same IP.

Thats what i did exactly.

The problem is that i get an error when i try to create the ceph monitors.

So if i execute :
pveceph createmon
i get :
Multiple IPs for ceph public network '10.100.100.0/24' detected on hp1-s1:
10.100.100.1
10.100.100.1
use 'mon-address' to specify one of them.

The ip 10.100.100.1 is set on both ports.


So what can i do ? Is this normal ?!

 

[Alwin]

Remove the IP of one interface during the creation and add it later on. The Ceph MON may need to be restarted after the second interface has its IP.

 

[fxandrei]

I ended up running :
pveceph createmon --mon-address 10.100.100.1

I did not change anything on the nics.
It seems to have worked.

 

[GoZippy]

I ended up running :
pveceph createmon --mon-address 10.100.100.1

I did not change anything on the nics.
It seems to have worked.

This was very helpful - I have same net range on cluster and private but different switches - one nic for private IP to router for public wan and then 4 nic per node bonded for server to server on different ip but all connected up to main switch through another server server switch... anyhow - had to tell createmon to use the vmbr address for each node attached to the bond so it would look there first apparently... would be nice to also be able to have the mon fail over to other nic and ip address on same nodes... any ideas there?

 

[freeflyer]

Same here. Servers connected with LACP on 2 physical wire..

pveceph createmon --mon-address

Helped a lot.. Thanks