Ceph Storage Unknown Status Error

[ks1]

Due to move of our servers to a new location, I had to change the IP Addressing on all 3 Servers. After changing the IP's on each server and changing the IP's in /etc/pve/ceph.conf, the CEPH storage is still grayed out on each Server and get Status unknow timeout Error (500). It appears that can't connect to the CEPH Cluster as shown in the below screenshot. How can I fix this? Thanks

 

[gurubert]

You cannot just change IPs in ceph.conf.
The first step is to add the new network to the Ceph public_neteork setting, then add new MONs with the new IPs to the cluster and after that remove the old MONs.
Only after that was successful the old network can be removed from public_network and the IPs of the new MONs written to ceph.conf
The last step would be to restart all OSDs, MGRs and MDSs so that they boot with an IP in the new public_network.
With ony three nodes you would do this node for node and you would need to make sure that all nodes see the old and the new IP network.

 

[ks1]

I have already changed ceph configuration in adding the the new IP addresses which is shown in my screen shots above. I removed the old IP address from the configuration. Do I need to re add the old IP's or am I out luck and remove CEPH and start over?

It does not allow you to create new monitors. Gives following error message that can't connect to ceph cluster.