Hello,
I am new in proxmox and after adding a new node to a proxmox cluster (I had 3 nodes and wanted to add one more), I realized that I was using a wrong IP xx.xx.80.195 instead of xx.xx.80.196, so I have stopped mds and monitor and modified the IP, but after that I created the monitor again and it was stopped, no port and no version.
I destroyed it and put back the IP I had before, which makes things worst as now I and getting error: "monitor address 'xx.xx.80.196' already in use (500)".
What I have done:
- Check if a ceph monitor service exists (systemctl status ceph-mon@my-node). It does and was running, so I stopped it and disabled it with systemctl disable ceph-mon@my-node
- Some threads are pointing to files whit the node name under /var/lib/ceph/mon/ and /etc/systemd/system/ceph.target.wants/, but in my case there is nothing.
- In the UI under monitor tab there is no monitor with my new node name. so it is not exist.
- I have 2 ceph pools and one CephFS for the ISOs if I try to see the content of them I cannot see the ISOs of my ISOs-vol01 pool. I am getting the error message: "unable to activate storage 'ISOs-vol01' - directory '/mnt/pve/ISOs-vol01' does not exist or is unreachable (500)"
- The file where I can see the 2 IPs I mentioned before is in the ceph.conf file: "mon_host = xx.xx.80.168 xx.xx.80.167 xx.xx.80.166 xx.xx.80.195 xx.xx.80.196"
- I also did a mkdir -p ceph-my-node-name and then pveceph mon destroy ceph-my-node-name to see if it will remove the IP some where.
My questions are:
- Can I just remove the 2 IPs from the /etc/ceph/ceph.conf file and after that create agin the monitor. Will be ceph good with that way?. I have read someone talking about inject a monmap, but this is a production environment, so I am wondering if there is something simpler.
- If it is ok to just edit the ceph.conf file I guess I will have to reboot all monitors in the cluster, is it safe to do that? I currently have 3 if I reboot one I guess it will be the same as if I reboot the node and everything will be fine, right? should I set the noout flag before?
- I even was thinking to remove the node completely from the proxmox cluster, but I am not sure this will remove the 2 IPs from ceph.conf file or any other place where it is configured.
I hope someone can give me some guide to solve the issue.
Thank you
I am new in proxmox and after adding a new node to a proxmox cluster (I had 3 nodes and wanted to add one more), I realized that I was using a wrong IP xx.xx.80.195 instead of xx.xx.80.196, so I have stopped mds and monitor and modified the IP, but after that I created the monitor again and it was stopped, no port and no version.
I destroyed it and put back the IP I had before, which makes things worst as now I and getting error: "monitor address 'xx.xx.80.196' already in use (500)".
What I have done:
- Check if a ceph monitor service exists (systemctl status ceph-mon@my-node). It does and was running, so I stopped it and disabled it with systemctl disable ceph-mon@my-node
- Some threads are pointing to files whit the node name under /var/lib/ceph/mon/ and /etc/systemd/system/ceph.target.wants/, but in my case there is nothing.
- In the UI under monitor tab there is no monitor with my new node name. so it is not exist.
- I have 2 ceph pools and one CephFS for the ISOs if I try to see the content of them I cannot see the ISOs of my ISOs-vol01 pool. I am getting the error message: "unable to activate storage 'ISOs-vol01' - directory '/mnt/pve/ISOs-vol01' does not exist or is unreachable (500)"
- The file where I can see the 2 IPs I mentioned before is in the ceph.conf file: "mon_host = xx.xx.80.168 xx.xx.80.167 xx.xx.80.166 xx.xx.80.195 xx.xx.80.196"
- I also did a mkdir -p ceph-my-node-name and then pveceph mon destroy ceph-my-node-name to see if it will remove the IP some where.
My questions are:
- Can I just remove the 2 IPs from the /etc/ceph/ceph.conf file and after that create agin the monitor. Will be ceph good with that way?. I have read someone talking about inject a monmap, but this is a production environment, so I am wondering if there is something simpler.
- If it is ok to just edit the ceph.conf file I guess I will have to reboot all monitors in the cluster, is it safe to do that? I currently have 3 if I reboot one I guess it will be the same as if I reboot the node and everything will be fine, right? should I set the noout flag before?
- I even was thinking to remove the node completely from the proxmox cluster, but I am not sure this will remove the 2 IPs from ceph.conf file or any other place where it is configured.
I hope someone can give me some guide to solve the issue.
Thank you