Ceph node on new HW

[stefanodm]

Hi everyone, maybe this question has already had answers, but I can't find anything definitive, so I'll try to ask you in the community.
We have a Proxmox cluster with 7 nodes of which 3 nodes have the sole function of Ceph node. (Without any Vm or CT).
One of these 3 nodes has different hardware than the other 2 but the OSD hard drives are the same and I would just replace the hardware to make them exactly the same. My idea is to simply replace Mainboard, CPU, RAM, HBA Controller and Network Cards, leave the same hard drives (Boot and OSD).
Reconfiguring offline the network, then restoring exactly the original network configuration with the new cards, and booting in my opinion should have no problems. Proxmox should configure the new hardware and start as if it was rebooted, but I'm probably missing something expecially form the Ceph point of view..Can you brief me about it?


proxmox-ve: 7.4-1

 

[supermicro_server]

Hi,
If you want to do a very crear soluzion, you can:

1) Tourn that node off
2) Delete that node from the Proxmox cluster
3) Change your hardware (Mainboard, CPU, RAM, HBA Controller and Network Cards, leave the same hard drives)
4) Re-Install proxmox (you could restore exacly the /etc/network/interface) but pay attention for the NIC name.
5) Join that node in to the Proxmox cluster
5) Install Ceph
6) Create OSD

I think that is the clenest approch,

MM

 

[stefanodm]

Hi,
If you want to do a very crear soluzion, you can:

1) Tourn that node off
2) Delete that node from the Proxmox cluster
3) Change your hardware (Mainboard, CPU, RAM, HBA Controller and Network Cards, leave the same hard drives)
4) Re-Install proxmox (you could restore exacly the /etc/network/interface) but pay attention for the NIC name.
5) Join that node in to the Proxmox cluster
5) Install Ceph
6) Create OSD

I think that is the clenest approch,

That's meaning that all osd's in that node must be destroyed before deleting that node from cluster if i'm not mistaken, itsn't?

 

[supermicro_server]

That's meaning that all osd's in that node must be destroyed before deleting that node from cluster if i'm not mistaken, itsn't?

I integrate the 2) point

• Delete that node from the Proxmox cluster
• After that, you will still see the OSDs (in down state) in the Ceph configuration
• Stop and destroy all OSD layer, ONE by ONE
  • ceph osd rm osd.X
  • ceph auth del osd.X
  • ceph osd crush remove osd.X
  • ceph osd crush rm <node_name>
  • ceph mon remove <node_name>
  • Check if there is still the monitor IP in /etc/pve/ceph.conf

My tip is try this proces in pre-production enviroment in orther to be more confident and than do it in production.

MM

 

[stefanodm]

I integrate the 2) point

• Delete that node from the Proxmox cluster
• After that, you will still see the OSDs (in down state) in the Ceph configuration
• Stop and destroy all OSD layer, ONE by ONE
  • ceph osd rm osd.X
  • ceph auth del osd.X
  • ceph osd crush remove osd.X
  • ceph osd crush rm <node_name>
  • ceph mon remove <node_name>
  • Check if there is still the monitor IP in /etc/pve/ceph.conf

My tip is try this proces in pre-production enviroment in orther to be more confident and than do it in production.

Thank you mate. Precious.