How to move a physical disk from one drive bay to an other in a proxmox node?

[ZKallo]

Hi,
we have a new, 4th node installed in a proxmox cluster but it's not fully configured. There is a 4TB SSD on it which has already an OSD created to test the extension of the storage space of the SSDDISK pool. We want to move one HDD disk from each old nodes to the new one but the SSD disk is on aesthetically wrong place. What should we do to move this disk to an other drive bay. Do we need to destroy the OSD?

Thanks
Zoltan

 

[Stoiko Ivanov]

What should we do to move this disk to an other drive bay. Do we need to destroy the OSD?

OSD detection should work independently of the slot a disk is plugged in (the disks get scanned for their meta-data and are then started) - however depending on how you setup your system moving a disk might rename it and other disks (e.g. /dev/sda becoming /dev/sdc) - if you never hardcoded any disk-names but used UUIDs (or something similarly persistent) this should also work. Finally it also depends a bit on the system/hardware (some raid-controllers (which btw. are not supported for Ceph OSDs) might simply not export the disk if it's in a different slot.

So summing up - it should work in most cases to simply shutdown the system plug the disk in the new slot and start it again.
Things I'd do just in case:
* create and test a backup of everything! (this is always a necessary prerequisite)
* set the noout flag before shutting down the node - to prevent unnecessary rebalancing - unset it after the node is back and the OSD got correctly started

I hope this helps!

 

[ZKallo]

Thank for the answer. I will try the process you've written in a test environment first before I do it in our production environment.

With regards,
Zoltan

 

[Stoiko Ivanov]

I will try the process you've written in a test environment first before I do it in our production environment.

Sounds like a very good idea!

Good luck and let us know how it went

 

[proxwolfe]

Sorry for hijacking this thread but my question is closely related:

Can I move an OSD from one _node_ to another without issues or is that going to pose a problem?

Thanks!

 

[Stoiko Ivanov]

Sorry for hijacking this thread but my question is closely related:

I'd suggest to open a new thread next time.

Can I move an OSD from one _node_ to another without issues or is that going to pose a problem?

one way to do this is to remove the osd from the first node, wipe it and create a new osd on the second node:
https://docs.ceph.com/en/mimic/rados/operations/add-or-rm-osds/

alternatively - check out this forum thread:
https://forum.proxmox.com/threads/move-osd-to-another-node.33965/

 

[proxwolfe]

It says there

if you don't have a separate (shared) journal device, you can just down and out the OSD and physically move it from one host to another. (hot-)plugging it in should automatically start the OSD service on the new host, and you can mark the osd as "in" on the GUI.

However, at least in my case, the OSD service does not start automatically.

I tried starting it manually with

systemctl start ceph-osd

But the system tells me

Failed to start ceph-osd.service: Unit ceph-osd.service not found.

(Note this is a newly installed Ceph node that does not run any Ceph devices. So that might be the reason why the services does not exist.)

How can I start (create?) the OSD service so that the device is recognized?

Thanks!

 

[ZKallo]

Sounds like a very good idea!

Good luck and let us know how it went

It worked in the test environment well but in the production environment stopping the node is not allowed momentarily by the CEO .
Is there any way not to stop the node? The disks are hot-plugged.