Moving Disks - External Array to Internal Hot Swap

[Seed]

Hello,

I have a couple HP Microservers with an 8088 card. This card attaches to an 8 bay array, really only 4 of them are in use, as it's a 2x8088. On this array are 3 disks. These 3 disks are 1/3 of a CEPH cluster in 3/2

What I want to do is shutdown this node and simply take the disks out of the array and put them inside the HP 8 in their hot swap bays, then remove the HBA and shutdown the external disk array that will then have 0 disks.

How well if at all will CEPH deal with this? Any ideas how I can do this if it's possible?

 

[t.lamprecht]

Hi,

What versions of Proxmox VE and Ceph are in use in this setup?

How well if at all will CEPH deal with this?

As you have three nodes and use 3/2 it will work, but it's pretty dangerous, you have no room left for failures. If another node fails, then Ceph is down (no reads, no writes).

So, can both the current slot and the intended target disk slot do hot-swap?

As then I'd do the following:

1. check if all systems are healthy, up do date, ...
2. identify which OSD belongs to which physical disk
3. set cluster-wide ceph noout to avoid meaningless object movement
3. set one OSD as out and down, hot-swap to new disk slot, check if all is well, start and put it in again (from Ceph POV)
4. if all was well, repeat for remaining OSDs
5. unset the cluster wide noout

Disclaimer: I did not tried this recently, but it should work.

If you cannot hot-swap during runtime, do 1., then 3., then power down + swap, then power up, after that check if the OSD are all there and started, then unset noout.

If a step fails (for both proposals) you can rollback (swap OSD back to original slot) and it should start and work again.