Moving Disks - External Array to Internal Hot Swap

Seed

Well-Known Member
Oct 18, 2019
101
63
48
124
Hello,

I have a couple HP Microservers with an 8088 card. This card attaches to an 8 bay array, really only 4 of them are in use, as it's a 2x8088. On this array are 3 disks. These 3 disks are 1/3 of a CEPH cluster in 3/2

What I want to do is shutdown this node and simply take the disks out of the array and put them inside the HP 8 in their hot swap bays, then remove the HBA and shutdown the external disk array that will then have 0 disks.

How well if at all will CEPH deal with this? Any ideas how I can do this if it's possible?
 
Hi,

What versions of Proxmox VE and Ceph are in use in this setup?

How well if at all will CEPH deal with this?

As you have three nodes and use 3/2 it will work, but it's pretty dangerous, you have no room left for failures. If another node fails, then Ceph is down (no reads, no writes).

So, can both the current slot and the intended target disk slot do hot-swap?

As then I'd do the following:

1. check if all systems are healthy, up do date, ...
2. identify which OSD belongs to which physical disk
3. set cluster-wide ceph noout to avoid meaningless object movement
3. set one OSD as out and down, hot-swap to new disk slot, check if all is well, start and put it in again (from Ceph POV)
4. if all was well, repeat for remaining OSDs
5. unset the cluster wide noout

Disclaimer: I did not tried this recently, but it should work.

If you cannot hot-swap during runtime, do 1., then 3., then power down + swap, then power up, after that check if the OSD are all there and started, then unset noout.

If a step fails (for both proposals) you can rollback (swap OSD back to original slot) and it should start and work again.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!