Ceph goes down after adding second boot disk for mirror


Renowned Member
Jan 11, 2012
Hi all,

I have a 3 node ceph cluster, consisting of HPE Gen9 servers. It's been running well since I set it up and I really enjoy the "no single point of failure" feature.

Now, during the installation I was using some S3700 100gb drives for boot zfs mirrors, however for one of the hosts, I only had one at the time. Now I want to add a second drive and convert the zfs pool to a mirror. Got it all working, but wanted to reboot the host in order to check that it can also boot with the second drive. Server boots fine but after 2-3 minutes at the proxmox login console, all my ceph services goes down.

Only way to get it back up is to either turn off the host or reboot it again without the newly added drive. Now, the new drive goes into port 2, whereas the ceph drives are plugged into 3-8. I suspect that the new drive changes the order of the drives (sda,b,c etc) and that ceph then falsely believes it's a ceph drive. However obviously it's not and ceph grinds to a halt. Now, the data and everything is secure, I had no issues there. It is however a little worrying. Tried to google around but couldn't really find anything.

Anyone tried this before? Where would I have to change the ceph config? I can imagine that each osd maps to a /dev/sdX and I suppose I need to find that file and change it.
Ceph knows about its disks, it check uuids and LVs for that. No order needed. Check in the logs what is happening during and after boot. This will give more clues to what is going on.


The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!