VM/CT Creation Issue When a OSD is "down"

HE_Cole

Member
Oct 25, 2018
45
1
6
32
Miami, FL
Hello Everyone!

I recently created and setup a Promox and Ceph cluster on my 5 servers and i am very happy with it so far.

I went to create my first CT and i was getting a error

error with cfs lock 'storage-VPS-MAIN-STOR1_ct': rbd rm 'vm-100-disk-2' error: rbd: error: image still has watchers
TASK ERROR: can't mount rbd volume vm-100-disk-2: received interrupt

AND

rbd: sysfs write failed
error with cfs lock 'storage-VPS-MAIN-STOR1_ct': can't unmap rbd volume vm-100-disk-0: rbd: sysfs write failed
TASK ERROR: command 'mkfs.ext4 -O mmp -E 'root_owner=100000:100000' /dev/rbd/VPS-MAIN-STOR1/vm-100-disk-0' failed: received interrupt

I checked my my OSD's and i noticed one was showing up as a partion not a osd.28 etc for some reason in the ceph logs i see
2018-11-24 17:33:07.057149 mon.he-s01-r01-pve01 mon.0 10.10.11.3:6789/0 12179 : cluster [INF] osd.28 failed (root=default,host=he-s04-r01-pve01) (connection refused reported by osd.13)

So i went to the server in the GUI under OSD and i set the bad OSd as out and destroyed it.

Now i can create CT's and VM's with no issues.

Is it normal when you lose a OSD for ceph/promox to give errors like above on ct/vm creation?

After destroying the osd i am unable to add it back and it is listed as a disk under disks but shows as PARTION, how do i re add it back after i destroyed it?


"I am going to replace that drive FYI but for after i replace it i need to be able to re add it without a reboot how do i do this?"

Thanks.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!