I showed plenty of room on my ceph when i went to move a disk but its says 1 osd full so i stopped move but it still shows error how do i delete the move and go back to using the vm where it was
help full OSD need help
- Thread starter Andrew Holybee
- Start date
Had a vm running on a nfs path and click moved disk to ceph. It had a issue in the move as it filled 1 OSD more then the others and halted the move. When I clicked stop I can't figure out how to delete the big copy I was hoping it would just go back to the path it was coming from but now my ceph is showing its still having issues. There seems to be false copies of the vm see attached, we should only have 1 103 vm and its a large one but it says there are multiples however its just 1 OSD that is near full not all of them so this seems to be a false reading.
We tried manually removing the disks but it didn't work using
rbd list --pool name -> to see your disks
rbd info --pool name vm-xxx-disk-x-> information about your disk
rbd rm --pool name vm-xxx-disk-x -> to delete the disk
consultant was able to get it VM's working but it hasn't healed. Any ideas on how to find bad backup and delete?
We tried manually removing the disks but it didn't work using
rbd list --pool name -> to see your disks
rbd info --pool name vm-xxx-disk-x-> information about your disk
rbd rm --pool name vm-xxx-disk-x -> to delete the disk
consultant was able to get it VM's working but it hasn't healed. Any ideas on how to find bad backup and delete?
Attachments
The only thing I can think is when we did the above commands we didn't have a dash after the disk and it said invalid directory.
rbd info --pool name vm-xxx-diskx-> instead of disk-x
rbd rm --pool name vm-xxx-diskx -> instead of disk-x
I am wondering if that is what did it
rbd info --pool name vm-xxx-diskx-> instead of disk-x
rbd rm --pool name vm-xxx-diskx -> instead of disk-x
I am wondering if that is what did it
seems like syntax is still hanging us up I can't even get rbd info to work can someone help me on the syntax here is a screen shot. You can see all the extra disks for vm103 I am just trying to run info so I can get info on which one is the right one and which is just filling up ceph.
Attachments
You can just delete them using the command
rbd -p *poolname* rm *diskname*
So yours should be
rbd -p cephStor rm vm-103-disk-1
rbd -p *poolname* rm *diskname*
So yours should be
rbd -p cephStor rm vm-103-disk-1
disks 1 and 2 should be the good ones 3, 4 and 5 should be the bad ones what would syntax be for info so I can check would it be
rbd -p cephStor info vm-103-disk-1
rbd -p cephStor info vm-103-disk-1
Awesome thanks i was just making sure the way I typed the info command was right. Attached you can see the vm 103 hardware screen where it shows disks 1 and 2 in use and not the other 3 /4/5 and in the OSD percentage you can see the one that is almost full preventing it from fully healing. It is backing up that vm so I am hoping it sees those other disks aren't being used and removes them. if not i will try and remove it this way thanks so much.
Attachments
This fixed it thanks so much I have removed disk 5 will be waiting til after hours to remove 4 and 3. Saved my bacon what a great community.
Yes this is what I wanted to try first would syntax be "ceph osd unset noout" ?
And pm05 not being a part of ceph wouldn't be affected by this correct?
And pm05 not being a part of ceph wouldn't be affected by this correct?
ok cool, pm05 is a new server that we added so it was never part of ceph. The issue we have had with ceph and proxmox is it will seemingly randomly loose quorum and fence. Or I will take down 1 node for maint and it will take down another node or multiple nodes.
looks like we just have the 3 pgs that haven't been restored is there a command that I need to manually get them to repair?
Last edited: