Deleting a ceph pool / storage is possible despite a running VM with disks on it!

[EllerholdAG]

Hello lovely people,

we're currently examining if PVE is a good alternative to our old ESXi environment and Ive set up some test machines (virtualized in ESXi) in order to test some things.

Ive created some Ceph pools, ran some tests (hard shutdown of the PVE nodes simulating failures) etc etc with different pool configurations.

Ive had 1 debian VM inside PVE to test when the ceph cluster stops accepting writes.

After I was done with one ceph pool I wanted to delete it. Sure enough I could!

A few moments later I discovered that my running VM was still on this exact ceph pool! There was no error in PVE "You cant delete storages / ceph pool that are still in use!". Am I right in thinking this is a bug?

Its no big deal because this was just a test VM and now I could test the restore functionality too, but I'd consider this a heavy bug and hopefully would never get this in a real (production) setup!

This was done on PVE 8.3.2 with ceph 19.2 (if thats any help).

 

[aaron]

After I was done with one ceph pool I wanted to delete it. Sure enough I could!

How did you remove the pool?

Because if I have a default situation and want to remove the pool via the GUI or the pveceph tool, I do get the error:

checking storage 'foo' for RBD images..
TASK ERROR: ceph pool 'foo' still in use by storage 'foo'

Did you remove the storage configuration beforehand? If so, then that is the step where we might want to think about preventing people from removing it too quickly.

 

[EllerholdAG]

I just tried it again:

• Click on One of the of Hosts -> Ceph -> Pools -> Click on "<pool>-data" -> Click Destroy ...
  • This does not work because it is still in use by a Storage!
  • Good!
• Click on Datacenter -> Storage -> Click on "<pool>" -> Remove (at the top).
  • This works! Storage successfully deleted!
  • Not good!
• One of the of Hosts -> Ceph -> Pools -> Click on "<pool>-data" -> Click Destroy
  • This works!
  • Technically this is correct because the corresponding storage is no longer there, but now all my data is lost!
• Do the same for "<pool>-metadata"
  • This works!
  • Technically this is correct because the corresponding storage is no longer there, but now all my data is lost!
• VM is still running but throws all kinds of errors... missing disk ;-)

Yeah its seems like you're correct and removing the Storage while still in use should not be possible (VM, CT, ...)