[SOLVED] Trim/discard With CEPH Rbd

[paulohnk1]

Hello

I realized that I'm deleting data from the vms but this space is not being released in Ceph. I found in the documentation that I should do a Fstrim in the RDB but I can't find its assembly text such as: fstrim /mnt/ myrbd.

Any idea?

Thank you

 

[Alwin]

I realized that I'm deleting data from the vms but this space is not being released in Ceph. I found in the documentation that I should do a Fstrim in the RDB but I can't find its assembly text such as: fstrim /mnt/ myrbd.

man fstrim
And the VM needs to have VirtIO SCSI as its disk controller.

 

[paulohnk1]

the vms already have the VirtIO SCSI driver and also fstrim was executed, but as in this article below, ceph's RBD does not see these deleted data, unless fstrim is executed on the RBD itself and on Proxmox I can't see where the RDB is mounted. I have more than 5TB of free space on the VM but Ceph does not.

https://ceph.io/geen-categorie/ceph-and-krbd-discard/

View attachment 20681

View attachment 20676

Thank You

 

[Alwin]

the vms already have the VirtIO SCSI driver and also fstrim was executed

Is the discard checkbox ticked on that disk?

fstrim is executed on the RBD itself and on Proxmox I can't see where the RDB is mounted.

It will not be and it doesn't need to be. Qemu uses librbd to talk with Ceph directly.

 

[paulohnk1]

Is the discard checkbox ticked on that disk?


It will not be and it doesn't need to be. Qemu uses librbd to talk with Ceph directly.

yes...

 

[paulohnk1]

I will reproduce the steps that worked for me:

rbd feature disable reply_3/vm-112-disk-0 object-map fast-diff deep-flatten

rbd map reply_3/vm-112-disk-0

rbd showmapped: 0 reply_3 vm-115-disk-0 - /dev /rbd0

mount /dev/rbd0 /mnt

fstrim -v /mnt

Now I can see the space released in Cephs

Thanks

 

[sahostking]

Hi

I've been searching aswell as we are playing with ceph with some test servers atm but found the same issue with an external source on rook-ceph which sais some options need to be enabled:

https://github.com/rook/rook/issues/6964

bdev_async_discard and bdev_enable_discard on the osds themselves

not sure if this could be it?