ext4 trim and Ceph

[alchemyx]

Hello, I have set up three node Ceph cluster to store VMs. It works very well. Altough I don't know whyfreeing spaces does not work. For example I create Linux VM with 500G disk. It only takes smallfraction of that space after installation. Then I create two big files let say 20G. With "ceph -w" I seethat "data usage" went up those 20G. Then I remove those files and execute in guest "fstrim -v /" andit says it trimmed that that data. But it is not freed in ceph. I am using IDE of course and ext4 insideof guest.What am I doing wrong?Thanks!

 

[spirit]

Hello, I have set up three node Ceph cluster to store VMs. It works very well. Altough I don't know whyfreeing spaces does not work. For example I create Linux VM with 500G disk. It only takes smallfraction of that space after installation. Then I create two big files let say 20G. With "ceph -w" I seethat "data usage" went up those 20G. Then I remove those files and execute in guest "fstrim -v /" andit says it trimmed that that data. But it is not freed in ceph. I am using IDE of course and ext4 insideof guest.What am I doing wrong?Thanks!

you need 3 more things:

use virtio-scsi (or scsi, but it's slower) for you guests disk controller.

and enable discard (you need to edit your vmid.conf file, and add "discard=on" to drive properties

and also,you need to enable discard on ceph-osd filesystem (xfs defaults,discard) for example.

 

[alchemyx]

you need 3 more things:use virtio-scsi (or scsi, but it's slower) for you guests disk controller.and enable discard (you need to edit your vmid.conf file, and add "discard=on" to drive propertiesand also,you need to enable discard on ceph-osd filesystem (xfs defaults,discard) for example.

Hello,

So I got somehow misleaded by this - http://ceph.com/docs/master/rbd/qemu-rbd/

"Note that this uses the IDE driver. The virtio driver does not support discard."

So adding discard=on to an IDE drive definition fixed it! Thanks!

And about last one - discard for OSD?
I don't know why would it be important in any way? I use xfs on spindle drive so discard seems to be unnecessary here.