[SOLVED] PVE3 + CEPH + TRIM (Discard) still not working

[athompso]

I'm reading in various other threads that Discard (TRIM) support actually works in CEPH-backed VMs, but that's not my experience:

[root@fs1 ~]# fstrim /
fstrim: /: FITRIM ioctl failed: Operation not supported

For that particular VM:

root@pve4:~# qm config 135-bash: warning: setlocale: LC_ALL: cannot change locale (en_CA.UTF-8)
bootdisk: virtio0
cores: 1
ide2: Hexen-PVE:iso/CentOS-6.6-x86_64-bin-DVD1.iso,media=cdrom
memory: 4096
name: fs1
net0: virtio=66:F2:CE:63:EC:66,bridge=vmbr0
numa: 1
ostype: l26
smbios1: uuid=64a10199-305f-4e85-acec-866db03bea73
sockets: 4
virtio0: rbd:vm-135-disk-1,cache=writeback,discard=on,size=32G

and of course:

root@pve4:~# pveversion pve-manager/3.4-11/6502936f (running kernel: 3.10.0-11-pve)

So... am I missing something obvious? I'd sure like to reclaim the (roughly) several hundreds of gigabytes of wasted space in my array! (Plus, backups should go a lot faster if I do this.)

The only thing I can find is from the CEPH docs at http://docs.ceph.com/docs/v0.94/rbd/qemu-rbd/:

For this to be available to the guest, it must be explicitly enabled for the block device. To do this, you must specify a discard_granularity associated with the drive:

and of course PVE does not provide that to QEMU... but if it never provides it, how on earth do other people make it work???

Thanks,
-Adam

 

[athompso]

Never mind. From the slightly newer CEPH documentation at http://docs.ceph.com/docs/master/rbd/qemu-rbd/#enabling-discard-trim, I also see:

Note that this uses the IDE driver. The virtio driver does not support discard.

So is there no way to get discard support without downgrading to the IDE driver? Even in a VM, that provides noticeably poor performance for me...

 

[mir]

Use scsi for disk (hardware tab)
Use virtio for scsi controller (Options tab)

 

[athompso]

Oh, OK... I thought that was equivalent to what using "virtio" in the first place did? If not, can you point me to something that explains the difference?

Edit: did some more reading; is "virtio" equivalent to "virtio-blk"?

 

[mir]

yes, virtio = virtio-blk. virtio-blk does not support trim. AFAIK development of virtio-blk has more or less stopped in favor of virtio-scsi so I doubt we well ever see trim coming to virtio-blk. See http://wiki.qemu.org/Features/VirtioSCSI