discard=on - why not default ?

[RolandK]

hello,
i found that discard=on (and using fstrim -a -v inside VM) is saving an astonishing amount of diskspace in our environment. i have VMs which shrink from 100GB to 20GB....

any reason why it's not set to ON by default when adding a virtual disk ?

 

[sg90]

discard = on whilst as you can say can save quite alot of storage as the disks are constantly made thin, it can put quite a large amount of IO onto the underlying disks if a guest suddenly removes or writes lots of data, as this all needs to be flushed.

So I imagine this is why it's off by default, and turned on if someone wants the benefits and their storage can handle the extra load.

 

[LnxBil]

any reason why it's not set to ON by default when adding a virtual disk ?

it only works on special storage backends, so this could be missleading if it's on by default. We can argue, that it would be good if you have ZFS, which always supports trimming in the guest but will not always really free the space completely if snapshots are used. With QCOW2, there are also some restrictions in place, so a general "discard yes" does not automatically imply that it will work in all circumstances.

 

[RolandK]

thanks for your replies

>With QCOW2, there are also some restrictions in place,

which ?

 

[LnxBil]

thanks for your replies

>With QCOW2, there are also some restrictions in place,

which ?

QCOW2 is sparese and discard is issued, but already allocated space is not freed until you offline convert the qcow2 and therefore compact the virtual disk image. (see qemu-img convert)

 

[RolandK]

that can't be true - what would explain then that i see the proxmox host's filesystem freespace shrink when i copy data into a VM and then see it grow after deletion and fstrim -a ?

 

[LnxBil]

that can't be true - what would explain then that i see the proxmox host's filesystem freespace shrink when i copy data into a VM and then see it grow after deletion and fstrim -a ?

The problem is that the underlying qcow2 does (in general) not use the same allocation size as your VM, so that you will e.g. have 2M allocation blocks in your QCOW2 and default 4K in your VM. Only if your free all 2MB of one allocation, the space could be reclaimed. Otherwise it would still allocate a 2M block but only with 4K of useful data in it. The whole space allocation problem with files is highly fragmetation related and hard to estimate. Therefore, often people compact (this is also true for other systems like VMware and Hyper-V) the disks regularly, use just flat files for better space estimates or use a storage backend that handels this automatically (in PVE this is e.g. ZFS, LVM-thin und CEPH with different blocksizes though).

 

[chrcoluk]

I think it should be on by default, a pfSense inside a VM with ZFS in the guest, I have space that cant be recovered now as it doesnt support trim on demand, so the space lost by discard been off is lost until I redo the guest filesystem (which I believe will trigger a device wide trim) or make a new ZVOL.

--

Reinstalled now, the vdev trim did occur, so several gig freed up, and now discard is enabled the trim for every transaction should keep space consumption down, pfSense only supports auto trim, not manual retrim.

 

[SInisterPisces]

This thread came up when I was searching for info on what to do with this checkbox.

What should enable for a VM's boot disk on a sparse ZFS pool? The underlying storage is a single NVME disk that does support trim via fstrim under Linux.

 

[LnxBil]

What should enable for a VM's boot disk on a sparse ZFS pool?

As already said above: with ZFS, enable discard everywhere and you're fine.

 

[SInisterPisces]

As already said above: with ZFS, enable discard everywhere and you're fine.

Thanks! I did a bit more reading yesterday and found that.
Sorry about that.