Hi all, I have a zfs pool with 6 1TB SSD drives. I've been doing a bit of testing with trim, and I don't really understand what Im seeing. Zpool trim on the host is supposed to clear space from VM disks right? "Discard" is set, It's set as scsi. I can fstrim on the VM, and the space gets freed just fine on the zfs pool on the host. But It doesn't seem to happen when I zfs trim on the host. Am I just misunderstanding it? Or forgetting something?
zfs trim
- Thread starter corbosman
- Start date
Not sure how that zpool trim is supposing to work but I can't image how that could replace a discard/trim from inside the guest. If you do a fstrim from the guest it knows what data is still being used and what should be deleted, because the guest has mounted the filesystem. If you then run fstrim it will tell the host what to delete.
I don't see how the host could know what to delete, specially if the host hasn't mounted the zvols.
I don't see how the host could know what to delete, specially if the host hasn't mounted the zvols.
Right lets start at the "beginning" 
The ways I've understand and use it:
Host: They give the KVM-QEMU VM guests a block device. If you allocate it on ZFS as a ZVOL, it's a block device for all practical purposes, and the guest can write there within the boundaries. BUT the fun here, once you've written to it, it's "allocated" according to ZFS. the methods to "un-allocate" it is to have a compression option set, which will "clear out" like a record full of zeroes... or when the ZSFS is told that block is "trimmed" (Need to check how well this actually works)
Guests: They see a a block device. When you've enabled the TRIM options, a FSTRIM type command given to the SCSI/SATA bus, will be forwarded to the host's storage. That is totally OS/Filesystem INSIDE the guest dependent on when/how/if/whether it'll sent the TRIM commands or not. Another method to clear the storage, is of course to write zeroes on all the empty blocks which the compressing ZFS volume will clear out.
zpool trim: (Believe it or not, but I run zfs inside my VM too
) this is a feature where ZFS will scan that zpool's blocks, and issue TRIM commands to the underlying storage based on the views from ZFS whether that blocks is used or not. It's NOT issueing any qm guest commands from a host, and neither pushing anything from inside the guest out... only looking at the blocks, and the unused blocks are then send as FSTRIM/TRIM commands to the underlying storage device.
There are some qemu-guest-agent commands to initiate FSTRIMs, but I found it's easier to run the TRIM/sdelete/etc. commands inside the VM to zero the blocks, rewritten as "empty" by the compressing ZFS volume, and then the ZFS TRIM just works.
The ways I've understand and use it:Host: They give the KVM-QEMU VM guests a block device. If you allocate it on ZFS as a ZVOL, it's a block device for all practical purposes, and the guest can write there within the boundaries. BUT the fun here, once you've written to it, it's "allocated" according to ZFS. the methods to "un-allocate" it is to have a compression option set, which will "clear out" like a record full of zeroes... or when the ZSFS is told that block is "trimmed" (Need to check how well this actually works)
Guests: They see a a block device. When you've enabled the TRIM options, a FSTRIM type command given to the SCSI/SATA bus, will be forwarded to the host's storage. That is totally OS/Filesystem INSIDE the guest dependent on when/how/if/whether it'll sent the TRIM commands or not. Another method to clear the storage, is of course to write zeroes on all the empty blocks which the compressing ZFS volume will clear out.
zpool trim: (Believe it or not, but I run zfs inside my VM too
) this is a feature where ZFS will scan that zpool's blocks, and issue TRIM commands to the underlying storage based on the views from ZFS whether that blocks is used or not. It's NOT issueing any qm guest commands from a host, and neither pushing anything from inside the guest out... only looking at the blocks, and the unused blocks are then send as FSTRIM/TRIM commands to the underlying storage device.There are some qemu-guest-agent commands to initiate FSTRIMs, but I found it's easier to run the TRIM/sdelete/etc. commands inside the VM to zero the blocks, rewritten as "empty" by the compressing ZFS volume, and then the ZFS TRIM just works.