zpool trim in Proxmox 6.1

[ThinkAgain]

Am I correct in assuming that I will need to create a cron job to run "zpool trim" on, e.g., a weekly basis?

Because the "autotrim" property doesn't appear to be set on zpools by default (neither "zfs get all" nor "zdb" show it), and I couldn't find any existing cronjob or timer that runs "zfs trim". And fstrim.timer (which can be enabled via "systemctl enable fstrim.timer") appears to be about fstrim, which I don't think works for zfs.

To clarify: This question is not about reducing the space used by thin provisioned VMs (which usually is addressed by running fstrim in VMs). I am looking to use the trim feature in zfs 0.8 for the SSDs built into the Promox host.

Thanks!

 

[Thalix]

I think zpool set autotrim=on $YOURPOOLNAME will do the job.

 

[ThinkAgain]

That would also be an option. However, the documentation of ZFS suggests that „zpool trim“ is better performance-wise:

Be aware that automatic trimming of recently freed data blocks can put significant stress on the underlying storage devices. This will vary depending of how well the specific device handles these commands. For lower end devices it is often possible to achieve most of the benefits of automatic trimming by running an on-demand (manual) TRIM periodically using the zpool trim command.

 

[fireon]

I've checked that here, but both, autotrim and trim are option that not available in zfs for linux. Did not find it here on proxmox.

 

[mir]

I've checked that here, but both, autotrim and trim are option that not available in zfs for linux. Did not find it here on proxmox.

It is this feature: https://www.illumos.org/issues/1701 which can be found in this commit: https://github.com/openzfs/zfs/commit/1b939560be5c51deecf875af9dada9d094633bf7

But as the documentation states this feature will only be available for SSD's so if the pool does not contain SSD's this feature has no effect since pools without SSD's will be skipped - SSD's for log or cache does not count.

 

[fireon]

Yes you are right! On my ssd pools this option is available.

 

[ThinkAgain]

Ok, thanks all. I take from this discussion that there is nothing yet built into pve to execute the trim, and that it's thus ok and maybe even required to, e.g., setup a cron job which regularly executes zpool trim for the SSD pools.

 

[Thalix]

I found a corresponding cronjob in my newly installed PVE 6.1. I assumed that this one does the trim?

/etc/cron.d # cat zfsutils-linux PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin # Scrub the second Sunday of every month 24 0 8-14 * * root [ $(date +\%w) -eq 0 ] && [ -x /usr/lib/zfs-linux/scrub ] && /usr/lib/zfs-linux/scrub

 

[ThinkAgain]

If I read this correctly, it's about scrubbing the pools, i.e. testing data integrity, while the trim is about discarding space on SSDs...

 

[fireon]

If I read this correctly, it's about scrubbing the pools, i.e. testing data integrity, while the trim is about discarding space on SSDs...

Yes.

 

[Thalix]

If I read this correctly, it's about scrubbing the pools, i.e. testing data integrity, while the trim is about discarding space on SSDs...

Tank you.

 

[ozgurerdogan]

One question here, I do regual disk defreag on windows and fstrim -av for linux. What is main difference when trimming inside wm and at pve pool trim ?

 

[ozgurerdogan]

Also default value for autotrim is off. Should I turn it on for all pools?

 

[ThinkAgain]

The above discussion suggests that the autotrim feature might not work, and that, in any case, a manually scheduled trim should be better performance-wise.

defrag has nothing to do with trim. Trim just discards unused data blocks on SSDs, defrag tries to reduce fragmentation (mainly helpful on HDDs).

And if you select "discard" as option for your virtual disks in Proxmox, the VM will discard internally and pass that on to the underlying disk, assuming your entire storage stack supports this. This won't take care of any data/trim needs triggered outside of your virtual disks, though.

So what's best really depends on what you're doing on your disks.

 

[ozgurerdogan]

Thank you . This reminded me that non-ssd disks should not be defragmented. All my disks are ssd and discard enabled.

But I could not understand this: "This won't take care of any data/trim needs triggered outside of your virtual disks, though."

You mean I should trim disk inside vm (fstrim -av) and also at node level with a cron "zpool trim" for pools?

 

[ThinkAgain]

If you have a pool of SSDs (or even just a single one) that should be trimmed, and if all data you put onto it is from virtual disks, you probably don't need to do anything else. But if you have a pool that contains much more than just your virtual disks (e.g. also you media collection, mails for a mail server, ...), my thinking is that a separate trim of that pool would be prudent.