Proxmox with ZFS + SSDs: Built-in TRIM cron job vs zfs autotrim?

good point, thanks for digging, then i disable periodic trim. Even if its working fine for over 3-4 months now, there seems to be a risk.
No, that was not my point. When you schedule the trim, you will see a greater slowdown on this drives then you might expect. Depending on the current load this might impact your whole system. I had this issue where the trim would take hours and run along ci pipelines, parallel to backups of some VMs, while the opensearch DB became overwhelmed. The whole system had writes down to <30mb on a 4 drive SATA SSD ZRAID2 and iowait was up in the seconds.

I still have periodic trim on, but I choose my time wisely and do not trust the consumer drives to be 'good'.
 
Last edited:
No, that was not my point. When you schedule the trim, you will see a greater slowdown on this drives then you might expect. Depending on the current load this might impact your whole system. I had this issue where the trim would take hours and run along ci pipelines, parallel to backups of some VMs, while the opensearch DB became overwhelmed. The whole system had writes down to <30mb on a 4 drive SATA SSD ZRAID2 and iowait was up in the seconds.

I still have periodic trim on, but I choose my time wisely and do not trust the consumer driver to be 'good'.
I understand, if it helps, here on 2tb 870 evo ssds, trim takes around 5minutes.
 
PVE ver 8.3.5 and haven't yet figured out how trim / discard options work in conjunction with each other.

I am aware of Autotrim being a synchronous trim that is issued after a block has been deleted on your pool.
Since it might have an impact on the performance of the Vms, let's leave this option aside.

The options org.debian: periodic-trim and auto-trim for both rpool and the zfs raid 10 I have created ,
are: not configured (org.debian: periodic-trim) and off (auto-trim). One of the posts in the first page mentions
<<When org.debian: periodic-trim is not present in pool, or the property is present but value is empty/invalid, they are treated as auto>>
Since my pool consists of mixed (enterprise though) sata ssd drives I fell into the forth explanation of the same post mentioning
<<When the >=3.1 support is properly implemented, pool with a mixed types of SSDs will be measured by whether all disks are of the recommended types. Users can enable the pool trim by setting the property to enable after checking all disks in pool carefully.>>
My drives are x2 micron 5300 max (mixed use) and x2 Intel-S4610 (mixed use) in a zfs raid 10 array.

Also if this is true <<the script checks the transport layer and triggers only on "nvme", but the transport layer of almost all SATA/SAS Based SSD's is sas. So in the end it wont get triggered ever>> then even if I set org.debian: periodic-trim as enabled, it still won't trigger because my SSDs are sata?

On the other hand all VMs during their creation are with discard option as ticked.
Is there an order the OS checks for priority of those options which comes first?
Does discard option as ticked make sense with org.debian: periodic-trim as not configured (so used as auto per explanation on the post of the first page) and autotrim off?
 
Last edited:
I suspect debian developers have overreacted to some edge cases, if there was en mass buggy behaviour with consumer SATA SSD's, Then Microsoft the owner of the biggest OS on the planet would be disabling Trim, instead its enabled per transaction and by scheduled trim by default.

Thank you to @Ramalama for the insight.

However looks like things have changed, and now at least some SATA are trimmed on auto mode.

Code:
  - NVMe SSDs are always considered safe (TRIM is inherently queued).
  - SATA SSDs are safe if the kernel reports queued TRIM support via the
    sysfs ata_device/trim attribute (requires SATA >= 3.1 and no kernel
    quirk disabling NCQ TRIM). Older SATA SSDs with non-queued (blocking)
    TRIM are skipped to avoid I/O stalls (See #983086).
  - Regular HDDs and SMR HDDs with TRIM are skipped.
  - SATA devices behind SAS HBAs are skipped because the HBA firmware
    handles the SCSI UNMAP to ATA TRIM translation internally, and
    the host cannot determine whether it issues queued (NCQ) or
    non-queued TRIM to the underlying drive.
  - SATA SSDs behind port multipliers (PMP) are supported; the
    individual device's trim capability is identified via the SCSI
    address to ATA device mapping.
  - Non-rotational devices without a recognizable ATA trim attribute
    (e.g. virtio, iSCSI, NBD, loop, PMEM) are skipped because the
    safety of TRIM cannot be verified.
  - Log, special, and dedup vdevs are classified like data vdevs.
    Cache and spare vdevs are excluded (not targets of zpool trim).

https://salsa.debian.org/zfsonlinux...fc72a4adb19d96be1c9d84ab2cdbb835?pow_referer=

command to check '
# cat /sys/class/ata_device/<device>/trim'
My local system boot ssd's are force untrimmed, I guess not surprising, as every SATA device in there is samsung. (forced unqueued, is blacklisted)
 
Last edited:
Thanks, @chrcoluk ! That's a great summary of the current state of things. I just learned some new commands, too.

I'm using an Intel DC S3520 (or rather, according to smartctl, a:
Code:
Model Family:     Dell Certified Intel S3520 Series SSDs
Device Model:     SSDSC2BB120G7R

TRIM Command:     Available, deterministic, zeroed
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
SATA 3.1 does include queued TRIM, but it's not included on every drive, so here's how I checked it using the above commands.

Looking up the kernel-reported TRIM support is a bit confusing if you haven't done it before, as it's based on the disk's PCIe path. At least, I was confused.
  • Start by getting the logical device name (e.g., /dev/sda in my case) from, e.g., lsblk.
  • Find the disk's PCIe path: ls /dev/disk/by-path -al | grep -i "sda"
  • That will get you a scary device list.
Bash:
root@pisces1:/sys/class/ata_device# ls /dev/disk/by-path -al | grep -i "sda"
lrwxrwxrwx 1 root root   9 Sep 21 13:57 pci-0000:00:17.0-ata-3 -> ../../sda
lrwxrwxrwx 1 root root   9 Sep 21 13:57 pci-0000:00:17.0-ata-3.0 -> ../../sda
lrwxrwxrwx 1 root root  10 Sep 21 13:57 pci-0000:00:17.0-ata-3.0-part1 -> ../../sda1
lrwxrwxrwx 1 root root  10 Sep 21 13:57 pci-0000:00:17.0-ata-3.0-part2 -> ../../sda2
lrwxrwxrwx 1 root root  10 Sep 21 13:57 pci-0000:00:17.0-ata-3.0-part3 -> ../../sda3
lrwxrwxrwx 1 root root  10 Sep 21 13:57 pci-0000:00:17.0-ata-3-part1 -> ../../sda1
lrwxrwxrwx 1 root root  10 Sep 21 13:57 pci-0000:00:17.0-ata-3-part2 -> ../../sda2
lrwxrwxrwx 1 root root  10 Sep 21 13:57 pci-0000:00:17.0-ata-3-part3 -> ../../sda3
  • Be glad you don't have to know why it's like that. I don't know why it's like that.
  • Print the kernel's queued TRIM support: # cat /sys/class/ata_device/dev3.0/trim
Mine outputs unqueued , which isn't the answer I wanted, but, alas.

That's not quite the end of the story.
PVE/ZFS uses cron to perform scheduled, manual TRIMs, as we've discussed.

Bash:
/etc/cron.d# cat zfsutils-linux
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# TRIM the first Sunday of every month.
24 0 1-7 * * root if [ $(date +\%w) -eq 0 ] && [ -x /usr/lib/zfs-linux/trim ]; then /usr/lib/zfs-linux/trim; fi

# Scrub the second Sunday of every month.
24 0 8-14 * * root if [ $(date +\%w) -eq 0 ] && [ -x /usr/lib/zfs-linux/scrub ]; then /usr/lib/zfs-linux/scrub; fi

If you'd like to see when that's happening, to which drives, try:
Bash:
root@pisces1:/etc/cron.d# zpool status -t
  pool: pisces1VirtStore
 state: ONLINE
  scan: scrub repaired 0B in 00:00:06 with 0 errors on Sun Sep 13 00:24:07 2026
config:

    NAME                                                 STATE     READ WRITE CKSUM
    pisces1VirtStore                                     ONLINE       0     0     0
      nvme-Samsung_SSD_970_EVO_Plus_1TB_S6S1NS0T805122H  ONLINE       0     0     0  (100% trimmed, completed at Sun Sep  6 00:24:09 2026)

root@pisces1:/etc/cron.d# zpool status -t rpool
  pool: rpool
 state: ONLINE
  scan: scrub repaired 0B in 00:00:31 with 0 errors on Sun Sep 13 00:24:32 2026
config:

    NAME                                           STATE     READ WRITE CKSUM
    rpool                                          ONLINE       0     0     0
      ata-SSDSC2BB120G7R_PHDV809404DJ150MGN-part3  ONLINE       0     0     0  (untrimmed)

Well, the NVME that I keep my guests on is TRIM'ing on a schedule,, at least.

But, my SATA SSD isn't getting trimmed at all, presumably because it doesn't supported queued TRIM operations (?).
I was hoping it would still manually TRIM.

However, it has no issues with a manual TRIM, which it completes in about 5-10 seconds. You can watch it go in real time with zfs trim $pool && watch zpool status -t $pool

Diff:
Every 2.0s: zpool status -t                                                                                                                                                 pisces1: Wed Sep 23 20:05:18 2026

  pool: pisces1VirtStore
 state: ONLINE
  scan: scrub repaired 0B in 00:00:06 with 0 errors on Sun Sep 13 00:24:07 2026
config:

        NAME                                                 STATE     READ WRITE CKSUM
        pisces1VirtStore                                     ONLINE       0     0     0
          nvme-Samsung_SSD_970_EVO_Plus_1TB_S6S1NS0T805122H  ONLINE       0     0     0  (100% trimmed, completed at Sun Sep  6 00:24:09 2026)

errors: No known data errors

  pool: rpool
 state: ONLINE
  scan: scrub repaired 0B in 00:00:31 with 0 errors on Sun Sep 13 00:24:32 2026
config:

        NAME                                           STATE     READ WRITE CKSUM
        rpool                                          ONLINE       0     0     0
          ata-SSDSC2BB120G7R_PHDV809404DJ150MGN-part3  ONLINE       0     0     0  (100% trimmed, completed at Wed Sep 23 20:04:10 2026)

errors: No known data errors

Does anyone know why the ZFS cron job doesn't pick up this SSD? I assumed the ZFS cron'd TRIM would still catch it even if it doesn't support queued TRIM.

I might end up enabling zfs autotrim on this pool. It's my boot device and guests don't live on it. I don't like the idea of never TRIMing the drive at all, and its I/O is pretty low as it is. What do y'all think?
 
  • Like
Reactions: chrcoluk
You are right it isnt live yet, this is the future behaviour once the appropriate package comes in the future.

If you mean the check to see if queued trim is supported or not, its not overcomplicating it, the lsblk command isnt satisfactory to see the trim status in relation to default trim behaviour, it doesnt report queueing or if blacklisted. In addition what I posted in how it works, is taken directly from the document of the patch.

The command I listed was taken directly from the committed code on how the trim code determines whether a disk gets trim in 'auto' or not.

Pretty much any SSD thats not from the stone age will report it supports trim, any command to test that isnt that useful, whats useful is to determine whether Debian (aka Proxmox) decides if it is trim worthy. Then people can make informed decisions if they need to override the behaviour themselves as @SInisterPisces has done.

I just trimmed my rpool manually, likely years worth of trim catch up, it took 18 seconds. Interestingly though, its no faster trimming again right after.
 
Last edited:
  • Like
Reactions: Impact