Proxmox with ZFS + SSDs: Built-in TRIM cron job vs zfs autotrim?

[lexxxel]

good point, thanks for digging, then i disable periodic trim. Even if its working fine for over 3-4 months now, there seems to be a risk.

No, that was not my point. When you schedule the trim, you will see a greater slowdown on this drives then you might expect. Depending on the current load this might impact your whole system. I had this issue where the trim would take hours and run along ci pipelines, parallel to backups of some VMs, while the opensearch DB became overwhelmed. The whole system had writes down to <30mb on a 4 drive SATA SSD ZRAID2 and iowait was up in the seconds.

I still have periodic trim on, but I choose my time wisely and do not trust the consumer drives to be 'good'.

 

[Ramalama]

No, that was not my point. When you schedule the trim, you will see a greater slowdown on this drives then you might expect. Depending on the current load this might impact your whole system. I had this issue where the trim would take hours and run along ci pipelines, parallel to backups of some VMs, while the opensearch DB became overwhelmed. The whole system had writes down to <30mb on a 4 drive SATA SSD ZRAID2 and iowait was up in the seconds.

I still have periodic trim on, but I choose my time wisely and do not trust the consumer driver to be 'good'.

I understand, if it helps, here on 2tb 870 evo ssds, trim takes around 5minutes.

 

[ieronymous]

PVE ver 8.3.5 and haven't yet figured out how trim / discard options work in conjunction with each other.

I am aware of Autotrim being a synchronous trim that is issued after a block has been deleted on your pool.
Since it might have an impact on the performance of the Vms, let's leave this option aside.

The options org.debian: periodic-trim and auto-trim for both rpool and the zfs raid 10 I have created ,
are: not configured (org.debian: periodic-trim) and off (auto-trim). One of the posts in the first page mentions
<<When org.debian: periodic-trim is not present in pool, or the property is present but value is empty/invalid, they are treated as auto>>
Since my pool consists of mixed (enterprise though) sata ssd drives I fell into the forth explanation of the same post mentioning
<<When the >=3.1 support is properly implemented, pool with a mixed types of SSDs will be measured by whether all disks are of the recommended types. Users can enable the pool trim by setting the property to enable after checking all disks in pool carefully.>>
My drives are x2 micron 5300 max (mixed use) and x2 Intel-S4610 (mixed use) in a zfs raid 10 array.

Also if this is true <<the script checks the transport layer and triggers only on "nvme", but the transport layer of almost all SATA/SAS Based SSD's is sas. So in the end it wont get triggered ever>> then even if I set org.debian: periodic-trim as enabled, it still won't trigger because my SSDs are sata?

On the other hand all VMs during their creation are with discard option as ticked.
Is there an order the OS checks for priority of those options which comes first?
Does discard option as ticked make sense with org.debian: periodic-trim as not configured (so used as auto per explanation on the post of the first page) and autotrim off?

 

[chrcoluk]

I suspect debian developers have overreacted to some edge cases, if there was en mass buggy behaviour with consumer SATA SSD's, Then Microsoft the owner of the biggest OS on the planet would be disabling Trim, instead its enabled per transaction and by scheduled trim by default.

Thank you to @Ramalama for the insight.

However looks like things have changed, and now at least some SATA are trimmed on auto mode.

  - NVMe SSDs are always considered safe (TRIM is inherently queued).
  - SATA SSDs are safe if the kernel reports queued TRIM support via the
    sysfs ata_device/trim attribute (requires SATA >= 3.1 and no kernel
    quirk disabling NCQ TRIM). Older SATA SSDs with non-queued (blocking)
    TRIM are skipped to avoid I/O stalls (See #983086).
  - Regular HDDs and SMR HDDs with TRIM are skipped.
  - SATA devices behind SAS HBAs are skipped because the HBA firmware
    handles the SCSI UNMAP to ATA TRIM translation internally, and
    the host cannot determine whether it issues queued (NCQ) or
    non-queued TRIM to the underlying drive.
  - SATA SSDs behind port multipliers (PMP) are supported; the
    individual device's trim capability is identified via the SCSI
    address to ATA device mapping.
  - Non-rotational devices without a recognizable ATA trim attribute
    (e.g. virtio, iSCSI, NBD, loop, PMEM) are skipped because the
    safety of TRIM cannot be verified.
  - Log, special, and dedup vdevs are classified like data vdevs.
    Cache and spare vdevs are excluded (not targets of zpool trim).

https://salsa.debian.org/zfsonlinux...fc72a4adb19d96be1c9d84ab2cdbb835?pow_referer=

command to check '
# cat /sys/class/ata_device/<device>/trim'
My local system boot ssd's are force untrimmed, I guess not surprising, as every SATA device in there is samsung. (forced unqueued, is blacklisted)