Dedicated log ssd + sync=always?

[JohnD]

Hello everybody,

i am currently using the following pool setup :

  pool: hddtank
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
    continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Tue Sep 19 09:31:04 2017
    522G scanned out of 810G at 280M/s, 0h17m to go
    0 resilvered, 64.38% done
config:

    NAME                      STATE     READ WRITE CKSUM
    hddtank                   DEGRADED     0     0     0
      mirror-0                ONLINE       0     0     0
        sda                   ONLINE       0     0     0
        sdb                   ONLINE       0     0     0
    logs
      mirror-1                DEGRADED     0     0     0
        nvme0n1p3             ONLINE       0     0     0
        replacing-1           UNAVAIL      0     0     0
          880080219137481614  UNAVAIL      0     0     0  was /dev/nvme1n1p3/old
          nvme1n1p3           ONLINE       0     0     0
    cache
      nvme0n1p5               ONLINE       0     0     0
      nvme1n1p5               UNAVAIL      0     0     0

Just ignore the resilver stuff (i had to replace one nvme drive).

So it is basically a pool with two 4 TB HDDs (mirror) and two NVMe SSDs (mirror) for SLOG and L2Arc.

Unfortunatly the performance under load is pretty bad.

My question:
I always read that "sync=always" should be slower than "sync=standard". On the other hand it sounds like a good idea to force ZFS to cache all write requests on the SSDs.
Could you explain that setting for me?

Thanks in advance,

John

 

[fabian]

with sync=standard, async writes are cached in memory (application can continue at this point), and then written to the disks asynchronously, sync writes are written to the (S)LOG AND memory (application can continue at this point), then to the disk asynchronously. with sync=always, all writes are handled as if they were issued as sync writes. since memory is faster then anything backing an (S)LOG, this will decrease performance. it will improve consistency though, as writing applications will now wait longer for their writes to be actually persisted even if they haven't requested to wait .