ZFS looking for guidance

[showiproute]

Hello everyone,
I am looking for advice about best practice regarding flash storage for ZFS pool on HDDs.

I currently have three "spinning rust" pools at my PVE server which I would like to speed up.
- RAID 1 (2x SAS 4TB)
- SAS 12TB
- RAIDz1 (3x SATA 18TB) for Proxmox Backup

For speeding up I have ordered
2x Transcend TS1TMTE220S (1TB each, 1.2 DWPD)
1x Optane 900p (280 GB, used)


I thought that I will create partitions on the Transcend and will use them as a mirrored special device for the ZFS pools.
Intel Optane would be used as SLOG device.


Is that a good approach?

 

[LnxBil]

Transcend TS1TMTE220S (1TB each, 1.2 DWPD)

Is this consumer grade? I havent found anything stating otherwise. This may not be the solution to your speed problems, yet you have to try.

Optane 900p (280 GB, used)

Most of the time, way too big for a SLOG device. It'll only store 5 seconds of sync writes.

 

[showiproute]

Is this consumer grade? I havent found anything stating otherwise. This may not be the solution to your speed problems, yet you have to try.

According to their product page I think it's designed for Enterprise workload --> 1.2 DWPD for 5 years; 2200 TBW in total
On the other hand the SSD does not feature a power-loss protection.

Most of the time, way too big for a SLOG device. It'll only store 5 seconds of sync writes.

What else would you do with that?

 

[LnxBil]

On the other hand the SSD does not feature a power-loss protection.

I also read the page and all the important information is not on it. Without power-loss protection, it is not really enterprise use.

What else would you do with that?

You could use it as swap and /tmp (which is normally in memory). I would not store anything important on it, you have not mirrored it. Optane is perfect for SLOG, bug 16 or 32 GB is normally sufficient for non-database workloads or home labs.

 

[showiproute]

I also read the page and all the important information is not on it. Without power-loss protection, it is not really enterprise use.

I thought I will use it mirrored as a special device for my ZFS pools - in general my server is backed by a UPS (can also fail, I know).

Would Optane be also useful as a L2ARC?

 

[Dunuin]

I thought I will use it mirrored as a special device for my ZFS pools - in general my server is backed by a UPS (can also fail, I know).

UPS won't help. Without powercaps on the SSD (and I can't see them on the PCB) the SSD can't cache sync writes in its volatile DRAM, so when doing them it will drop to nearly HDD performance and write amplification (and therefore SSD wear) will be terrible as the writes can't be collected and optimized before writing them to the NAND. Only way to get good sync write performance is to get a HW Raid with BBU+cache or real Enterprise grade SSD with a power-loss protection.

 

[showiproute]

UPS won't help. Without powercaps on the SSD (and I can't see them on the PCB) the SSD can't cache sync writes in its volatile DRAM, so when doing them it will drop down to nearly HDD performance and write amplification (and therefore SSD wear) will be terrible as the writes can't be collected and optimized before writing them to the NAND. Only way to get good sync write performance is to get a HW Raid with BBU+cache or real Enterprise grade SSD with a power-loss protection.

I will use Optane for ZIL/Log and the mirrored SSDs for metadata as a special device

 

[Dunuin]

I will use Optane for ZIL/Log and the mirrored SSDs for metadata as a special device

Ok, good point. Then the missing PLP might not be that bad.

 

[LnxBil]

Would Optane be also useful as a L2ARC?

L2ARC does not perform that well as any other cache layer like bcache or flashcache, so you have to try. I tried it once and it wasn't that good. Put more RAM in the machine, it'll help MUCH, MUCH more.