Calculation L2ARC & ZIL for my setup

[MaLe]

Hi there,

this is my setup for the PBS backup server:

- 4-Core Intel CPU (not the fastet one)
- 20GB of RAM
- 2x 250GB NVMe disks for OS + L2ARC/ZIL
- 4x 8TB spinning disks (RAID10)
- 32GB RAID1 partition for the OS on NVMe disks

So there is aprox. 220GB free space on each NVMe. To increase the speed of the spinnimg disks, I would like to calculate the optimal size for the L2ARC & ZIL partitions.

These are my toughts:

- 8GB for ZIL (Half of the RAM) as mirror set (ZIL seems to be critical for operation)
- the rest of the remaining space for L2ARC as stripe set (afaik there is no mirror possible)

Does anybody has better recommendation for my setup? Maybe also special setup for the zpools to relieve the CPU?

Thanks & Best Regards
Martin

 

[Dunuin]

I wouldn't use SLOG or L2ARC for that. Use special devices instead.
I would install PBS using a ZFS mirror and only give that PBS a 32GB system partition. Then create a new partition using the remaining 217GB of unallocated space on both NVMe. Then add a "special mirror" to your HDD pool using these two 217GB partitions. So something like this: zpool add YourHDDpool special mirror /dev/disk/by-id/nvme-YourFirstSSD-part4 /dev/disk/by-id/nvme-YourSecondSSD-part4

But keep in mind that special devices are no cache. If you loose that mirror on the SSDs all data on the HDDs is lost too. And this isn't retroactive. Only new data written to the pool will benefit from additional performance. Special devices should give you sync+async read+write boost as with that all metadata will be stored on the SSDs and the HDDs only have to store the data. Without a special device mirror your HDDs would have to store metadata+data and are hit by way more IO so you sooner run into the IOPS performance bottleneck of HDDs.
A SLOG would only help you with sync writes. And a L2ARC only with reads.

 

[MaLe]

My knowledge about ZFS is expandable Many thanks for your reply, that's very usefull to me. What you suggested is exactly what I want to achieve - a better overall read&write performance for the HDD pool.

What I've also learned after some research is, that these SSD used for special devices should have a power-loss protection. Otherwise in case of power failure the data stored in a RAM cache of the SSD will be deleted, so data on the SSD can become courrupt and the associated pool could be lost as well.

Is there a recommendation regarding the size of the special device in relation to the pool size?

 

[Dunuin]

What I've also learned after some research is, that these SSD used for special devices should have a power-loss protection. Otherwise in case of power failure the data stored in a RAM cache of the SSD will be deleted, so data on the SSD can become courrupt and the associated pool could be lost as well.

That's why you always should use a UPS and enterprise SSDs for ZFS instead of consumer SSDs. Not just for special devices.

Is there a recommendation regarding the size of the special device in relation to the pool size?

It should be around 0,4% of the HDDs capacity. See here how to calculate it: https://forum.level1techs.com/t/zfs-metadata-special-device-z/159954