ZFS performance tips!

[ant12]

Hi,

I have 2 proxmox hosts each with local zfs with draid2 with 8 sas disks, 2 sas ssd nvme for zfs cache and other 2 sas ssd nvme for zfs log. The utilization of zfs cache is very low only 458G out of 1.5TB. What can I do ?

Thanks!

 

[leesteken]

I have 2 proxmox hosts each with local zfs with draid2 with 8 sas disks, 2 sas ssd nvme for zfs cache and other 2 sas ssd nvme for zfs log. The utilization of zfs cache is very low only 458G out of 1.5TB. What can I do ?

Regarding Proxmox and VMs: use stripes or mirrors instead of (d)RAIDz(1/2/3) if you want better IOPS. This is not Proxmox specific , so do more research about ZFS and learn that L2ARC and SLOG don't usually help for IOPS. Maybe a special device would help?

 

[IsThisThingOn]

L2ARC is an extension of ARC.
So when you say L2ARC usage is low, that could be a positive thing because data did not evict ARC in the first place.

Why are you not happy with the low usage? Are you unhappy with performance?

 

[Kingneutron]

> 2 sas ssd nvme for zfs cache and other 2 sas ssd nvme for zfs log. The utilization of zfs cache is very low only 458G out of 1.5TB

You're allocating wayheyhey the hell too much L2ARC. Detach those and convert them to a mirrored Special device.

https://forum.level1techs.com/t/zfs-metadata-special-device-z/159954

Unless you're doing NFS / lots of sync writes, you probably don't even need a slog, either

 

[IsThisThingOn]

Agree with @Kingneutron, you probably have some misconceptions about L2ARC and how that works. Mirrors and L2ARC don’t make a lot of sense.

https://github.com/jameskimmel/opinions_about_tech_stuff/blob/main/ZFS/ZFS tuning flowchart.md