ZFS performance tips!

[uberarchiver]

options zfs zfs_dirty_data_max_max=[default 4G?]

, could possibly help if the write during a txg period (default 5 secs) if accumualted data is more that default.

Needs to be set at module load.

options zfs zfs_dirty_data_max=[can cap the max_max dynamically]

Hint: Set data_max_max high and cap with data_max dynamically.

 

[ludwinator]

lots of tips, but nobody seems to ask obvious questions like:

- what kind of VMs are you running?
- how are the VMs stored? plain files or zvols?
- how much load do you have on your disks?
- is there a disk bound bottleneck? read or write?
- compression, dedup, encryption?
- regarding ARC/L2ARC: did you watch the actual hit rate to these caches?

 

[uberarchiver]

lots of tips, but nobody seems to ask obvious questions like:

- what kind of VMs are you running?
- how are the VMs stored? plain files or zvols?
- how much load do you have on your disks?
- is there a disk bound bottleneck? read or write?
- compression, dedup, encryption?
- regarding ARC/L2ARC: did you watch the actual hit rate to these caches?

You did notice his attachment in this post ?

 

[ludwinator]

sorry no, thanks for point me to it.
but that answers only one question: L2ARC does not help at all at his workload.

 

[uberarchiver]

Well, actually the 16G of ARC which is allocated is not helping him very much. The hitrate is good-ish.

My guess is that 32 or 64G would be much more helpful. And would make the L2ARC do better, too.

But you are entirely right that knowing more about the workload would be very helpful.

 

[waltar]

- regarding ARC/L2ARC: did you watch the actual hit rate to these caches?

I'm asking myself what this arc hit rates would be worth in reality ...
having 64 clamav virus scanjobs running on a 64core host with 24x 16TB in 4 vdevs raidz2 and used cpu in top is about 30% while arc_summery say
Cache Hit Ratio: 75.18% 278.30G
Cache Miss Ratio: 24.82% 91.87G
Actual Hit Ratio: 74.24% 274.82G
I have the feeling there is more a 75% hit miss and just 25% hit as seeing cpu usage in top to clamav's but that for anybody could think of it's own.

 

[ant12]

There any option to use NMVE as the first layer of data ? I'm using only for block (zvol) not for files (zfs)!

 

[ludwinator]

Forget L2ARC and create mirror pool on you freed nvme disks. Then put the zvols there and test it.

 

[ant12]

And the 8 SAS disks ?

 

[IsThisThingOn]

And the 8 SAS disks ?

Mirror. Still poor performance compared to any SSD, but better than RAIDZ