Thanks! The checksum algorithm doesn't seem to matter, but on my slower Xeon, when you doHi,
As I understand, SIMD is used for fletcher4 checksums(maybe I am wrong).
cat /proc/spl/kstat/zfs/fletcher_4_bench | grep fastest
fastest avx512f avx2
You can change the checksum algo to sha256(create a test dataset and see if with sha256 fio will get better results).
cat /proc/spl/kstat/zfs/vdev_raidz_bench and look at the "scalar" (non-SIMD) row:
gen_p (RAID-Z) is 1.13GB/sec, and gen_pq (RAID-Z2) is 290MB/sec, and gen_pqr (RAID-Z3) is only 132MB/sec. SIMD makes everything 5-7x faster.