Does zfs mirror read performance scale by vdevs num?

gladtent7039

New Member
Apr 26, 2024
2
0
1
I googled and was told that N-way mirror should improve read performance, but test result indicate otherwise.
Is my testing method incorrect, or is there a misunderstanding?

https://openzfs.org/wiki/Features#Improve_N-way_mirror_read_performance

test result:

PVE 8.2.2
2 nvme ssd zfs mirror

Code:
pool: data
 state: ONLINE
config:


        NAME                                               STATE     READ WRITE CKSUM
        data                                               ONLINE       0     0     0
          mirror-0                                         ONLINE       0     0     0
            nvme-SanDisk_Ultra_3D_NVMe_2TB_23452H800317    ONLINE       0     0     0
            nvme-SanDisk_Ultra_3D_NVMe_2TB_23452H800837_1  ONLINE       0     0     0

fio benchmark
Code:
[global]
name=nvme-seq-read
time_based
ramp_time=5
runtime=90
readwrite=read
bs=256k
ioengine=libaio
direct=1
numjobs=1
iodepth=32
group_reporting=1
[nvme0]
filename=/data/benchmark.dat
size=256G

fio test got about 3300m/s

iostat:
Code:
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.17    0.00   13.93    0.00    0.00   85.91
Device             tps    kB_read/s    kB_wrtn/s    kB_dscd/s    kB_read    kB_wrtn    kB_dscd
nvme0n1       13095.00   1676160.00         0.00         0.00    1676160          0          0
nvme1n1       13131.00   1680768.00         0.00         0.00    1680768          0          0

If I tear down zfs pool and run same fio test on single ssd driver, also get 3500m/s
Code:
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00    1.09    0.00    0.00   98.91
Device             tps    kB_read/s    kB_wrtn/s    kB_dscd/s    kB_read    kB_wrtn    kB_dscd
nvme0n1           0.00         0.00         0.00         0.00          0          0          0
nvme1n1       27622.00   3535616.00         0.00         0.00    3535616          0          0

So, my expectation is that in a ZFS mirror setup, I should achieve speeds of around 6000~7000 MB/s.
 
Last edited: