ZFS storage confusion

[jcbhltz22]

I have a 16 disk (8TB SAS HDD) RAIDZ2 pool with 2x8 disk udevs and performance seems awful. Writing a 15GB movie file maxes out at 120MB/s, which is the speed of a single disk in the pool.

I know there are many possible reasons, but im wondering if I can at least narrow it down a little, this is my current config.

- compression off
- recordsize is 1M (all storage is large movie files, 5-20GB each)
- block size 16k (left default)

Guest is using an ext4 raw volume. running the same copy outside the guest has the same results so doubtful this would be the issue.

 

[Nemesiz]

I use 12 disks in 2x Raidz2 of 6 combination.

Doing copy 40GB file from ramfs to ZFS `zpool iostat` report as 600-1300 MiB/s. Using pv tool report as 622 MiB/s average. Using dd with bs=200M : 828 MB/s

I suggest you to troubleshoot with tools like atop or others who could show disk activity and other stats.

If not the disks are bottleneck then look at ZFS settings related to RAM.

 

[jcbhltz22]

I did manage to get somewhere; on the PVE host I can get 1000-2000, in my guest I get 80-120.