Proxmox performance with install on SSD ZFS RAID1 versus HDD RAID1

[Mike.S]

I'm attempting to turbo charge my old Dell Poweredge R515 servers (PCI 2.0) with dual NVME PCI cards and M.2 sticks. I'm experimenting with the QNAP QM2-2P-244A and the Ableconn PEXM2-130 (Startech) cards. I'm attempting to use 2 Kingston 480GB DC1000b for OS and 2 Samsung 960GB 983 DCT for WAL/DB.

My current configuration has the OS on the Perc H700/SAS 300GB 10K drives in RAID1. I installed Proxmox on the 2 Kingston DC1000b using ZFS RAID1.

Surprisingly the hard disks are faster on some IO operations. Also performance is twice as fast to ZFS RAID1 with sync disabled than it is to just one device (no ZFS. no RAID1). I'm trying to figure out why.

​

 

[aaron]

According to the data sheet, the Kingston DC1000B 480G has only 20k IOPS for writes and ~200k for reads. That is not a great value for write IOPS.

What are you comparing in your benchmarks? As all are done with a block size of 4k I assume it is IOPS and not bandwidth.

A benchmark with sync disabled only tells you how fast the cache is as write ACKs will be returned before the data has been written to disk.

You could try to let the tests run for 600 seconds (10min) to reduce the impact of any involved caches.