(Too?) Big IO Delay peaks

[guletz]

After revising many many zfs parameters I've decided to set sync=disabled in the pool (we have UPS and controller with battery) and the problem is gone...

Thx.

Hi,

WRONG the problem is not go away, you only avoid the problem. sync = disable even with the state of the art UPS, will not help at all in a case of kernel crash. In a such case you can lose at most 5 sec of data. For any VM with DB this can be very dangerous! If you keep sync = disable, maybe you will consider to lower this risk by changing the zfs cache flush time from default(5 sec) to let say 3 sec(this will also impact on performance).

Good luck /Bafta

 

[guletz]

.... and more for your problem !!!

As @budy clear say for you, you must "convert" your HW Raid / zfs to JBOD(aka HBA) /zfs and for one more reason:
- you use PMX 5.x, and for zfs point of view you have only one HW RAID disk, but in reality you have SSD => your ashift = 9 or 12(if you are luky), insted of ashift = 13(recomandable value for any SSD pool)
- and also NO trim support exist in zfs => if also your pool have a several years => write performance will decline(as you see with your old sever tests) => a very compliacated series of tasks for make a simple trim on your SSDs
- in a near future(this century I hope) you will need to upgrade to PMX 6.x where the NEW zfs can deal with trim/fstrim, but not with HW raid

So, it is up to you ... if you want to solve HW-RAID problem, or you want to "cut the tail of the cat"(a Romanian old sentence) . Yes you could cut the "tail" using 3 different task or you can cut in one task(more or less)