disk IO slow down proxmox system

[Maher Khalil]

Hello
I have dell R640 server with the below mentioned disks
2 x 1.6TB SSD =====> ZFS RAID 0 each disk in separate pool
4 x 1.6TB SSD =====> ZFS RAID z
1 x 1.6TB NVMe =====> ZFS RAID 0
all disks are enterprise
RAM 1024 ECC
My problem sever IO sometimes very high
I checked with Grafana and found that this happen when some machines do high read and right io, so I decreased the io as mentioned on the attached photo but the problem is not resolved
I have confusion in setting Read limit (ops/s) and write limit (ops/s)
can someone guide me in this settings to reasonable settings?

 

[jdancer]

This is what I use to optimize IOPS:

Set write cache enable (WCE) to 1 on SAS drives
Set VM cache to none
Set VM to use VirtIO-single SCSI controller and enable IO thread and discard option
Set VM CPU type to 'host'
Set VM VirtIO Multiqueue to number of cores/vCPUs

If using Linux:
Set Linux VMs IO scheduler to none/noop

If using Ceph:
Set RBD pool to use the 'krbd' option

I don't run Windows VMs.

 

[Maher Khalil]

what about the amount of io as parameter shown on the attached photo? how can I optimize them?