How do I get the best random IO performance for VM workloads with this hardware

[jimbob]

I have the following hardware and would like some advice on how to best configure the storage for High I/O performance with a reasonable level of redundancy.

The Proxmox host will have a mix of linux and windows VMs, but primarily for Windows VMs running infrastructure services: AD, App Server, File Server, VDI & Remote Desktop Session Hosts. The session host VMs will be the most resource hungry with up to 30 users per session host.

Dell PowerEdge R640

• 2 x Intel® Xeon® SP Gold
• 16-Core @ 2.10GHz Skylake
  Hyper-Threading-Technology Virtualization (Intel-VT)
• 512 GB DDR4 ECC
• 4 x 3.84 TB NVMe SSD Datacenter Edition
• 2 x 960 GB NVMe SSD Datacenter Edition

The use of enterprise nvme disks isn't an option, but I do have the two 960GB NVMEs that could be partitioned and used for L2ARC, ZIL, Special Device etc. in a RAID 1 mirror - I now that enterprise nvmes with power loss protection is recommended but I don't have that option.

I currently have the zfs_arc_max set to 16GB with 8TB of storage - should this be increased if there is sufficient host RAM? I would like to reserve as much RAM as possible for VMs as each user on a session host needs about 2-4GB and previous experiments have seen the VMs competing for RAM when letting ZFS manage the RAM at its default 50%.

 

[Dunuin]

L2ARC, SLOG and special devices won't do any magic. If your 960 GB SSDs aren't enterprise Intel Optanes this won't help much, as a 4x NVMe striped mirror should be faster than a 2x NVMe mirror, these 960GB NVMes might even slow down the other NVMes.