SSD setup for VM host

[pos-sudo]

What? Using two pools is a waste of money. There is no benefit in using two disks for the OS and two disks for the VMs besides having PVE and the VMs separate in case of a reinstall and that depends on the capabilities of the admin. 4 disks outperform 2 in all occasions, even using 4x 1 TB instead of 2x 2 TB + OS-SSDs will be faster and most probably cheaper.

Thanks for your answer! Does the Intel D3-S4510 1.92TB in your opinion fits with proxmox ?

 

[LnxBil]

Does the Intel D3-S4510 1.92TB in your opinion fits with proxmox ?

I haven't laid my hands on it, so I cannot say if it works or but, there a quick google search looks promising with respect to enterprise hardware. That is the main goal here.

We use hardware manfucaturer provided ssds with their warranty (e.g. build and sold all-in-one Dell/HP/Fujitsu/Cisco) for many years and hadn't had any problems with compatibility or failures. As long as you're not going with entry-level hardware (single socket stuff, embedded MegaRaid), you should not have a problem with this.

For best compatibility: If your vendor has a support matrix, look for Ubuntu compatibility. The PVE kernel is taken from Ubuntu, so the hardware should work. If that does not exist, using the RedHat hardware compatibility should also be fine.

 

[spirit]

I'm using s4510 && s4610 in production in ceph cluster, they works really fine with syncronous writes (10000-20000 iops with 4k block).
(vs 400 iops with mx500

They should be fine with zfs too.

 

[pos-sudo]

I'm using s4510 && s4610 in production in ceph cluster, they works really fine with syncronous writes (10000-20000 iops with 4k block).
(vs 400 iops with mx500

They should be fine with zfs too.

Thanks you very much! We now know what to do I let you guys know when we moved out the disks and test with the s4510.

 

[mvs]

We just tested on Intel D3-S4510 with a plesk environment, and if we test write with:

dd if=/dev/zero of=/root/testfile bs=1024M count=1 oflag=direct
The speed seems normal within under 2 seconds 1.1Gb copied. But in Proxmox we see a IO delay spike to 45% and then immediatly to 0% is this normal with this tests?

It is quite normal to see high IO delay during benchmark.
The problem with your test is that it does not reflect your typical workload. Run fio with --bs=4k --direct=1 --sync=1 --rw=randwrite to see the difference between mx500 and S4510. You can also run pveperf on your ZFS pool to see how many fsyncs/s you can achieve (datacenter grade SSDs have usually 5000+ fsyncs/s, MX500 just around 300 fsyncs/s).

 

[pos-sudo]

It is quite normal to see high IO delay during benchmark.
The problem with your test is that it does not reflect your typical workload. Run fio with --bs=4k --direct=1 --sync=1 --rw=randwrite to see the difference between mx500 and S4510. You can also run pveperf on your ZFS pool to see how many fsyncs/s you can achieve (datacenter grade SSDs have usually 5000+ fsyncs/s, MX500 just around 300 fsyncs/s).

Weird enough...

pveperf MX500

root@stb01:~# pveperf /rpool/
CPU BOGOMIPS: 231792.20
REGEX/SECOND: 3000297
HD SIZE: 893.86 GB (rpool)
FSYNCS/SECOND: 10971.13
DNS EXT: 16.00 ms
DNS INT: 65.45 ms (netbeheer.eu)

pveperf S4510

root@stb01:~# pveperf /vmpool/
CPU BOGOMIPS: 231792.20
REGEX/SECOND: 3052978
HD SIZE: 1447.02 GB (vmpool)
FSYNCS/SECOND: 2585.96
DNS EXT: 16.09 ms
DNS INT: 44.72 ms (netbeheer.eu)

Edit: My bad, this was a test with sync disabled for RPOOL (MX500) and sync standard for VMPOOL (S4510). After a new test with both enabled MX500 has 409 FSYNCS and both disabled has S4510 22K.
I think this test result is very usefull now I know we can definitely run critical applications on the S4510 without a high IOWAIT

 

[pos-sudo]

Yesterday we migrated to the new Intel D3 S4510 1.92TB disks and we are good with the IO now. This thread can thus be marked as resolved.