Which is better? ssd-caching or os and mysql on SSD

[sahostking]

We have a dell r610 server with 6 x 2.5 inch.

Considering one of these options to host a few cPanel VMs on proxmox host.
Which should provide the best performance?

2 x 1TB SSD disks in RAID 1and 4 x 2 TB SSD disks in RAID 10.
Then for VMS we will set OS and MySQL on SSD disks and /home directory where data is on SATA.

alternatively.

2 x 1TB SSD disks in RAID 1 for SSD Caching and 4 x 2TB Disks in RAID 10.

Thoughts?

Full SSD is too costly currently for this client.

 

[Tapio Lehtonen]

4 x 2 TB SSD disks in RAID 10

The RAID 10 is faster, since it is a 2 wide stripe, if the 1 TB SSD and 2 TB SSD disks have similar speeds.

 

[sahostking]

woops I'm an idiot. I meant for the first option " x 2 TB SATA disks" for data

 

[LnxBil]

How do you want to have SSD caching? Via CacheCade?

 

[Vladimir Bulgaru]

Hey!

From my experience, the very first questions that you need to address are

1. what are you trying to achieve?
2. where is the bottleneck?

Cheatsheet
When considering the technology used, here's what may be useful in terms of speed assessment:

1. SAS is usually faster than SATA (although fast SATA can manage same speeds as a slow SAS)
2. RAID0 doubles the write speed, but leaves the read speed the same
3. RAID1 doubles the read speed, but leaves the write speed the same
   
4. RAID10 doubles the write speed and the read speed
   
5. PCIe SSDs tend to be as fast or faster than RAID10
6. The perception of how snappy is the OS comes from RW speed and IOPS

In theory, you will get the best speeds out of SAS drives in RAID10 when talking about non-PCIe drives. But this is not the maximum speed you can attain, nor does it guarantee good IOPS.

Methodology

1. I would encourage you to analyse what you want to achieve with the speed. At some point, you can be bottlenecked by bus speed, IOPS, hypervisor settings and file system configuration. Since you are using SSDs, it is very unlikely that the client OSes will run slow because of the speed of the drives. It may easily be the case that you haven't tweaked the client VMs (here's one of my examples). Or the RAID controller is performing poorly.
2. Map all the options you have: what is the ideal file system for data integrity, what are the possible disk setups, if you are able to add a PCIe device, if the client VMs and hypervisor are tweaked to offer the best speeds. You may also need to estimate the space needed (e.g. RAID 1 and 10 cut the space into half and you may not be able to afford it). Also check how failure prone and failure tolerant is your system. If it's a mission critical machine and downtime is not tolerable, you may want to stay away from SATA drives, HW RAIDs and non-redundant PCIe SSDs.
3. Decide based on your goals and options. E.g.:
   1. Cheap: SATA drives in RAID10 handled by HW RAID controller. I would strengthen the setup by avoiding the HW RAID and go with the ZFS RAID10.
   2. IOPS: PCIe SSD. Those tend to have speeds beyond 1GBps and incredible IOPS.
   3. Speed: PCIe SSD or SAS SSDs in RAID10. Should bring you to at least 1GBps. Or PCIe SSD.
   4. Reliability (non mission critical): enterprise PCIe SSD (very reliable, even if older, especially SLC type) and back up on ZFS RAID10 (can even go with slow spinning drives)
   5. Reliability (mission critical): create a cluster and replicate the VM(-s) across at least 2 machines. Would still go with enterprise PCIe SSDs.

Conclusions
I went a bit off-topic with this post. Answering your question directly, yes - the 4 SATA SSDs in RAID10 should offer best speeds. But what i tried to elaborate on is - the RW speed may not be the thing that you need to worry about. So when choosing the storage solution, factor in the other aspects as well, such as IOPS, failure risk, data integrity and goals that you try to achieve with the system.

Hope it helps.

 

[LnxBil]

This is a nice cheat sheet and adequate for ballpark numbers, but the details may also be interesting. The closer you look the more edge cases arise. The numbers are also implementation detail and can vary.

RAID0 doubles the write speed, but leaves the read speed the same

Both, the read and write speed are doubled (with two devices) for files greater than at least twice stripe size. You will - however - loose everything if one disk breaks.

RAID10 doubles the write speed and the read speed

You need 4 devices and you will quadruple the read speed and double the write speed of a single device.

PCIe SSDs tend to be as fast or faster than RAID10

Very general rule of thumb, depends on the number of devices in a RAID10 and the used HBA. You can also RAID PCIe SSDs with a good mainboard, so PCIe and RAID do not have to be mutual exclusive.

 

[Vladimir Bulgaru]

@LnxBil, you are correct regarding RAID speeds and PCIe RAID, thank you for correction!

 

[udo]

We have a dell r610 server with 6 x 2.5 inch.

Considering one of these options to host a few cPanel VMs on proxmox host.
Which should provide the best performance?

2 x 1TB SSD disks in RAID 1and 4 x 2 TB SSD disks in RAID 10.
Then for VMS we will set OS and MySQL on SSD disks and /home directory where data is on SATA.

alternatively.

2 x 1TB SSD disks in RAID 1 for SSD Caching and 4 x 2TB Disks in RAID 10.

Thoughts?

Full SSD is too costly currently for this client.

Hi,
you don't wrote which perc do you have in the R610.
With the BBU-Version you can't use pass through - for zfs you must work with single raid-0 which raided with zfs (ugly)!
ZFS (zol) on SSD-Raid 1 is much slower than an raid-1 with the lsi-controller (don't mean the H330).
Esp. with heavy IO. In this case you get an jitter with ZFS, which is gone with the hardware-raid.

Udo

 

[sahostking]

Perc 6/i is being used but we do have some h700s around I think I should switch this server to use.
We also have SAS 10k disks which we could probably use and test with.

 

[sahostking]

found a old Perc 6/ir which is in IT Mode. Should work best for ZFS though.

Will test with all 3 raid controllers but I assume the H700 in HWRaid with BBU in writeback should provide the best speeds.