SATA vs SAS for ceph

[lifeboy]

Hi all,

In your practical experience, when I choose new hardware for a cluster, is there any noticable difference between using SATA or SAS drives. I know SAS drives can have a 12Gb/s interface and I think SATA can only do 6Gb/s, but in my experience the drives themselves can't write at 12Gb/s anyway, so it makes little if any difference.

I use a combination of SSD's and SAS drives in my current cluster (in different ceph pools), but I suspect that if I choose SATA enterprise class drives for this project, it will get the same level of performance.

I think with ceph the hard error rate of drives becomes less relevant that if I had used some level of RAID.

Also, if I go with SATA, I can use AMD Epyc processors (and I don't want to use a different supplier), which gives me a lot of extra cores per unit at a lesser price, which of course all adds up to a better deal in the end.

I'd like to specifically hear from you what your experience is in this regard.

 

[SINOS]

We are using SAS SSDs in some servers because back in the day NVMe was super new and astronomically expensive.
Nowadays I'd say "go NVMe, ignore SAS" if you need high performance and/or low latency.

For mainstream or read intense workloads, an Enterprise SATA SSD is still excellent, if you get enough of them they can also still reach a higher level of raw performance.

About HDDs:
I'd ignore SAS here too, the difference between SATA & SAS HDDs is so small... and if you really need more performance, just go with some cheap read-centric SSDs instead of HDDs.

Also, if I go with SATA, I can use AMD Epyc processors

You can use SAS with EPYC, too. Just use an HBA card instead of the onboard Ports or choose a Mainboard that provides onboard SAS instead of SATA.

but in my experience the drives themselves can't write at 12Gb/s anyway

This is true for HDDs, not for SSDs. SAS SSDs are capable of ~500k IOPS / 2GB/s for sure.

 

[lifeboy]

We are using SAS SSDs in some servers because back in the day NVMe was super new and astronomically expensive.
Nowadays I'd say "go NVMe, ignore SAS" if you need high performance and/or low latency.

For mainstream or read intense workloads, an Enterprise SATA SSD is still excellent, if you get enough of them they can also still reach a higher level of raw performance.

About HDDs:
I'd ignore SAS here too, the difference between SATA & SAS HDDs is so small... and if you really need more performance, just go with some cheap read-centric SSDs instead of HDDs.


You can use SAS with EPYC, too. Just use an HBA card instead of the onboard Ports or choose a Mainboard that provides onboard SAS instead of SATA.


This is true for HDDs, not for SSDs. SAS SSDs are capable of ~500k IOPS / 2GB/s for sure.

Thanks for this. We are using the spindle drives for backups and important bulk storage more than for transactional processes. We also use NVMe SSDs for the high perfomance pool.

 

[SINOS]

We are using the spindle drives for backups and important bulk storage more than for transactional processes.

In that case, some good SATA drives, rated for 24/7 will do a great job, no real need for SAS imho.
Personally, I made great experiences with the "Toshiba Enterprise Capacity" (MG ...... ) Drives and the Toshiba N300 NAS drives.