performance

[flexyz]

Hi

I am ceph rookie and have been working with lost of storage systems and find ceph very, very interessting - I have also been looking at Nutanix, and looks very cool but also too expensive

A 3 node setup with SSD/PCIE journal logs and with SAS spinning drives or maybe all flash, 10GE on the storage network - is performance good, great or awsome? - know it is hard to say, but if i follow the best practice what result could I exspect?

If anyone have any speed testing I would appreciate it

Thanks!
Felix

 

[spirit]

Hi,
here the result of my ceph cluster, 3 nodes (2x10core 3,1ghz each), 18 ssd intel s3610.

randread / read 4k : 700.000 iops
randwrite/ write 4k : 120.000 iops

for larger block, I can easily reach 10GB/s for reads.

The main bottleneck is cpu, mainly if you need a lot of iops. If you have big block workload (less iops, bigger bandwidth), you need less cpu.

This results are with multiple clients.
I can reach around 70000iops by disk in qemu (also cpu frequency is important in client if you need a lot of iops).


For write performance, It should improve in coming ceph releases, with the new bluestore backend.

 

[flexyz]

Thanks - and are you using the s3610 for the journal also and all SATA drives or PCIE? is 10GE storage network with Jumbo frames?

In general how big should the journal drive be?

 

[spirit]

Thanks - and are you using the s3610 for the journal also and all SATA drives or PCIE? is 10GE storage network with Jumbo frames?

In general how big should the journal drive be?

I have the journal&&datas on s3610 (1osd - 1 disk), sata drive. Yes jumbo frame.

for the size, the journal is fixed at 5GB by default.