Trying to run some benchmark on this newly ext4 converted ceph cluster. Is it best to run 1 to 32 thread individual benchmark or without any thread mention -t is good enough? Which thread is it exactly, ceph disk thread or op thread?
Ceph OSD failure causing Proxmox node to crash
- Thread starter wahmed
- Start date
Hi,
AFAIK this parameter is (was?) only nessessary for cephfs. On the mailingliste was ask for the use with rbd; but no answer from the devs...
Due the now aktive parameter "filestore_omap_backend": "leveldb" I assume the extended attributes are not longer used...
Udo
Hi,
depends what you like to meassure. A single VM use only one thread. With one thread you get only small writes/reads. With more threads (like many concurrent VM-access) you get better values.
I use simply without -t to compare different configs/setups. Sometimes also with -t 1.
rbd threads. rados bench run on the "client" node. op treads are on the osd nodes (in your setup client + osd node are (partly) the same)
Udo
Would Ceph benefit from BBU cached RAID controller? or in JBOD mode cache does not come in effect in Ceph?
The write performance that you have posted so far Udo, are they on cached RAID card?
The write performance that you have posted so far Udo, are they on cached RAID card?
Hi,
answered in the new thread.
no, this values are due the SSD for journaling (DC S3700).
Udo
Finally ran some benchmarks on the cluster with SSD added for journals. To be honest not really sure what to make of it. Numbers do not look that great.
-Caches were dropped for each and every read.
-All OSDs benched at 200+ mb/s with #rados tell osd.<id> bench
-Each SSD used have 50,000+ IOPS
-Rados benchmarks was done for 1 to 16 threads.
Following are the screenshot for 4096, 131072 and 4M blocks from thread 1 to 16:
Block 4096
Block 131072
Block 4M
Block Not Defined
In suggestions on how to improve this and what to check specifically is much appreciated.
-Caches were dropped for each and every read.
-All OSDs benched at 200+ mb/s with #rados tell osd.<id> bench
-Each SSD used have 50,000+ IOPS
-Rados benchmarks was done for 1 to 16 threads.
Following are the screenshot for 4096, 131072 and 4M blocks from thread 1 to 16:
Block 4096
Block 131072
Block 4M
Block Not Defined
In suggestions on how to improve this and what to check specifically is much appreciated.
Last edited: