Ceph - Bad performance in qemu-guests

[spirit]

Re: Ceph - Bad performance with small IO

I'm using new intel s3610 1,6TB in production. (cheaper than s3700, better endurance than s3500)

 

[Mn1sh]

Re: Ceph - Bad performance with small IO

Thanks phildefer for the link and clarifying the rbd-cache question.

Changing the cache-setting to writeback and tuning debug and sharding really helped a lot:

View attachment 2592

Thanks to everyone!

@spirit or maybe someone else can answer:
To further enhance performance I would also like to disable cephx as the cluster runs in a safe network, but the last time I tried to disable it, it left me with an unusable cluster.

Do I need to shutdown the ceph-cluster completely in order for this to work?

Do you add following to /etc/ceph/ceph.conf ?

filestore_fd_cache_size = 64
filestore_fd_cache_shards = 32
cephx sign messages = false
cephx require signatures = false
rbd_cache = true

And then enable cache option = writeback for qemu guest/vm under Hardware/Edit Disk ?


Do we have to shutdown and start ceph-cluster completely for the above changes Or just restarting guest vm should be enough?

 

[udo]

Do you add following to /etc/ceph/ceph.conf ?

filestore_fd_cache_size = 64
filestore_fd_cache_shards = 32
cephx sign messages = false
cephx require signatures = false
rbd_cache = true

And then enable cache option = writeback for qemu guest/vm under Hardware/Edit Disk ?


Do we have to shutdown and start ceph-cluster completely for the above changes Or just restarting guest vm should be enough?

Hi,
you should comment the "rbd_cache = true" in ceph.conf. With active settings, the qemu-cache settings will be ignored.
Without rdb_cache in ceph.conf the default will be used and qemu will overwrite (mean use the own) cache settings.

Udo