Ceph - Bad performance in qemu-guests

Re: Ceph - Bad performance with small IO

Thanks phildefer for the link and clarifying the rbd-cache question.

Changing the cache-setting to writeback and tuning debug and sharding really helped a lot:

View attachment 2592

Thanks to everyone! :)

@spirit or maybe someone else can answer:
To further enhance performance I would also like to disable cephx as the cluster runs in a safe network, but the last time I tried to disable it, it left me with an unusable cluster.

Do I need to shutdown the ceph-cluster completely in order for this to work?




Do you add following to /etc/ceph/ceph.conf ?

filestore_fd_cache_size = 64
filestore_fd_cache_shards = 32
cephx sign messages = false
cephx require signatures = false
rbd_cache = true

And then enable cache option = writeback for qemu guest/vm under Hardware/Edit Disk ?


Do we have to shutdown and start ceph-cluster completely for the above changes Or just restarting guest vm should be enough?
 
Do you add following to /etc/ceph/ceph.conf ?

filestore_fd_cache_size = 64
filestore_fd_cache_shards = 32
cephx sign messages = false
cephx require signatures = false
rbd_cache = true

And then enable cache option = writeback for qemu guest/vm under Hardware/Edit Disk ?


Do we have to shutdown and start ceph-cluster completely for the above changes Or just restarting guest vm should be enough?
Hi,
you should comment the "rbd_cache = true" in ceph.conf. With active settings, the qemu-cache settings will be ignored.
Without rdb_cache in ceph.conf the default will be used and qemu will overwrite (mean use the own) cache settings.

Udo
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!