ceph : [client] rbd cache = true override qemu cache=none|writeback

[spirit]

Hi,

a small note about ceph,

currently if you define

[client]
rbd_cache=true|false

in /etc/ceph/ceph.conf

it's override the qemu cache=none|writeback.
(so you'll always have cache enable of disable from rbd_cache value for all your vms)

It's a bug, and ceph team have send a fix this week.

So, for now, it's better to remove rbd_cache from ceph.conf

 

[udo]

Hi,

a small note about ceph,

currently if you define

[client]
rbd_cache=true|false

in /etc/ceph/ceph.conf

it's override the qemu cache=none|writeback.
(so you'll always have cache enable of disable from rbd_cache value for all your vms)

It's a bug, and ceph team have send a fix this week.

So, for now, it's better to remove rbd_cache from ceph.conf

Hi Spirit,
is an removing of rbd_cache enough?
Because since giant rbd cache is enabled by default (mail from sage about the giant 0.87 release):

* The client-side caching for librbd is now enabled by default (rbd
  cache = true).  A safety option (rbd cache writethrough until flush
  = true) is also enabled so that writeback caching is not used until
  the library observes a 'flush' command, indicating that the librbd
  users is passing that operation through from the guest VM.

Or do you mean with "all your vms" the vms, which aren't use ceph storage also?

Udo

 

[spirit]

>>is an removing of rbd_cache enough?
yes

>>Because since giant rbd cache is enabled by default (mail from sage about the giant 0.87 release):

In any case, in qemu, we always defined cache=none or cache=writeback, this enable or disable rbd cache in all case.




>>Or do you mean with "all your vms" the vms, which aren't use ceph storage also?
No, of course, only the qemu disks using rbd/ceph.

 

[udo]

>>is an removing of rbd_cache enough?
yes

>>Because since giant rbd cache is enabled by default (mail from sage about the giant 0.87 release):

In any case, in qemu, we always defined cache=none or cache=writeback, this enable or disable rbd cache in all case.

Hi Spirit,
thanks for the info.

Are other cache params affected by qemu?
E.G. I have in ceph.conf the entry

#rbd cache = true
rbd cache writethrough until flush = true

Is this setting used by qemu? Or, due commented rbd_cache, useless?

Udo

 

[spirit]

Hi Spirit,
thanks for the info.

Are other cache params affected by qemu?
E.G. I have in ceph.conf the entry

#rbd cache = true
rbd cache writethrough until flush = true

Is this setting used by qemu? Or, due commented rbd_cache, useless?

Udo

It's only the rbd_cache value. (which is setup by qemu cache option).
Other options in ceph.conf still apply.

 

[gkovacs]

In any case, in qemu, we always defined cache=none or cache=writeback, this enable or disable rbd cache in all case.

Is this still true? rbd_cache only active if cache=none is set for the disk of the KVM guest?

 

[aderumier]

Is this still true? rbd_cache only active if cache=none is set for the disk of the KVM guest?

no,it's has been fixed a long time ago.

qemu cache=none -> rbd_cache=false
qemu cache=write-> rbd_cache=true

 

[Jarek]

http://docs.ceph.com/docs/master/rbd/qemu-rbd/#qemu-cache-options