ram usage with bluestore

[Stefano Giunchi]

this is my test cluster:
node A: n.3 filesystem 1TB OSDs
node B: n.2 filesystem 1TB OSDs, n.1 bluestore 1TB OSDs,
node C: n.6 bluestore 300GB OSDs

I noticed that bluestore OSDs take 3.5GB of RAM each, while the fileystem ones take 0.7 GB each.

Following this thread, I added this to ceph.conf:

[osd]
bluestore cache size = 1G

Now everything is ok, but is this the right thing to do? Or bluestore OSDs take as much RAM as is available, but releases it if needed?

 

[sg90]

filestore can use the system page cache so the ram use you see is only what the OSD requires to operate.

With bluestore it can’t use the page cache for caching so has its own inbuilt ram cache.

The value you set will be used as an aim to keep the operation ram and the cache ram within target, you will want to set this as close to the default which is 4GB per OSD as you can to allow space for caching.

If 1GB is the best you can do without hitting swap (which will just have a negative impact) then leave it at this.

 

[Stefano Giunchi]

thank you for the info.

 

[sb-jw]

The bluestore_cache_* options are no longer needed. They are replaced
by osd_memory_target, defaulting to 4GB. BlueStore will expand
and contract its cache to attempt to stay within this
limit. Users upgrading should note this is a higher default
than the previous bluestore_cache_size of 1GB, so OSDs using
BlueStore will use more memory by default.

https://ceph.com/releases/v12-2-10-luminous-released/