Increase Ceph recovery speed

[Thaillie]

I'm in the middle of migrating my current osd's to Bluestore but the recovery speed is quite low (5600kb/s ~10 objects/s). Is there a way to increase the speed?

I currently have no virtual machines running on the cluster so performance doesn't matter at the moment. Only the recovery is running.

 

[fabian]

See http://docs.ceph.com/docs/luminous/rados/configuration/osd-config-ref/#backfilling for some knobs to play around with

 

[bradkollmyer]

I found the following worked well for me:

ceph tell 'osd.*' injectargs '--osd-max-backfills 16'
ceph tell 'osd.*' injectargs '--osd-recovery-max-active 4'

 

[fxandrei]

What are the current settings ? How could i read them ?
Did anyone had problems with these settings ?

 

[budy]

You should be able to query these values like this (note that you'd need to query each osd daemon for it's individual settings, hence the slightly different call):

ceph daemon osd.1 config get osd_max_backfills
ceph daemon osd.1 config get osd_recovery_max_active

I haven't been using injectargs a lot, since I had no need for that, but when I tried it out, it worked flawlessly.