Ceph: Increasing pg/pgp_num from 1024->2048

J

John.N

Active Member
Sep 21, 2019
24
3
43
34
Hello,

I recently passed 41 OSDs on our cluster, and the PG calc suggests we should be at 2048 pgs now.
Also, I'm going to be adding 5 OSDs soon too.

I've read some people saying that it's best to increase pg_num in increments of 256.

Has anyone got experience in increasing from 1024 to 2048 or maybe more, and what do you guys think?


Thanks!
 
Depending on your Ceph version, the autoscaler could be of use. The increase will happen automatically in the background.
https://docs.ceph.com/en/nautilus/rados/operations/placement-groups/

On the other hand when you set the new PG count Ceph will start to create the PGs and then rebalance/recover the data. Might be more resource intensive then the autoscaler, you may use osd_max_backfills or others to tune that process. Depending on your hardware of course.
 
Hello @Alwin , Luminous has no autoscaler AFAIK.

I have osd_max_backfills set to '1', so I'm thinking of just increasing pg_num , let it run slowly (for hours probably) and then pgp_num.

What do you think?
 
Thank you for your link @RokaKen .
I'm running with 2x10G LAG and SSD disks (not NVMe), do you think that going up in 128 increments and with backfill set to '1' be OK for my users, as I want minimal impact on performance?

Of course it's going to run in the least busy time of day.
 
Hello John,

Can you share please the procedure (commands) for increasing the PG from 1024 to 2048?

Thanks.
 
You must log in or register to reply here.
Top Bottom