[SOLVED] Ceph OSD Pool Error - PG Count too high?

[emilhozan]

Hey all,

I have a second cluster currently with 9x nodes. Ceph storage, each node has 4x HDDS and 1x SSD; 1x SAS drive that houses PVE OS.

I adjusted the crush map for an SSD rule / pool and another for HDD rule / pool. I followed the ceph calc and was able to create the pool okay for the SSDs but for HDD I am getting an error:

TASK ERROR: mon_command failed - For better initial performance on pools expected to store a large number of objects, consider supplying the expected_num_objects parameter when creating the pool.Lÿ


When looking it up, I saw this link but I don't quite understand the response:

https://forum.proxmox.com/threads/creating-ceph-pool-according-to-pg-calc.54040/

To be clear, I aim at creating 2x pools: 1x for SSD (which worked fine) and another for HDD (this is where I'm having issues). Here's the screen shot with the details I filled in:


Should I lower the PG count?
I changed the "Target PGs per OSD" to 100 and got 2048 "Suggested PG Count" but ran into the same error when trying to configure it as such in PVE UI.

 

[RobFantini]

It sounds like you should lower the PG count.

size should be 3 ? check in to pve and other suggestions on that.

we have lowered PG's in the past . naturally the more data you have the longer it takes to complete. latency may get really slow , so do the drop when little or not data i/o required.

AFAIK you'd need to lower from cli. here is an example

ceph osd pool set  <POOLNAME>   pg_num 512
ceph osd pool set  <POOLNAME>  pgp_num 512

Now I am not a ceph expert, and https://ceph.com/pgcalc/ is not something I totally understand. However i think you do not want to use 100% at data. leave a percentage for the other pool you will add later on.

 

[Alwin]

I changed the "Target PGs per OSD" to 100 and got 2048 "Suggested PG Count" but ran into the same error when trying to configure it as such in PVE UI.

Size of 3 will give you 1024 PGs. And don't worry, with Ceph Nautilus it is no problem to change the PG count later on.