Alert messages in CEPH

Nov 7, 2019
14
0
1
47
Hello,

We have configured a Proxmox VE cluster with Ceph shared storage.

Two alert messages have appeared in CEPH for two days.

1 subtrees have overcompromised pool target_size_bytes

Clusters ['ceph-vm'] compromise the storage available in 1,289x due to target_size_bytes 0 in the clusters []


Can anyone help us?

Thank you
 
Hi alvin,

Thanks for answering so fast.

Initially we activate the autoscaling but then deactivate it.

Do you recommend activating it?

It's cluster consists of 3 hosts.

2 SSD (System) x host
3 SSD for CEPH x host

osd_pool_default_min_size = 2
osd_pool_default_size = 3

Pg_Num 512

1 single Pool

CEPH INFO:

# begin crush map
tunable choose_local_tries 0
tunable choose_local_fallback_tries 0
tunable choose_total_tries 50
tunable chooseleaf_descend_once 1
tunable chooseleaf_vary_r 1
tunable chooseleaf_stable 1
tunable straw_calc_version 1
tunable allowed_bucket_algs 54

# devices
device 0 osd.0 class ssd
device 1 osd.1 class ssd
device 2 osd.2 class ssd
device 3 osd.3 class ssd
device 4 osd. 4 class ssd
device 5 osd.5 class ssd
device 6 osd.6 class ssd
device 7 osd.7 class ssd
device 8 osd. 8 class ssd

# types
type 0 osd
type 1 host
type 2 chassis
type 3 rack
type 4 row
type 5 pdu
type 6 pod
type 7 room
type 8 datacenter
type 9 zone
type 10 region
type 11 root

# buckets
host S073 {
id -3 # do not change unnecessarily
id -4 class ssd # do not change unnecessarily
# weight 2,728
alg straw2
hash 0 # rjenkins1
item osd.0 weight 0.909
item osd.1 weight 0.909
item osd.2 weight 0.909
}
host S075 {
id -5 # do not change unnecessarily
id -6 class ssd # do not change unnecessarily
# weight 2,728
alg straw2
hash 0 # rjenkins1
item osd.3 weight 0.909
item osd.4 weight 0.909
item osd.5 weight 0.909
}
host S071 {
id -7 # do not change unnecessarily
id -8 class ssd # do not change unnecessarily
# weight 2,728
alg straw2
hash 0 # rjenkins1
item osd.6 weight 0.909
item osd.7 weight 0.909
item osd.8 weight 0.909
}
root default {
id -1 # do not change unnecessarily
id -2 class ssd # do not change unnecessarily
# weight 8,183
alg straw2
hash 0 # rjenkins1
item S073 weight 2,728
item S075 weight 2,728
item S071 weight 2,728
}

# rules
rule replicated_rule {
id 0
type replicated
min_size 1
max_size 10
step take default
step chooseleaf firstn 0 type host
step emit
}

# end crush map
 
I have activated the autoscale

1) ceph osd pool set ceph-vm pg_autoscale_mode on

And set PG_Num and PGP_Num to 256 (The cluster will not grow in capacity)

2) ceph osd pool set ceph-vm pg_num 256

3) ceph osd pool set ceph-vm pgp_num 256

But CEPH HEALTH_WARN still appears:

1 subtrees have overcommitted pool target_size_bytes

Pools ['ceph-vm'] overcommit available storage by 1.765x due to target_size_bytes 0 on pools []

1 subtrees have overcommitted pool target_size_ratio

Pools ['ceph-vm'] overcommit available storage by 1,763x due to target_size_ratio 0.000 on pools [


I'm lost.

What can I do, please?

Thanks for the help
CPD.PNG
 
What can I do, please?
Wait. Ceph needs time to re-balance / recover. The PGs in 'active+clean' state should shrink to 256 PG and all should be 'active+clean'.
 
Hi,

Errors no longer appear but health warning messages are still displayed

1 subtrees have overcommitted pool target_size_bytes

1 subtrees have overcommitted pool target_size_ratio


I need to do something else?

Is the configuration correct?

Thank you,
Captura2.PNG
 
I have adjusted the cluster:

ceph config set global mon_target_pg_per_osd 100

and

ceph osd pool set mypool target_size_ratio .9

But the messages still appear

What I do?

Should I disable the autoscaler?
 
As it says in the message, did you try to set it to 0?
 
Is the pool ceph-vm, empty?
 
What does the ceph osd pool autoscale-status show?
 
oot@S071:~# ceph osd pool autoscale-status
POOL SIZE TARGET SIZE RATE RAW CAPACITY RATIO TARGET RATIO BIAS PG_NUM NEW PG_NUM AUTOSCALE
ceph-vm 5172G 3.0 8379G 1.8521 0.9000 1.0 256
 

Attachments

  • Captura4.PNG
    Captura4.PNG
    19.3 KB · Views: 21

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!