ceph active+cleans PGs occur

[halnabriss]

Hi,
I have a healthy Ceph storage but in the status it shows active+clean PGs, as shown here:
# ceph -s
cluster f877d510-6946-4a66-bfbb-06b0ee12ae28
health HEALTH_OK
monmap e3: 3 mons at {0=10.1.1.1:6789/0,1=10.1.1.2:6789/0,2=10.1.1.3:6789/0}
election epoch 74, quorum 0,1,2 0,1,2
osdmap e328: 3 osds: 3 up, 3 in
pgmap v2345230: 160 pgs, 2 pools, 1197 GB data, 308 kobjects
3668 GB used, 3764 GB / 7433 GB avail
160 active+clean
client io 823 kB/s wr, 110 op/s

does that makes a problem as it occurred after some failures in my environment? and how to fix it?

thanks

 

[Ashley]

As it says all 160PGS are active and clean and your health is OK.

Normally this would have some doing scrubs, however it just look's like currently all scrubs are up to-date so all PG's are just in the clean state.

 

[halnabriss]

Thank you Ashley for your reply, but how I can do the scrubs ??

 

[Ashley]

Scrubs are done automaticly, unless disabled but your output shows they are not. No need for you to do anything.

 

[halnabriss]

@Ashley , does it take long time ?? the 160PGs number didn't change for two days now!

 

[Ashley]

Are you monitoring 24/7 in a small cluster with fast disks the scrubs will finish fairly quickly. And once there completed it will just go back to 100% active.

 

[halnabriss]

actually it didn't change , it still 160 PGs active+clean

 

[halnabriss]

Thanks Ashley for your replies,

but after 24 hours of this post I still have the same status :
# ceph -w
cluster f877d510-6946-4a66-bfbb-06b0ee12ae28
health HEALTH_OK
monmap e3: 3 mons at {0=10.1.1.1:6789/0,1=10.1.1.2:6789/0,2=10.1.1.3:6789/0}
election epoch 74, quorum 0,1,2 0,1,2
osdmap e332: 3 osds: 3 up, 3 in
pgmap v2389940: 160 pgs, 2 pools, 1198 GB data, 308 kobjects
3669 GB used, 3763 GB / 7433 GB avail
160 active+clean
client io 4078 B/s rd, 942 kB/s wr, 133 op/s

2017-05-21 17:40:00.934685 mon.0 [INF] pgmap v2389940: 160 pgs: 160 active+clean; 1198 GB data, 3669 GB used, 3763 GB / 7433 GB avail; 4078 B/s rd, 942 kB/s wr, 133 op/s
2017-05-21 17:40:02.943253 mon.0 [INF] pgmap v2389941: 160 pgs: 160 active+clean; 1198 GB data, 3669 GB used, 3763 GB / 7433 GB avail; 63913 B/s rd, 817 kB/s wr, 125 op/s
2017-05-21 17:40:04.948385 mon.0 [INF] pgmap v2389942: 160 pgs: 160 active+clean; 1198 GB data, 3669 GB used, 3763 GB / 7433 GB avail; 46918 B/s rd, 595 kB/s wr, 84 op/s
2017-05-21 17:40:05.953824 mon.0 [INF] pgmap v2389943: 160 pgs: 160 active+clean; 1198 GB data, 3669 GB used, 3763 GB / 7433 GB avail; 1219 kB/s rd, 1385 kB/s wr, 179 op/s
2017-05-21 17:40:07.960032 mon.0 [INF] pgmap v2389944: 160 pgs: 160 active+clean; 1198 GB data, 3669 GB used, 3763 GB / 7433 GB avail; 1231 kB/s rd, 1178 kB/s wr, 192 op/s
2017-05-21 17:40:09.969438 mon.0 [INF] pgmap v2389945: 160 pgs: 160 active+clean; 1198 GB data, 3669 GB used, 3763 GB / 7433 GB avail; 34663 B/s rd, 588 kB/s wr, 93 op/s


In the OSD logs this is what I have, I see PGs scrubbed, but the number of active+clean PGs still 160!!

# tail -f /var/log/ceph/ceph-osd.0.log
2017-05-21 09:08:19.469543 7f3d356f3700 0 log_channel(cluster) log [INF] : 4.44 deep-scrub starts
2017-05-21 09:13:37.292235 7f3d356f3700 0 log_channel(cluster) log [INF] : 4.44 deep-scrub ok
2017-05-21 16:12:43.724090 7f3d356f3700 0 log_channel(cluster) log [INF] : 4.19 scrub starts
2017-05-21 16:13:26.575925 7f3d356f3700 0 log_channel(cluster) log [INF] : 4.19 scrub ok
2017-05-21 16:51:13.064190 7f3d356f3700 0 log_channel(cluster) log [INF] : 5.7 scrub starts
2017-05-21 16:51:22.696194 7f3d356f3700 0 log_channel(cluster) log [INF] : 5.7 scrub ok
2017-05-21 17:28:43.838920 7f3d356f3700 0 log_channel(cluster) log [INF] : 5.1a scrub starts
2017-05-21 17:28:53.250315 7f3d356f3700 0 log_channel(cluster) log [INF] : 5.1a scrub ok
2017-05-21 17:32:03.996542 7f3d356f3700 0 log_channel(cluster) log [INF] : 5.1b scrub starts
2017-05-21 17:32:11.929624 7f3d356f3700 0 log_channel(cluster) log [INF] : 5.1b scrub ok

 

[spirit]

Hi Guys,

what's the problem here ? active+clean is the normal state when all is ok.

http://docs.ceph.com/docs/master/rados/operations/pg-states/
"
when checking a cluster’s status (e.g., running ceph -w or ceph -s), Ceph will report on the status of the placement groups. A placement group has one or more states. The optimum state for placement groups in the placement group map is active + clean.

"

 

[Ashley]

As expected the scan is taking only 5 minutes, so if your not watching at that 5 minutes you won't see the live status change.

 

[CLLlAgOB]

I have problem to.
two months "512 active+clean" stay freezy
ceph -s
cluster
health HEALTH_OK
monmap e5: 3 mons at {
election epoch 132, quorum 0,1,2 0,1,2
osdmap e7451: 15 osds: 15 up, 15 in
flags sortbitwise,require_jewel_osds
pgmap v820437: 512 pgs, 1 pools, 1459 GB data, 364 kobjects
4383 GB used, 5168 GB / 9552 GB avail
512 active+clean
client io 38719 B/s wr, 0 op/s rd, 3 op/s wr

 

[Ashley]

Don't see any issue your cluster is healthy and fine.