Hi
So I've built my Single Node Proxmox CEPH system and did not have a lot of issues the first time around. Other than some of my disks being a little old and giving errors.
I wanted to start from scratch so I did not have the crund from me learning, but the second time around I simple can not get the cluster to leave HEALTH_WARN after adding the 'Pool'
I get the following
Degraded data redundancy: 600 pgs unclean, 600 pgs degraded, 600 pgs undersized
pg 8.225 is stuck unclean for 55182.755301, current state active+undersized+degraded, last acting [1]
pg 8.226 is stuck undersized for 564.443895, current state active+undersized+degraded, last acting [8]
pg 8.227 is stuck undersized for 573.287082, current state active+undersized+degraded, last acting [10]
pg 8.228 is stuck undersized for 559.448944, current state active+undersized+degraded, last acting [11]
pg 8.229 is stuck undersized for 559.448007, current state active+undersized+degraded, last acting [11]
pg 8.22a is stuck undersized for 571.265785, current state active+undersized+degraded, last acting [1]
pg 8.22b is stuck undersized for 568.252327, current state active+undersized+degraded, last acting [4]
pg 8.22c is stuck undersized for 565.452299, current state active+undersized+degraded, last acting [3]
pg 8.22d is stuck undersized for 571.267499, current state active+undersized+degraded, last acting [9]
pg 8.22e is stuck undersized for 564.453585, current state active+undersized+degraded, last acting [2]
pg 8.22f is stuck undersized for 569.249852, current state active+undersized+degraded, last acting [7]
pg 8.230 is stuck undersized for 564.444850, current state active+undersized+degraded, last acting [8]
pg 8.231 is stuck undersized for 573.288098, current state active+undersized+degraded, last acting [10]
pg 8.232 is stuck undersized for 571.263607, current state active+undersized+degraded, last acting [6]
pg 8.233 is stuck undersized for 570.265214, current state active+undersized+degraded, last acting [0]
pg 8.234 is stuck undersized for 564.444178, current state active+undersized+degraded, last acting [8]
pg 8.235 is stuck undersized for 574.304178, current state active+undersized+degraded, last acting [5]
pg 8.236 is stuck undersized for 571.267513, current state active+undersized+degraded, last acting [9]
pg 8.237 is stuck undersized for 565.452799, current state active+undersized+degraded, last acting [3]
pg 8.238 is stuck undersized for 573.288273, current state active+undersized+degraded, last acting [10]
pg 8.239 is stuck undersized for 564.443146, current state active+undersized+degraded, last acting [8]
pg 8.23a is stuck undersized for 565.451165, current state active+undersized+degraded, last acting [3]
pg 8.23b is stuck undersized for 564.443410, current state active+undersized+degraded, last acting [8]
pg 8.23c is stuck undersized for 565.449937, current state active+undersized+degraded, last acting [3]
pg 8.23d is stuck undersized for 564.452942, current state active+undersized+degraded, last acting [2]
pg 8.23e is stuck undersized for 573.287530, current state active+undersized+degraded, last acting [10]
pg 8.23f is stuck undersized for 571.263310, current state active+undersized+degraded, last acting [6]
pg 8.240 is stuck undersized for 564.443736, current state active+undersized+degraded, last acting [8]
pg 8.241 is stuck undersized for 568.252967, current state active+undersized+degraded, last acting [4]
pg 8.242 is stuck undersized for 571.266203, current state active+undersized+degraded, last acting [9]
pg 8.243 is stuck undersized for 571.266709, current state active+undersized+degraded, last acting [1]
pg 8.244 is stuck undersized for 571.266725, current state active+undersized+degraded, last acting [6]
pg 8.245 is stuck undersized for 564.452589, current state active+undersized+degraded, last acting [2]
pg 8.246 is stuck undersized for 559.449208, current state active+undersized+degraded, last acting [11]
pg 8.247 is stuck undersized for 570.265209, current state active+undersized+degraded, last acting [0]
pg 8.248 is stuck undersized for 569.247919, current state active+undersized+degraded, last acting [7]
pg 8.249 is stuck undersized for 571.263496, current state active+undersized+degraded, last acting [6]
pg 8.24a is stuck undersized for 573.286598, current state active+undersized+degraded, last acting [10]
pg 8.24b is stuck undersized for 565.452930, current state active+undersized+degraded, last acting [3]
pg 8.24c is stuck undersized for 559.449207, current state active+undersized+degraded, last acting [11]
pg 8.24d is stuck undersized for 571.267996, current state active+undersized+degraded, last acting [1]
pg 8.24e is stuck undersized for 569.249893, current state active+undersized+degraded, last acting [7]
pg 8.24f is stuck undersized for 570.264109, current state active+undersized+degraded, last acting [0]
pg 8.250 is stuck undersized for 569.249955, current state active+undersized+degraded, last acting [7]
pg 8.251 is stuck undersized for 568.252723, current state active+undersized+degraded, last acting [4]
pg 8.252 is stuck undersized for 564.453174, current state active+undersized+degraded, last acting [2]
pg 8.253 is stuck undersized for 574.304120, current state active+undersized+degraded, last acting [5]
pg 8.254 is stuck undersized for 559.448704, current state active+undersized+degraded, last acting [11]
pg 8.255 is stuck undersized for 564.453662, current state active+undersized+degraded, last acting [2]
pg 8.256 is stuck undersized for 569.247453, current state active+undersized+degraded, last acting [7]
pg 8.257 is stuck undersized for 571.266890, current state active+undersized+degraded, last acting [6]
I added the osd crush chooseleaf type = 0 to ceph.conf but I'm not 100% sure I did it before creating the monitor and I had trouble with the manager not starting (initially)
If I remove the Pool the cluster goes back to HEALTHY.
I'm totally at a lost to find what I did wrong.
Mike
So I've built my Single Node Proxmox CEPH system and did not have a lot of issues the first time around. Other than some of my disks being a little old and giving errors.
I wanted to start from scratch so I did not have the crund from me learning, but the second time around I simple can not get the cluster to leave HEALTH_WARN after adding the 'Pool'
I get the following
Degraded data redundancy: 600 pgs unclean, 600 pgs degraded, 600 pgs undersized
pg 8.225 is stuck unclean for 55182.755301, current state active+undersized+degraded, last acting [1]
pg 8.226 is stuck undersized for 564.443895, current state active+undersized+degraded, last acting [8]
pg 8.227 is stuck undersized for 573.287082, current state active+undersized+degraded, last acting [10]
pg 8.228 is stuck undersized for 559.448944, current state active+undersized+degraded, last acting [11]
pg 8.229 is stuck undersized for 559.448007, current state active+undersized+degraded, last acting [11]
pg 8.22a is stuck undersized for 571.265785, current state active+undersized+degraded, last acting [1]
pg 8.22b is stuck undersized for 568.252327, current state active+undersized+degraded, last acting [4]
pg 8.22c is stuck undersized for 565.452299, current state active+undersized+degraded, last acting [3]
pg 8.22d is stuck undersized for 571.267499, current state active+undersized+degraded, last acting [9]
pg 8.22e is stuck undersized for 564.453585, current state active+undersized+degraded, last acting [2]
pg 8.22f is stuck undersized for 569.249852, current state active+undersized+degraded, last acting [7]
pg 8.230 is stuck undersized for 564.444850, current state active+undersized+degraded, last acting [8]
pg 8.231 is stuck undersized for 573.288098, current state active+undersized+degraded, last acting [10]
pg 8.232 is stuck undersized for 571.263607, current state active+undersized+degraded, last acting [6]
pg 8.233 is stuck undersized for 570.265214, current state active+undersized+degraded, last acting [0]
pg 8.234 is stuck undersized for 564.444178, current state active+undersized+degraded, last acting [8]
pg 8.235 is stuck undersized for 574.304178, current state active+undersized+degraded, last acting [5]
pg 8.236 is stuck undersized for 571.267513, current state active+undersized+degraded, last acting [9]
pg 8.237 is stuck undersized for 565.452799, current state active+undersized+degraded, last acting [3]
pg 8.238 is stuck undersized for 573.288273, current state active+undersized+degraded, last acting [10]
pg 8.239 is stuck undersized for 564.443146, current state active+undersized+degraded, last acting [8]
pg 8.23a is stuck undersized for 565.451165, current state active+undersized+degraded, last acting [3]
pg 8.23b is stuck undersized for 564.443410, current state active+undersized+degraded, last acting [8]
pg 8.23c is stuck undersized for 565.449937, current state active+undersized+degraded, last acting [3]
pg 8.23d is stuck undersized for 564.452942, current state active+undersized+degraded, last acting [2]
pg 8.23e is stuck undersized for 573.287530, current state active+undersized+degraded, last acting [10]
pg 8.23f is stuck undersized for 571.263310, current state active+undersized+degraded, last acting [6]
pg 8.240 is stuck undersized for 564.443736, current state active+undersized+degraded, last acting [8]
pg 8.241 is stuck undersized for 568.252967, current state active+undersized+degraded, last acting [4]
pg 8.242 is stuck undersized for 571.266203, current state active+undersized+degraded, last acting [9]
pg 8.243 is stuck undersized for 571.266709, current state active+undersized+degraded, last acting [1]
pg 8.244 is stuck undersized for 571.266725, current state active+undersized+degraded, last acting [6]
pg 8.245 is stuck undersized for 564.452589, current state active+undersized+degraded, last acting [2]
pg 8.246 is stuck undersized for 559.449208, current state active+undersized+degraded, last acting [11]
pg 8.247 is stuck undersized for 570.265209, current state active+undersized+degraded, last acting [0]
pg 8.248 is stuck undersized for 569.247919, current state active+undersized+degraded, last acting [7]
pg 8.249 is stuck undersized for 571.263496, current state active+undersized+degraded, last acting [6]
pg 8.24a is stuck undersized for 573.286598, current state active+undersized+degraded, last acting [10]
pg 8.24b is stuck undersized for 565.452930, current state active+undersized+degraded, last acting [3]
pg 8.24c is stuck undersized for 559.449207, current state active+undersized+degraded, last acting [11]
pg 8.24d is stuck undersized for 571.267996, current state active+undersized+degraded, last acting [1]
pg 8.24e is stuck undersized for 569.249893, current state active+undersized+degraded, last acting [7]
pg 8.24f is stuck undersized for 570.264109, current state active+undersized+degraded, last acting [0]
pg 8.250 is stuck undersized for 569.249955, current state active+undersized+degraded, last acting [7]
pg 8.251 is stuck undersized for 568.252723, current state active+undersized+degraded, last acting [4]
pg 8.252 is stuck undersized for 564.453174, current state active+undersized+degraded, last acting [2]
pg 8.253 is stuck undersized for 574.304120, current state active+undersized+degraded, last acting [5]
pg 8.254 is stuck undersized for 559.448704, current state active+undersized+degraded, last acting [11]
pg 8.255 is stuck undersized for 564.453662, current state active+undersized+degraded, last acting [2]
pg 8.256 is stuck undersized for 569.247453, current state active+undersized+degraded, last acting [7]
pg 8.257 is stuck undersized for 571.266890, current state active+undersized+degraded, last acting [6]
I added the osd crush chooseleaf type = 0 to ceph.conf but I'm not 100% sure I did it before creating the monitor and I had trouble with the manager not starting (initially)
If I remove the Pool the cluster goes back to HEALTHY.
I'm totally at a lost to find what I did wrong.
Mike