Noticed this after my second proxmox node lost power. On reboot the mon did not want to come up. Upon investigation it was due to the mon directory being owned by root and not ceph.
What I am seeing is when the monitor on pvehost2 (the system that had file permissions issues) is up. The OSD map appears to always be correct and all data appears to be available. However, under the proxmox UI a majority of the OSDs show as bluestore OR filestore intermittently. All OSDs in this cluster are bluestore. The one that does not change osd.18 was added after the power outage while the system was down but before the failed mon was noticed.
I am not sure what it is doing, but every time it comes up another monitor is elected but they all vote for that one anyway.
Any ideas? Right now I am probably going to just fail the monitor and reinitialize it to get 3 monitors in quorum again.
What I am seeing is when the monitor on pvehost2 (the system that had file permissions issues) is up. The OSD map appears to always be correct and all data appears to be available. However, under the proxmox UI a majority of the OSDs show as bluestore OR filestore intermittently. All OSDs in this cluster are bluestore. The one that does not change osd.18 was added after the power outage while the system was down but before the failed mon was noticed.
I am not sure what it is doing, but every time it comes up another monitor is elected but they all vote for that one anyway.
Any ideas? Right now I am probably going to just fail the monitor and reinitialize it to get 3 monitors in quorum again.
Last edited: