Is my ceph.conf file correct

trilljester

Active Member
Oct 9, 2018
13
0
41
50
I have been updating my Proxmox cluster over the past couple of weeks going from Proxmox 5.x up to it's current level of Proxmox 7.

I had some issues when going to Proxmox 7 where I basically had to rebuild my Ceph OSDs on each of my nodes because they weren't mounting properly. I basically destroyed each OSD and recreated and let the cluster balance them out and refill. I did see some documentation during the upgrades of the Ceph versions that my ceph.conf might be the cause of issues, and that being related to the keyring configuration items.

Here is my current ceph.conf. Can someone tell me if I need to update/tweek/fix any issues in there? I do recall something about remove the ceph.osd block and moving that keyring configuration item to the [global] section, is that correct? Any assistance is appreciated!

Code:
[global]
         auth client required = cephx
         auth cluster required = cephx
         auth service required = cephx
         cluster network = 172.16.0.0/24
         fsid = fa932d2b-d9cb-44cf-95af-a5411e8896d5
         mon allow pool delete = true
         mon_host = 172.16.0.2 172.16.0.3 172.16.0.4 172.16.0.5
         osd journal size = 5120
         osd pool default min size = 2
         osd pool default size = 3
         public network = 172.16.0.0/24

[client]
         keyring = /etc/pve/priv/$cluster.$name.keyring

[mds]
         keyring = /var/lib/ceph/mds/ceph-$id/keyring

[osd]
         keyring = /var/lib/ceph/osd/ceph-$id/keyring

[mds.pve-host1]
         host = pve-host1
         mds standby for name = pve

[mds.pve-host2]
         host = pve-host2
         mds standby for name = pve

[mds.pve-hos3]
         host = pve-host3
         mds standby for name = pve

[mon.pve-host2]
         host = pve-host2
         mon addr = 172.16.0.5:6789

[mon.pve-host1]
         host = pve-host1
         mon addr = 172.16.0.3:6789

[mon.pve-host3]
         host = pve-host3
         mon addr = 172.16.0.4:6789
 
I'm only just getting my head around the platform, but one thing seems strange, you have 4 IP addresses listed under [global]mon_host, but only 3 [mon.{node}] entries.