Is my ceph.conf file correct

trilljester

Active Member
Oct 9, 2018
13
0
41
50
I have been updating my Proxmox cluster over the past couple of weeks going from Proxmox 5.x up to it's current level of Proxmox 7.

I had some issues when going to Proxmox 7 where I basically had to rebuild my Ceph OSDs on each of my nodes because they weren't mounting properly. I basically destroyed each OSD and recreated and let the cluster balance them out and refill. I did see some documentation during the upgrades of the Ceph versions that my ceph.conf might be the cause of issues, and that being related to the keyring configuration items.

Here is my current ceph.conf. Can someone tell me if I need to update/tweek/fix any issues in there? I do recall something about remove the ceph.osd block and moving that keyring configuration item to the [global] section, is that correct? Any assistance is appreciated!

Code:
[global]
         auth client required = cephx
         auth cluster required = cephx
         auth service required = cephx
         cluster network = 172.16.0.0/24
         fsid = fa932d2b-d9cb-44cf-95af-a5411e8896d5
         mon allow pool delete = true
         mon_host = 172.16.0.2 172.16.0.3 172.16.0.4 172.16.0.5
         osd journal size = 5120
         osd pool default min size = 2
         osd pool default size = 3
         public network = 172.16.0.0/24

[client]
         keyring = /etc/pve/priv/$cluster.$name.keyring

[mds]
         keyring = /var/lib/ceph/mds/ceph-$id/keyring

[osd]
         keyring = /var/lib/ceph/osd/ceph-$id/keyring

[mds.pve-host1]
         host = pve-host1
         mds standby for name = pve

[mds.pve-host2]
         host = pve-host2
         mds standby for name = pve

[mds.pve-hos3]
         host = pve-host3
         mds standby for name = pve

[mon.pve-host2]
         host = pve-host2
         mon addr = 172.16.0.5:6789

[mon.pve-host1]
         host = pve-host1
         mon addr = 172.16.0.3:6789

[mon.pve-host3]
         host = pve-host3
         mon addr = 172.16.0.4:6789
 
I'm only just getting my head around the platform, but one thing seems strange, you have 4 IP addresses listed under [global]mon_host, but only 3 [mon.{node}] entries.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!