[SOLVED] Ceph MON Service Not Starting on pve2 and pve3 in a Virtualized Proxmox Cluster

[shbaek]

pve version: 8.3.4
ceph version: 19.2

I have only one physical network interface on my physical Proxmox server, so I don't have the opportunity to configure separate Ceph cluster and Ceph public networks.

To work around this, I created virtual networks using Linux bridges on the physical server and assigned these bridges to VMs, which I set up as pve1, pve2, and pve3.

After that, I configured bond0 on these VMs and created Linux VLANs based on this bond.

• The Ceph public network is configured as a Linux VLAN with IP range 10.10.50.11/24 to 10.10.50.13/24 using VLAN ID 50.
• The Ceph cluster network is configured as a Linux VLAN with IP range 10.10.51.11/24 to 10.10.51.13/24 using VLAN ID 51.

However, after installing Ceph on all nodes, the MON service runs properly on pve1, but on pve2 and pve3, it gets added but does not start.

What could be the reason for this issue?


->Real physical node network


->virtual pve network linux vlan



->virtual pve network linux vlan

 

[shbaek]

I used to allocate mtu to 9000, but when I removed all of them, it worked normally.