we are working on a small Proxmox enviroment with 3 nodes and mesh network.
All servers have
2x 10GB copper in mesh for Ceph cluster network
2x 1GB copper in mesh (corosync)
2x 1GB free for exiting network to customer
2x10GB SFP+ not used yet. (might be used for VM traffic and Ceph public subnet but we test it with the 2 free 1GB interfaces first.)
I already used the https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server#Routed_Setup_.28simple.29 routed with fallback.
I can ping all nodes trough the mesh but Proxmox seems to ignore this mesh completely since it does not have any 'interfaces'. So building a cluster fails since proxmox and corosync are not aware of the mesh.
Of course we can fall back to the RSTP way but then all one nodes has double the traffic passing trough to the 3th node.
Yes we are aware that running Proxmox and Ceph on the same servers is not ideal, but it is only for 2 windows enviroments with minimal troughput.
All 3 servers have 2x 500GB SSD zfs Raid1 and 3x 1TB SSD for Ceph OSD's and a load of CPU and memory power.
Is there anyone who can help with the correct methods for configuring the meshes?
All servers have
2x 10GB copper in mesh for Ceph cluster network
2x 1GB copper in mesh (corosync)
2x 1GB free for exiting network to customer
2x10GB SFP+ not used yet. (might be used for VM traffic and Ceph public subnet but we test it with the 2 free 1GB interfaces first.)
I already used the https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server#Routed_Setup_.28simple.29 routed with fallback.
I can ping all nodes trough the mesh but Proxmox seems to ignore this mesh completely since it does not have any 'interfaces'. So building a cluster fails since proxmox and corosync are not aware of the mesh.
Of course we can fall back to the RSTP way but then all one nodes has double the traffic passing trough to the 3th node.
Yes we are aware that running Proxmox and Ceph on the same servers is not ideal, but it is only for 2 windows enviroments with minimal troughput.
All 3 servers have 2x 500GB SSD zfs Raid1 and 3x 1TB SSD for Ceph OSD's and a load of CPU and memory power.
Is there anyone who can help with the correct methods for configuring the meshes?