full mesh frr network for ceph

[telvenes]

Hello,

Been running full mesh frr network for my ceph and it works great. But im struggeling with getting access from vms, is there someone else here that have been able to solve this?

 

[aaron]

Yeah well, if the VMs need to get direct access to the Ceph Public network, which is the full-mesh routed (frr) one, that's going to be tricky :-/
A switched network would be a lot simpler for that scenario.

 

[telvenes]

thanks for your reply.

I have solved this temproary adding a route ceph cluster ip via management ip. but this limits the whole setup trough the very slow management network. was really hoping someone had a solution for it

 

[telvenes]

I have also found this that sounds to be something simular what I ask of but I really do not understand how this solves it:

G

[TUTORIAL] Thread 'Full mesh (routed setup) + EVPN + VM Bridge'

Jun 10, 2024

At Geco-it, we use the SDS Linstor solution.

To connect our hypervisors without investing in switches we use a full mesh network

To use Linstor storage in VMs (container volumes), we need access to the Linstor satellite network.

So we need a VM Storage bridge...

To setup full mesh network, you must follow:

• Full Mesh Network for Ceph Server
• [Full mesh (routed setup) + EVPN] it is feasible even by using SDN!

Infrastructure​

 +------------------+    +------------------+    +------------------+
 |     Node 1       |    |     Node 2       |    |...

• Geco-iT
• mesh sdn
• Replies: 8
• Forum: Proxmox VE: Networking and Firewall

 

[telvenes]

what about installing frr on on the vm and let it be part of the announcment if we example add a vmbridge into the frr for the hosts?