Ceph with Open VSwitch

[ednt]

We want to setup a Cluster with 6 Ceph Nodes splitet in 2 Racks.

In the pased we have used Linux Bridge and Bond to build the Network for Ceph 2x 10 GB each Node with 2 VLAN
for Access and Sync for Ceph.

the Question is:
is it Recommended to built this hypervisor with Ceph Network used Open V Switch and ist there expierience with stabilitity on productiv system.

In the target Configuration we want to have the 6 Ceph Nodes that are also used as hypervisor and 8 Nodes dedicated
as hypervisor all Units connected with 2x10GB Bond to the core Switch.

 

[spirit]

I have seen some bugreport with openvswitch on proxmox6, I can confirm it's really stable.

Do you have a special need for using ovs ? I personnaly prefer linux bridge in production. (Never problem, no fear of ovs userland daemon crash...)

 

[ednt]

Target is to split customer that vms from one customer can´t see other customers.

Cluster crow and demands always more complicated from project to project
and we want a maximum on open possibilities with this installation.

 

[Alwin]

This is more a topic of seperation of concerns and independent of the use of Ceph. You can use multiple bridges that are vlan-aware and you have the full linux stack available. More concerning is, judging from your provided information, that Ceph traffic (and possible corosync) will run on the same 2x10 GbE as the customer traffic. This will definitely create interference. Ranging from performance loss to node resets (HA active).

Separate corosync traffic and storage traffic onto their own physical network, so the will not interfere with nor disrupted by other network traffic.