Ceph on a VLAN

[TwiX]

Hi, I usually use 2 x 2x10Gb network cards like this :

vmbr0 : LACP bond0 with 2x10Gb ports
ceph (pub/priv) : LACP bond1 with 2x10Gb ports

So we may say that there are 2 dedicated interfaces, one for vms, the other for ceph.
But it requires 4 ports on my switch.

I'm wondering if it could be possible to use Ceph on a VLAN, using a 2x25 Gb card instead (so 50Gb with ceph vlan versus 40 Gb). All services running on a 2x25Gb lacp link.
That's what Dell vxrail do (vlan for vsan, vlan for vmotion, vlans for vms,...)

What do you think ?

Thanks !

Antoine

 

[aaron]

Unless you can make sure with QOS that certain VLANs or packet types have a guaranteed bandwidth / latency you will sooner or later run into problems.

The big issue is if Corosync is on the same physical network and if you have HA guests in the cluster. Worst case scenario is that Ceph is using all the bandwidth which means that the latency is going up for other services. If a node is losing contact to the cluster because the latency for corosync packets is too high, and if you have HA enabled guests on it, the watchdog will become active and after 2 minutes without the cluster the node will fence itself (hard reset).

When used in such a setup and such a situation occurs, it usually affects all nodes and thus it appears as if the whole cluster restarted unexpectedly.

 

[TwiX]

Hi, thanks

Understood.