Corosync best network options

[Johnnr]

I have 3 nodes, and lets assume I have unlimited resources
what is the best way to set up a redundant corosync network?

So far I have tried a point to point mesh network using 2 ports on each device like below



I have also set up the Corosync network through 2 switches like below:

My question is what are the pros and cons of each configuration, both offer the same redundancy. Will I have latency issues with the mesh network? or is it an advantage as I don't have the running costs of 2 extra switches?

This question might have already been asked but i have had a search and can't find anything this specific

 

[Robin C.]

I would prefer the second option with switches.

a) it's more scalable (e.g. if you add more nodes)
b) if the right link of Host3 goes down, it'll have to go through Host2 to reach Host1. This is more of a pain to set up.


In practice... It probably won't matter a lot. But you said "unlimited resources", so I'd recommend option 2.

But if you wanted to do it really well, you would add a link between the switches (MLAG / EVPN-MH / Virtual Chassis / whatever - please don't just connect the switches though, you might get not-so-nice loops depending on your setup). This way, you get even more redundancy...

 

[MarkusKo]

for corosync you should configure 2 dedicated network interfaces on each host, no LAG, no bridge, different subnets.
connect each host to each switch. corosync only needs 1Gbit so you could buy cheap unmanaged desktop switches just for that.
or create a vlan on each switch for the corosync network.

don't bother with direct connections between hosts for corosync. that's more of an option for redundant storage / migration network or if you use ceph (don't use ceph if you only have 3 hosts) so you could use 25Gbit++ without an expensive switch.

docs