CEPH Network with 2 Switches

[Younex]

Hi,

i have 2 infiniband switches where i have one switch active for ceph traffic.

The aim is to use the second switch either for failover or perfomance (or both whatever is possible with 2 switches acting independently and not trunked together).

The problem is i can´t truncate both switches so i need a different solution instead of active-backup method with a bond.

Maybe if i use broadcast or any other mode (balance-tlb...)?

 

[Daniel S.]

how about lacp? if the switches support it?

 

[Younex]

No, i have 2 voltaire 4036 switch which can´t be configured for this as far as i know.

 

[Alwin]

The problem is i can´t truncate both switches so i need a different solution instead of active-backup method with a bond.

I don't quiet understand, the active-backup should work, at least by a quick search. But I do not know, how Ceph will react to it on EoIB.
https://community.mellanox.com/s/ar...-bond--lag--interface-over-infiniband-network

 

[sb-jw]

with 2 switches acting independently and not trunked together

the active-backup should work

@Alwin that's the problem here. If one server has an active link on the other switch, they unable to communicate with the rest, when no link between the switches are existing.

 

[Alwin]

@Alwin that's the problem here. If one server has an active link on the other switch, they unable to communicate with the rest, when no link between the switches are existing.

Ture, I did not clarify my thought on this. Because in the bond settings you can define the primary slave, so when both links are up, the bond uses that primary. So if the switch dies, then all would fail-over. If only that link dies, then OFC it wouldn't work.

Search for 'primary', to see the explanation.
https://www.kernel.org/doc/Documentation/networking/bonding.txt