Question about LACP and stacked switches

[kev974]

Hi everybody,

Im getting confused about LACP bonding and stacked switches (lets say switch 1 and 2).

I have 2 Dell powerconnect 8132F switches stacked together.

I configured LACP bonding on my hosts as :

iface bond0 inet manual
slaves enp5s0f0 enp6s0f0
bond_miimon 100
bond_mode 802.3ad

Each host has one nic connected to both switches:enp5s0f0 to switch 1 and enp6s0f0 to switch 2

So everything seems to work corectly,if i poweroff one switch traffic still goes into the other switch,etc.

My question is why is it working without having configured LACP in my switches?
Should I configure LACP on them by putting for example port 19 to channel-group 1 and putting it in trunk mode?
Does they switch port connected to enp5s0f0 and enp6s0f0 belong to LACP?

What's more we are planning to connect this cluster to another one with two other stacked identical switches (let's say switch 3 and 4) by 2 dark fiber (SFP+).
To connect stack1 with stack2 should we use LACP by just putting for exemple :

port 20 in Switch 1 connected to port 20 Switch 3
port 20 inf Switch 2 connected to port 20 Switch 4

and all ports in channel-group 1 or 2?

Thank for your answers.

 

[curtisb]

I cannot tell you why it is working if you know for a fact the 8132F stack doesn't have those switchport in a LACP-enabled trunk. I would go ahead and set the ports to a channel group and enabled LACP. Example below:

console(config)# interface gigabitethernet 1/0/19
console(config-if-1/0/19)# channel-group 1 mode active​

The option 'active' is what makes it LACP and enabled the LACP PDUs to be sent to the receiving end (and vice-versa).

You would repeat that for the inter-switch link between your two stacks. Just make sure to use a different channel-group ID.

PM your running config if you have more problems, happy to put that old knowledge to work.

--Curtis

 

[kev974]

I cannot tell you why it is working if you know for a fact the 8132F stack doesn't have those switchport in a LACP-enabled trunk. I would go ahead and set the ports to a channel group and enabled LACP. Example below:

console(config)# interface gigabitethernet 1/0/19
console(config-if-1/0/19)# channel-group 1 mode active​

The option 'active' is what makes it LACP and enabled the LACP PDUs to be sent to the receiving end (and vice-versa).

You would repeat that for the inter-switch link between your two stacks. Just make sure to use a different channel-group ID.

PM your running config if you have more problems, happy to put that old knowledge to work.

--Curtis

Thank you for your answer Curtis,

I'm going to put LACP on my switches but should i put every ports connnected to my hosts (enp5s0f0 and enp6s0f0) in channel-group 1 ,in that case my channel group 1 would have Te 1/0/4-6 and Te 2/0/4-6

Or should i just put one port between the two switches in the same stack in port-channel 1,in that case channel-group 1 should have Te 1/0/19 and Te 2/0/19?

Or both in that case channel-group 1 would have Te 1/0/4-6,Te 2/0/4-6,Te 1/0-19,Te 2/0/19

For the inter-stack connection i should put another port (for example Te 1/0/20 and Te 2/0/20) on both stack and put them in Channel-group 2?

Thank you in advance

 

[curtisb]

Each host needs a separate channel group. That channel group would comprise ports from both switches (e.g. te1/0/4 and te2/0/4).

If you have two hosts currently, you'll have two separate channel groups facing your hosts. Another way to think of it is if you have a bond with two slave interfaces, your channel group will also have two interfaces.

To the inter-stack connection, you are correct.

In the end, you'll have n+1 channel groups per stack, where n is the number of hosts connected to that stack.

 

[kev974]

Thank you very much Curtis,

So i was not doing LACP correcly since the begining...

My configuration should be :

SW1 te 1/0/23-24 -> SW 2 te 2/0/23-24 = Stacking ports

For inter-stack connexion :

SW1 te 1/0/22 -> SW3 te 1/0/22 in channel-group 11
SW2 te 2/0/22 -> SW4 te 2/0/22 in channel-group 11

For hosts :

SW1 te 1/0/4 -> Host 1 enp5s0f0 in channel group 4
SW2 te 2/0/4-> host 1 enp6s0f0 in channel group 4

SW 1 te 1/0/5 -> host 2 enp5s0f0 in channel group 5
SW 2 te 2/0/5 -> host 2 enp5s0f0 in channel group 5

SW 1 te 1/0/6 -> host 3 enp5s0f0 in channel group 6
SW 2 te 2/0/6 -> host 3 enp5s0f0 in channel group 6

Then put Po4,5,6 in switchport mode general + switchport general allowed vlan 1,23,198 (corresponding in my vmbr1,vmbr23,vmbr198 on top of bond0.1,bond0.23,bond0.198)

Is it correct?

And to understand what i'm doing wrong until now with non-stacked swtich i have put one cable between SW1 -> SW2 and put them in channel-group 1 and that's all...and redundancy seems to work...

Should i put

SW1 gi 23-24 -> SW2 gi 23-24 in channel-group 11

SW1 gi 1/0/4 -> Host 1 enp5s0f0 in channel group 4
SW2 gi 2/0/4-> host 1 enp6s0f0 in channel group 4

SW 1 gi 1/0/5 -> host 2 enp5s0f0 in channel group 5
SW 2 gi 2/0/5 -> host 2 enp5s0f0 in channel group 5

SW 1 gi 1/0/6 -> host 3 enp5s0f0 in channel group 6
SW 2 gi 2/0/6 -> host 3 enp5s0f0 in channel group 6

Is that what i shoud do?

Thank you in advance

 

[curtisb]

Ah, now I better understand. SW1 & SW2 have stacking ports designated as 23-24 on both switches. When enabled as a stack port the designated interfaces switch from Ethernet protocols to Broadcom priority tech called Higig. It is part of the ASIC in that switch. These interfaces don't need to be in a channel group, part of the Higig spec is to make a single control plane across both stacked switches. For all intended purposes sw1 and sw2 should be viewed logically as a single switch. That is why redundancy was working already.

So no, you cannot add those stack interfaces to a port channel even if you wanted to. They no longer speak that language and the switch will prevent you from doing it.

Did I miss your question?

 

[kev974]

Ah, now I better understand. SW1 & SW2 have stacking ports designated as 23-24 on both switches. When enabled as a stack port the designated interfaces switch from Ethernet protocols to Broadcom priority tech called Higig. It is part of the ASIC in that switch. These interfaces don't need to be in a channel group, part of the Higig spec is to make a single control plane across both stacked switches. For all intended purposes sw1 and sw2 should be viewed logically as a single switch. That is why redundancy was working already.

So no, you cannot add those stack interfaces to a port channel even if you wanted to. They no longer speak that language and the switch will prevent you from doing it.

Did I miss your question?

If SW1 and SW2 are stacked with port 23-24 should i even put hosts ports in separated channel-group like this:


SW1 te 1/0/4 -> Host 1 enp5s0f0 in channel group 4
SW2 te 2/0/4-> host 1 enp6s0f0 in channel group 4

SW 1 te 1/0/5 -> host 2 enp5s0f0 in channel group 5
SW 2 te 2/0/5 -> host 2 enp5s0f0 in channel group 5

SW 1 te 1/0/6 -> host 3 enp5s0f0 in channel group 6
SW 2 te 2/0/6 -> host 3 enp5s0f0 in channel group 6

For inter-stack connexion :

SW1 te 1/0/22 -> SW3 te 1/0/22 in channel-group 11
SW2 te 2/0/22 -> SW4 te 2/0/22 in channel-group 11?

nd to understand what i'm doing wrong until now with non-stacked swtich i have put one cable between SW1 -> SW2 and put them in channel-group 1 and that's all...and redundancy seems to work...

Should i put

SW1 gi 23-24 -> SW2 gi 23-24 in channel-group 11

SW1 gi 1/0/4 -> Host 1 enp5s0f0 in channel group 4
SW2 gi 2/0/4-> host 1 enp6s0f0 in channel group 4

SW 1 gi 1/0/5 -> host 2 enp5s0f0 in channel group 5
SW 2 gi 2/0/5 -> host 2 enp5s0f0 in channel group 5

SW 1 gi 1/0/6 -> host 3 enp5s0f0 in channel group 6
SW 2 gi 2/0/6 -> host 3 enp5s0f0 in channel group 6

In this question this is for two others SW,the SWs are not stacked. They have just Two cable in channel-group 11 with LACP.

 

[curtisb]

If SW1 and SW2 are stacked with port 23-24 should i even put hosts ports in separated channel-group like this:


SW1 te 1/0/4 -> Host 1 enp5s0f0 in channel group 4
SW2 te 2/0/4-> host 1 enp6s0f0 in channel group 4

SW 1 te 1/0/5 -> host 2 enp5s0f0 in channel group 5
SW 2 te 2/0/5 -> host 2 enp5s0f0 in channel group 5

SW 1 te 1/0/6 -> host 3 enp5s0f0 in channel group 6
SW 2 te 2/0/6 -> host 3 enp5s0f0 in channel group 6

You have to put them in separate channel groups, each represents a single server. What you have here is 100% correct.

For inter-stack connexion :

SW1 te 1/0/22 -> SW3 te 1/0/22 in channel-group 11
SW2 te 2/0/22 -> SW4 te 2/0/22 in channel-group 11?

This is correct, both links are part of the same switch so they need to be in the same channel-group for hashing to work correctly. The channel group number is only locally significant, you could use different channel group numbers of each stack and things would work fine. Most folks consider it a best practice to use the same number on both sides to keep life simple.

In this question this is for two others SW,the SWs are not stacked. They have just Two cable in channel-group 11 with LACP.

Ah gotcha, you are just going to move that to be the connection between the two switch stacks. Sounds like you got the concept down.

 

[kev974]

You have to put them in separate channel groups, each represents a single server. What you have here is 100% correct.


This is correct, both links are part of the same switch so they need to be in the same channel-group for hashing to work correctly. The channel group number is only locally significant, you could use different channel group numbers of each stack and things would work fine. Most folks consider it a best practice to use the same number on both sides to keep life simple.


Ah gotcha, you are just going to move that to be the connection between the two switch stacks. Sounds like you got the concept down.

Thank you very much for your answers Curtis!Glad to understand what i am doing!

Really appreciate your help