Proxmox LACP bonding and TP-Link Managed Switch

[Lyve]

I want to connect my Proxmox host to my TP-Link switch via two nics as LACP LAG and configured Proxmox like that:

auto lo
iface lo inet loopback

auto enP3p49s0
iface enP3p49s0 inet manual

auto enP4p65s0
iface enP4p65s0 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves enP3p49s0 enP4p65s0
        bond-miimon 100
        bond-mode 802.3ad
        bond-xmit-hash-policy layer2
#Switch Ports 5 6

auto vmbr0
iface vmbr0 inet static
        address 192.168.20.33/24
        gateway 192.168.20.1
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0

I did that via the GUI:


However, whenever I configure the LAG on my TP-Link Switch, I cannot reach Proxmox anymore. This only works when LACP for the two ports is disabled.

I'm wondering about two things:

Why can I even reach Proxmox as long as I did not configure my switch to use LACP LAG?
Why can't I reach Proxmox anymore if I enable the LAG?

This how I configured LAG on the switch:


Only if I remove this configuration, I can reach the Proxmox host. Is it possible that LAG on the Proxmox host is not active, even though I configured it via the GUI to do so? I already restarted the Proxmox host already multiple times, no change.

This is the output of ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enP4p65s0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
    link/ether c0:74:2b:ff:5c:3b brd ff:ff:ff:ff:ff:ff permaddr c0:74:2b:ff:5c:3a
3: enP3p49s0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
    link/ether c0:74:2b:ff:5c:3b brd ff:ff:ff:ff:ff:ff
4: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether c0:74:2b:ff:5c:3b brd ff:ff:ff:ff:ff:ff
5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether c0:74:2b:ff:5c:3b brd ff:ff:ff:ff:ff:ff
    inet 192.168.20.33/24 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::c274:2bff:feff:5c3b/64 scope link
       valid_lft forever preferred_lft forever

Any help is greatly appreciated.

 

[B.Otto]

Hello,

you could try setting the LACP-Mode to 'Active' in your switches. Someone has to start sending LACPDUs and I can't remember whats the default setting in Linux/PVE.

Kind regards,
Benedikt

 

[Lyve]

you could try setting the LACP-Mode to 'Active' in your switches. Someone has to start sending LACPDUs and I can't remember whats the default setting in Linux/PVE.

Thanks a lot, this did not help unfortunately, but for others with this problem, I figured it out:

Be sure to check your port isolation settings The LAG needs to be added to the port forwarding list, a newly created LAG on a TP-Link switch will not automatically inherit the settings from the individual ports if not part of a LAG

 

[dacurly]

Thanks a lot, this did not help unfortunately, but for others with this problem, I figured it out:

Be sure to check your port isolation settings The LAG needs to be added to the port forwarding list, a newly created LAG on a TP-Link switch will not automatically inherit the settings from the individual ports if not part of a LAG

Hi Lyve,

I have the same issue, could you please share the port isolations settings that need to be changed to the LAG with Proxmox works?

Thanks.

 

[Lyve]

Hi Lyve,

I have the same issue, could you please share the port isolations settings that need to be changed to the LAG with Proxmox works?

Thanks.

You need to go to L2 Features / Port / Port Isolation.



Then press Edit at the top right, select your LAG ports at the top section and select all ports it should be able to communicate with at the bottom.


Best,

Lyve

 

[dacurly]

You need to go to L2 Features / Port / Port Isolation.

View attachment 57489

Then press Edit at the top right, select your LAG ports at the top section and select all ports it should be able to communicate with at the bottom.
View attachment 57490

Best,

Lyve

Thank you, Lyve!
Your reply was immensely helpful in solving my LAG configuration issue.
Your support and expertise are much appreciated!

 

[umask022]

@Lyve did you set anything else to have it working?
I have similar case and unfortunately i'm not able to deal with it.

auto eno8303
iface eno8303 inet manual

auto eno8403
iface eno8403 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno8303 eno8403
        bond-miimon 100
        bond-mode 802.3ad
        bond-xmit-hash-policy layer2

auto vmbr0
iface vmbr0 inet manual
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

auto vmbr0.100
iface vmbr0.100 inet static
        address 10.0.0.10/24
        gateway 10.0.0.254

auto vmbr0.200
iface vmbr0.200 inet manual

I have 2 TL-SG3428 connected with LACP already (2 ports, several VLANs).
I'm creating LACP to test config only for one port, then i'm enabling VLANs (in the same way like for previous LAG) and connectivity is interrupting, Proxmox is not able to connect anywhere. It's not Port isolation issue as i can see entry there - seems like it's not a bug on TL-SG3428 switch.

Any idea how to deal with it?

 

[Lyve]

@Lyve did you set anything else to have it working?
I have similar case and unfortunately i'm not able to deal with it.

Differences are very small, this is my interfaces config:

auto lo
iface lo inet loopback

auto enp4s0
iface enp4s0 inet manual

auto enp1s0
iface enp1s0 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves enp1s0 enp4s0
        bond-miimon 100
        bond-mode 802.3ad
        bond-xmit-hash-policy layer2
        mtu 9000

auto vmbr0
iface vmbr0 inet static
        address 192.168.20.7/24
        gateway 192.168.20.1
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
        mtu 9000

What I notice is that you do not have a static IP on VLAN 0 but instead only for VLAN 100, did you enable tagged VLAN for the bridged port on your TP Link Switch?

 

[umask022]

Yes, i have config like that:
VLAN1 untagged
VLAN100 tagged
VLAN200 tagged

Other difference is MTU but i don't think it's an issue here.

Everything is working fine but when i'm selecting 2 ports and enabling LCAP (creatig LAG2) Proxmox is not responding anymore.
I've also added LAG2 to all VLANs (1 untagged, rest tagged) and nothing... When i remove LAG from switch all is going back to normal and Proxmox is responding.

Initial config was without bond0 (vmbr0 with eno8303 and all vlans like it's configured currently) but i want to increase bandwidth a little bit.

Interesting is that i've succesfully connected 2 TP-Link switches with LAG1 LACP by creating LAG and assigning VLANS to it - now i'm trying to do the same for Proxmox but for now without luck.

EDIT:
@Lyve
I've configured static lag in the same way like i did for LACP between switches and all is working fine...


EDIT2:
Finally! It's working fine. Problem was caused by switch firmware version. After upgrade LACP is working file - like LAG config or LACP between 2 switches.