[SOLVED] Getting LACP to work on Proxmox 8.1

[mrzippy]

Hello All,

I'm trying to get LACP working on a bridge interface.

Some background:

The bridge interface comprises 1 bond: 2 10g SFP connections to a Cisco 2960-X switch. Those two ports are configured as trunks in a channel-group, LACP.
Port config from switch here:

description ProxMox Trunk r330
switchport trunk allowed vlan 1,10,20,30,66,99
switchport mode trunk
channel-group 1 mode active

I have the SFP+ connected to a card on my server, enp1s0f0, and enp1s0f1

Below is my current config from /etc/network/interfaces:


auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual
mtu 9000

auto eno2
iface eno2 inet manual
mtu 9000

auto enp1s0f0
iface enp1s0f0 inet manual
mtu 9000
hwaddress ether d0:bf:9c:02:98:98

auto enp1s0f1
iface enp1s0f1 inet manual
mtu 9000
hwaddress ether d0:bf:9c:02:98:9c

auto enp2s0f0
iface enp2s0f0 inet manual
mtu 9000
hwaddress ether a0:36:9f:4b:bc:d4

auto enp2s0f1
iface enp2s0f1 inet manual
mtu 9000
hwaddress ether a0:36:9f:4b:bc:d6

auto bond0
iface bond0 inet manual
bond-slaves enp1s0f0 enp1s0f1
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
mtu 9000
hwaddress ether 00:00:00:02:98:9c
#10GB Trunks

auto bond1
iface bond1 inet manual
bond-slaves enp2s0f0 enp2s0f1
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer3+4
hwaddress ether 00:00:00:4b:dc:b5
#HA 10GB Copper

auto vmbr0
iface vmbr0 inet static
address 10.99.99.200/24
gateway 10.99.99.1
bridge-ports eno1 eno2
bridge-stp off
bridge-fd 0
mtu 9000
#MGMT interface

auto vmbr1
iface vmbr1 inet manual
bridge-ports bond0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
mtu 9000
hwaddress ether 00:00:00:02:98:99
#VM Networks

auto vmbr2
iface vmbr2 inet static
address 10.99.100.4/24
bridge-ports bond1
bridge-stp off
bridge-fd 0
mtu 9000
#HA Cluster


If I shut down one of the interfaces in Bond0, all things work fine, my vms can be assigned to different vlans, all is well. But if I enable the second link in the group, then I get a loop error showing on the console and I lose connectivity to some hosts etc...

Any suggestions on where next to go?

 

[mrzippy]

I should note: I put in "dummy" hwaddress ether on vmbr1 and Bond0... but the hwaddress on the various interfaces match their actual MAC addressses.

 

[mrzippy]

So... I stumbled across a post on Reddit, and i think the solution has been found: On the bond interface members.. i needed to add a line to them that declared their bond-master. So, using the config posted above:

auto enp1s0f0
iface enp1s0f0 inet manual
mtu 9000
bond-master bond0
hwaddress ether d0:bf:9c:02:98:98

auto enp1s0f1
iface enp1s0f1 inet manual
mtu 9000
bond-master bond0
hwaddress ether d0:bf:9c:02:98:9c


And then restarting the network stack (note: an easy way to do it without a full system reboot is to edit /etc/interfaces/network, add the relevant lines, save. Then go into the GUI, and add / edit a comment on any interface. Apply Configuration, and it will reload).

This has my LAG running correctly, speeds showing as the aggregated link speed for the vmbr interface. Don't know if I needed to leave the custom HW addresses in for the interfaces, but it does no harm at this juncture.

This may be a bug / feature request to put in, where this bond-master line can be added via GUI when creating LACP (another line in the popup) or automatically so when a bond interface is created, and slaves added... it then goes back and adds the bond-master etc to the interfaces. Either way, if this saves someone else some frustration, I'm glad.

Cheers.

 

[wahmed]

The bond-master is not required to get LACP working. The syntax bond-slaves <primary-interface> <secondary-interface> is enough. Bond-master is old way of doing things. Adding it to the GUI would be going backward. If you remove or comment out the bond-master then reboot, I believe you will see everything working without it.
The hwaddresses also are not required unless your requirement calls for it.

 

[mrzippy]

That's a nice thought.. but it did not work without out the bond-master statements. I tried without and even after reboots, would get a loop message on the console. It may be old.. but it's working and stable. Following the wiki / reccommended setup did not allow the LACP group to work.