[SOLVED] Upgrade to Proxmox 7 - Bond (LACP) Interface not working anymore

houbidoo

Member
Mar 16, 2015
8
1
21
Hello,

Yesterday I upgraded 2 identical servers (Supermicro with Intel i350 NICs onboard) to Proxmox 7.0. The Servers are connected via 2* bond interfaces to a Juniper Switch with ae Interfaces and LACP configure). A third server with different hardware was upgraded, too.

One Server is working like normal, on the second ones the bond Interfaces do not work anymore and on the Juniper Switch I can see that the Server does not send any LACP PDUs. The third server has the same behaivour..

On server physical interfaces are up, bond Interfaces are status down.
Bridge 0 is down (IPv4 configured), Bridge 1 is up (no IP to proxmox host, only used bei VMs)

Any idea?

Config Juniper Switch:
-->> to working Proxmox Server 1

set interfaces ge-0/0/14 ether-options 802.3ad ae7
set interfaces ge-1/0/14 ether-options 802.3ad ae7
set interfaces ae7 description Host01_Kunden_Management
set interfaces ae7 aggregated-ether-options lacp active
set interfaces ae7 unit 0 description Host01_Kunden_Management
set interfaces ae7 unit 0 family ethernet-switching vlan members VLAN700

set interfaces ge-0/0/14 description Host01_Kunden_Management
set interfaces ge-0/0/14 ether-options 802.3ad ae7
set interfaces ge-1/0/14 description Host01_Kunden_Management
set interfaces ge-1/0/14 ether-options 802.3ad ae7
.....second LACP is configured the same way


Config Juniper Switch:
-->> to NOT working Proxmox Server 2

set interfaces ge-0/0/12 ether-options 802.3ad ae4
set interfaces ge-1/0/12 ether-options 802.3ad ae4
set interfaces ae4 description Host02_Kunden_Management
set interfaces ae4 aggregated-ether-options lacp active
set interfaces ae4 unit 0 description Host02_Kunden_Management
set interfaces ae4 unit 0 family ethernet-switching vlan members VLAN700

set interfaces ge-0/0/12 description Host02_Kunden_Management
set interfaces ge-0/0/12 ether-options 802.3ad ae4
set interfaces ge-1/0/12 description Host02_Kunden_Management
set interfaces ge-1/0/12 ether-options 802.3ad ae4


Network Config working Proxmox Server 1
auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto bond0
iface bond0 inet manual
bond-slaves eno1 eno2
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
#MGMT LACP

auto bond1
iface bond1 inet manual
bond-slaves eno3 eno4
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
#VM Netzwerk LACP

auto vmbr0
iface vmbr0 inet static
address 10.240.0.201/24
gateway 10.240.0.254
bridge-ports bond0
bridge-stp off
bridge-fd 0
#MGMT IP

auto vmbr1
iface vmbr1 inet manual
bridge-ports bond1
bridge-stp off
bridge-fd 0
#VM Netzwerk IP


Network Config working Proxmox Server 2
auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

auto eno3
iface eno3 inet manual

auto eno4
iface eno4 inet manual

auto bond0
iface bond0 inet manual
bond-slaves eno1 eno2
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
#LACP MGMT

auto bond1
iface bond1 inet manual
bond-slaves eno3 eno4
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
#LACP VM Netzwerk

auto vmbr0
iface vmbr0 inet static
address 10.240.0.202/24
gateway 10.240.0.254
bridge-ports bond0
bridge-stp off
bridge-fd 0
#Management

auto vmbr1
iface vmbr1 inet manual
bridge-ports bond1
bridge-stp off
bridge-fd 0
#VM Netzwerk


LACP Statistices on Juniper switch to NOT working Proxmox Interfaces (no LACP PDUs received from Proxmox host)
show lacp statistics interfaces ae4
Aggregated interface: ae4
LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx
ge-0/0/12 0 3035 0 0
ge-1/0/12 0 3006 0 0

show lacp statistics interfaces ae5
Aggregated interface: ae5
LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx
ge-0/0/13 0 3035 0 0
ge-1/0/13 0 3013 0 0

show lacp statistics interfaces ae6
Aggregated interface: ae6
LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx
ge-0/0/11 0 3351 0 0
ge-1/0/11 0 3323 0 0
 
Last edited:

houbidoo

Member
Mar 16, 2015
8
1
21
Did find the solution:

In the /etc/network/interfaces the following statement appears on the servers that were not working anymore after upgrading to 7.0

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

auto eno3
iface eno3 inet manual

auto eno4
iface eno4 inet manual

After deleting them it worked again. Checked it on both not working servers...
 
  • Like
Reactions: mickaelbeaudry

kliment.toshkov

New Member
May 27, 2021
20
0
1
42
Confirm - had the same problem and resolved it by your recommendation.
The issue is that even fresh install creates those lines, effectively rendering the machine no network access.
 

ph0x

Renowned Member
Jul 5, 2020
1,192
187
63
/dev/null
Would be interesting to find the root cause for this behavior, because here it also works with the auto statements.
 

Astraea

Member
Aug 25, 2018
113
12
23
After I upgraded my cluster from 6 to 7 I did not have any network connectivity. what ended up fixing the issues was logging into each node and running the following command

systemctl enable networking; reboot

After that, the node rebooted and came back online and everything was working again. Not sure why the networking stack got disabled during the update. This is on a 5 node cluster with multiple NICs with and without bonds that was using ifupdown2 before the upgrade.
 
  • Like
Reactions: juliokele

juliokele

Active Member
Nov 25, 2016
41
11
28
47
After I upgraded my cluster from 6 to 7 I did not have any network connectivity. what ended up fixing the issues was logging into each node and running the following command

systemctl enable networking; reboot

After that, the node rebooted and came back online and everything was working again. Not sure why the networking stack got disabled during the update. This is on a 5 node cluster with multiple NICs with and without bonds that was using ifupdown2 before the upgrade.

I removed the ifupdown package leftover configs, after reboot no network connectivity, all network interfaces are down.
After "networking" service enable and reboot, all network interfaces are up.
Maybe "networking" service not enabled by default and ifupdown2 don't working without this?
 
Last edited:

dan.ger

Member
May 13, 2019
35
4
13
Same Problem here after upgrade 6 to 7 (3x Bonds LCAP 803.ad). No ifupdown2 installed, but net-tools (ifconfig).

But I have no luck with:
Code:
systemctl enable networking; reboot

Disable auto ensxx fixed the issue, so i have to investigate...
 
Last edited:

dan.ger

Member
May 13, 2019
35
4
13
Same Problem here after upgrade 6 to 7 (3x Bonds LCAP 803.ad). No ifupdown2 installed, but net-tools (ifconfig).

But I have no luck with:
Code:
systemctl enable networking; reboot

Disable auto ensxx fixed the issue, so i have to investigate...
After installing ifupdown2 everything works fine.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!