Network not coming up after Upgrade 8.0 to 8.1 caused by ifupdown2 and *dpkg-old files

[Andreas Pflug]

I have a cluster of some machines, each having two bonded interfaces (and, luckily, a simple dedicated maintenance interface).
After dist-upgrading from 8.0.4 to 8.1.4, which upgrades also ifupdown to ifupdown2, the bonded interface get disabled immediately after enabling bonding, disabling all traffic.

The /etc/network/interfaces config looks like this:

auto enp33s0f0
iface enp33s0f0 inet manual
        up ifconfig enp33s0f0 promisc up

auto enp33s0f1
iface enp33s0f1 inet manual
        up ifconfig enp33s0f1 promisc up
auto bond0

iface bond0 inet manual
        bond-slaves enp33s0f0 enp33s0f1
        bond-miimon 100
        bond-mode 802.3ad
        bond-lacp_rate fast

When performing ifdown enp33s0f0 twice after reboot, the first will execute silently (i.e. ifdown knows the interface and shuts it down), the second fails with "cannot find interfaces: enp33s0f0 (interface was probably never up ?)". So apparently networking did bring up the interface initially.

So I did:

ifdown bond1
ifdown enp33s0f0
ifdown enp33s0f1
ifup -a

And kern.log shows

bond0: (slave enp33s0f0): link status definitely up, 10000 Mbps full duplex
bond0: (slave enp33s0f1): link status definitely up, 10000 Mbps full duplex
bond0: active interface up!
bond0: (slave enp33s0f0): link status definitely down, disabling slave
bond0: active interface up!
bond0: (slave enp33s0f1): link status definitely down, disabling slave
bond0: now running without any active interface!

resulting in failing traffic (ethtool shows no link).

To regain networking, I have to manually enable the interfaces using ifconfig or ifup (yes, the latter will work also)

What's going on here, how can I fix it?

 

[bbgeek17]

Its been my experience that the slave interfaces must not be up for the bond to come up. In fact on some Linux OS's the bond wont come up if slave interface is up.
I'd recommend that you try to remove/comment the enp* configuration and dont try to manually manage the underlying interfaces.

Good luck

PS
you may find this interesting https://www.claudiokuenzler.com/blog/1121/debian-11-bullseye-problem-bond-bonding-lacp-interfaces

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Andreas Pflug]

I already tried that, same result with all "interface enp*" removed.

 

[bbgeek17]

I already tried that, same result with all "interface enp*" removed.

You are asking for help, but not providing much information to work with. An updated config with corresponding boot sequence log, followed up status command outputs would help community members to better understand your situation. Many times a poster would come here with sparse details and after two pages of prodding it would turn out that a simple "ip a" was holding the answer to the problem.

In the meantime, I'd recommend playing around with options described here: https://wiki.linuxfoundation.org/networking/bonding#:~:text=The parameters are as follows:
ie up_delay, use_carrier, etc. Try to boot with bond interface set to not-auto-enable, or config files completely moved away. Your goal is to understand whether the init of the bond is happening too early (for whatever reason) and needs to be delayed.
Ie if you boot without any bond config, then move the files in-place and "ifup" - does everything just work?

Good luck

PS also do review the link and corresponding bugzilla. It could be a bug thats not been fixed or the fix is not incorporated yet into PVE packaging.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Andreas Pflug]

Having debugged already a bit, I was quite sure the problem is caused by ifupdown2, posting this thread in the hope somebody recognises the issue.

As you suggested, I removed all auto <iface> on bond and vmbr, and rebooted. The the enp interfaces did NOT come up.
Then, <ICODE>ifup enp33s0f0</ICODE> did the job: interface up as expected.
Next, <ICODE> ifup bond0</ICODE> would bring down enp33s0fo again.

I uninstalled ifupdown2 (together with pve-manager...) and installed the ancient ifupdown, reboot and voila, the original interfaces definition with all interfaces to auto-start worked as expected: everything configured and up.

Accordingly, I updated the title of this thread: the problem is the ifupdown2 package, doing unexpected stuff.

There are exactly zero prompt messages when executing ifup, and the kern.log messages are those mentioned in the first post.

Next try, re-installed proxmox-ve including ifupdown2:
Then, ifup enp33s0f0 and ifup -d bond0. Here, a line caught my attention:

debug: bond: pre-up : running script /etc/network/if-pre-up.d/ifenslave.dpkg-old.

After deleting the old file, ifupdown2 worked as expected. Apparently, ifupdown2 doesn't know debian's concept of dpkg, and executes outdated files. Reported to and confirmed by Proxmox, see https://bugzilla.proxmox.com/show_bug.cgi?id=5197

So if anybody hits this thread:
Make sure that there are no files /etc/network/*.d/*.dpkg-old