ifupdown2/networking Issues - "Apply Configuration" Will Not Bring Up Interfaces

[davidi]

Hello! I am having a lot of issues with trying to apply any networking changes without rebooting the server. I installed ifupdown2. Whenever I make an edit to the network in Proxmox GUI and click on Apply Configuration, I lose access to the node. The only way I can bring it bring it back online is by going to the physical server and either rebooting it or by typing systemctl restart networking.service from the cli and that will bring it back online.

If I do ifreload -av, that fails to bring it back online.

I've been searching for a solution for the past few days and I cannot seem to find one. I have 3 nodes in the cluster and they all behave exactly the same. So, I'm assuming whatever the issue is, it's affecting all 3 nodes. Any help in figuring this out would be much appreciated!

Installed packages:

proxmox-ve: 7.4-1 (running kernel: 5.15.126-1-pve)
pve-manager: 7.4-17 (running version: 7.4-17/513c62be)
pve-kernel-5.15: 7.4-7
pve-kernel-5.13: 7.1-9
pve-kernel-5.15.126-1-pve: 5.15.126-1
pve-kernel-5.15.116-1-pve: 5.15.116-1
pve-kernel-5.13.19-6-pve: 5.13.19-15
ceph-fuse: 14.2.21-1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx4
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4.1
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.4-2
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-3
libpve-rs-perl: 0.7.7
libpve-storage-perl: 7.4-3
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
proxmox-backup-client: 2.4.3-1
proxmox-backup-file-restore: 2.4.3-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.2
proxmox-widget-toolkit: 3.7.3
pve-cluster: 7.3-3
pve-container: 4.4-6
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-4~bpo11+1
pve-firewall: 4.3-5
pve-firmware: 3.6-5
pve-ha-manager: 3.6.1
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-2
qemu-server: 7.4-4
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.11-pve1

Repo Info:

deb http://ftp.us.debian.org/debian bullseye main contrib

deb http://ftp.us.debian.org/debian bullseye-updates main contrib

# PVE pve-no-subscription repository provided by proxmox.com,
# # NOT recommended for production use
deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription

# security updates
deb http://security.debian.org bullseye-security main contrib

 

[Moayad]

Hi,

May you provide us with node network configuration, and the output of `ifreload -av`?

 

[davidi]

Hi @Moayad

Sure, no problem.

Interfaces:

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

auto eno3
iface eno3 inet manual

iface eno4 inet manual

auto enp5s0f0
iface enp5s0f0 inet static
        address 192.168.30.72/24
        mtu 9000

auto enp5s0f1
iface enp5s0f1 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno2 eno3
        bond-miimon 100
        bond-mode balance-tlb

auto vmbr0
iface vmbr0 inet static
        address 192.168.10.72/24
        gateway 192.168.10.1
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
        address 192.168.11.72/24
        bridge-ports eno4
        bridge-stp off
        bridge-fd 0

ifreload -av output

╰─# ifreload -av
info: requesting link dump
info: requesting address dump
info: requesting netconf dump
info: loading builtin modules from ['/usr/share/ifupdown2/addons']
info: module openvswitch not loaded (module init failed: no /usr/bin/ovs-vsctl found)
info: module openvswitch_port not loaded (module init failed: no /usr/bin/ovs-vsctl found)
info: module ppp not loaded (module init failed: no /usr/bin/pon found)
info: module batman_adv not loaded (module init failed: no /usr/sbin/batctl found)
info: executing /sbin/sysctl net.bridge.bridge-allow-multiple-vlans
info: module mstpctl not loaded (module init failed: no /sbin/mstpctl found)
info: executing /bin/ip rule show
info: executing /bin/ip -6 rule show
info: address: using default mtu 1500
info: address: max_mtu undefined
info: executing /usr/sbin/ip vrf id
info: mgmt vrf_context = False
info: dhclient: dhclient_retry_on_failure set to 0
info: executing /bin/ip addr help
info: address metric support: OK
info: module ppp not loaded (module init failed: no /usr/bin/pon found)
info: module mstpctl not loaded (module init failed: no /sbin/mstpctl found)
info: module batman_adv not loaded (module init failed: no /usr/sbin/batctl found)
info: module openvswitch_port not loaded (module init failed: no /usr/bin/ovs-vsctl found)
info: module openvswitch not loaded (module init failed: no /usr/bin/ovs-vsctl found)
info: looking for user scripts under /etc/network
info: loading scripts under /etc/network/if-pre-up.d ...
info: loading scripts under /etc/network/if-up.d ...
info: loading scripts under /etc/network/if-post-up.d ...
info: loading scripts under /etc/network/if-pre-down.d ...
info: loading scripts under /etc/network/if-down.d ...
info: loading scripts under /etc/network/if-post-down.d ...
info: 'link_master_slave' is set. slave admin state changes will be delayed till the masters admin state change.
info: using mgmt iface default prefix eth
info: processing interfaces file /etc/network/interfaces
info: no interfaces to down ..
info: reload: scheduling up on interfaces: ['lo', 'eno1', 'eno2', 'eno3', 'enp5s0f0', 'enp5s0f1', 'bond0', 'vmbr0', 'vmbr1']
info: lo: running ops ...
info: executing /sbin/sysctl net.mpls.conf.lo.input=0
info: executing /etc/network/if-up.d/postfix
info: executing /etc/network/if-up.d/chrony
info: enp5s0f0: running ops ...
info: executing /sbin/sysctl net.mpls.conf.enp5s0f0.input=0
info: executing /etc/network/if-up.d/postfix
info: executing /etc/network/if-up.d/chrony
info: enp5s0f1: running ops ...
info: executing /sbin/sysctl net.mpls.conf.enp5s0f1.input=0
info: executing /etc/network/if-up.d/postfix
info: executing /etc/network/if-up.d/chrony
info: eno1: running ops ...
info: vrf: syncing table map to /etc/iproute2/rt_tables.d/ifupdown2_vrf_map.conf
info: vrf: dumping iproute2_vrf_map
info: {}
info: executing /sbin/sysctl net.mpls.conf.eno1.input=0
info: executing /etc/network/if-up.d/postfix
info: executing /etc/network/if-up.d/chrony
info: eno3: running ops ...
info: executing /sbin/sysctl net.mpls.conf.eno3.input=0
info: executing /etc/network/if-up.d/postfix
info: executing /etc/network/if-up.d/chrony
info: eno2: running ops ...
info: executing /sbin/sysctl net.mpls.conf.eno2.input=0
info: executing /etc/network/if-up.d/postfix
info: executing /etc/network/if-up.d/chrony
info: bond0: running ops ...
info: bond0: already exists, no change detected
info: bond0: invalid bond mac detected - resetting to eno1's mac (00:24:e8:52:a4:d1)
info: bond0: netlink: ip link set dev bond0 down
info: bond0: netlink: ip link set dev bond0 address 00:24:e8:52:a4:d1
info: bond0: netlink: ip link set dev bond0 up
info: vmbr0: applying bridge port configuration: ['bond0']
info: vmbr0: bond0: bridge-pathcosts: no configuration detected, resetting to default 100
info: bond0: netlink: ip link set dev bond0: bridge port attributes
info: executing /sbin/sysctl net.mpls.conf.bond0.input=0
info: executing /etc/network/if-up.d/postfix
info: executing /etc/network/if-up.d/chrony
info: vmbr0: running ops ...
info: vmbr0: bridge already exists
info: vmbr0: applying bridge settings
info: vmbr0: reset bridge-hashel to default: 4
info: vmbr0: reset bridge-hashmax to default: 512
info: reading '/sys/class/net/vmbr0/bridge/stp_state'
info: vmbr0: netlink: ip link set dev vmbr0 type bridge (with attributes)
info: vmbr0: port fwpr133p0 will stay enslaved as it matches with bridge-ports-condone-regex
info: vmbr0: port fwpr112p0 will stay enslaved as it matches with bridge-ports-condone-regex
info: vmbr0: port bond0: already processed
info: vmbr0: applying bridge configuration specific to ports
info: vmbr0: processing bridge config for port bond0
info: vmbr0: setting bridge mac to port bond0 mac
info: vmbr0: netlink: ip link set dev vmbr0 down
info: vmbr0: netlink: ip link set dev vmbr0 address 00:24:e8:52:a4:d1
info: vmbr0: netlink: ip link set dev vmbr0 up
info: executing /sbin/sysctl net.mpls.conf.vmbr0.input=0
info: writing '0' to file /proc/sys/net/ipv4/conf/vmbr0/arp_accept
info: executing /bin/ip route add default via 192.168.10.1 proto kernel dev vmbr0 onlink
info: executing /etc/network/if-up.d/postfix
info: executing /etc/network/if-up.d/chrony
info: eno4: running ops ...
info: vmbr1: applying bridge port configuration: ['eno4']
info: vmbr1: eno4: bridge-pathcosts: no configuration detected, resetting to default 100
info: eno4: netlink: ip link set dev eno4: bridge port attributes
info: executing /sbin/sysctl net.mpls.conf.eno4.input=0
info: executing /etc/network/if-up.d/postfix
info: executing /etc/network/if-up.d/chrony
info: vmbr1: running ops ...
info: vmbr1: bridge already exists
info: vmbr1: applying bridge settings
info: vmbr1: reset bridge-hashel to default: 4
info: vmbr1: reset bridge-hashmax to default: 512
info: reading '/sys/class/net/vmbr1/bridge/stp_state'
info: vmbr1: netlink: ip link set dev vmbr1 type bridge (with attributes)
info: vmbr1: port eno4: already processed
info: vmbr1: applying bridge configuration specific to ports
info: vmbr1: processing bridge config for port eno4
info: bridge mac is already inherited from eno4
info: executing /sbin/sysctl net.mpls.conf.vmbr1.input=0
info: writing '0' to file /proc/sys/net/ipv4/conf/vmbr1/arp_accept
info: executing /etc/network/if-up.d/postfix
info: executing /etc/network/if-up.d/chrony
info: exit status 0

 

[Moayad]

Do you have NetworkManager installed? if yes please consider to remote it. if not, I would test another/simple network config with `ifreload` in order to narrow down the issue (Make sure to have a backup of the current config first).

 

[davidi]

Hi @Moayad

I checked to see if NetworkManager was installed, and it was not. So, I did as you suggested, I made a backup of the interfaces file, started clean and kept adding to it until Apply Configuration quit working. It quit working when I added the Bond interface, once I deleted the Bond interface, Apply Configuration worked like it's supposed to. When I get some time, I will have to dig into why the Bond interface is giving me issues.

Thank you for your help! I appreciate it