[SOLVED] Proxmox offline after update to 9.0.4 and systemd-boot removal

[julsssark]

Proxmox was running great under 9.0.3. Today I updated to 9.0.4 and removed systemd-boot, and my host no longer has a network connection.

• I can connect to the console directly using monitor/keyboard and everything boots fine but I can't ping the gateway using its IP address, apt update fails, etc. I am only using ipv4.
• My host is not setting the ip address correctly on its VLAN. journalctl -u networking shows an error in netlink "vmrb0.50: cannot create vlan vmbr0.50 50: operation failed with 'Operation not supported' (95)"
• Systemctrl shows that everything is running except RPC pipe file system and pvefw-logger (assuming because the network is down), and the pve lxc containers
• I tried booting into an older kernel and I had the same result
• The switch port is a trunk and I've made no changes to it in months

Results of "ip a", interfaces config and hosts file are below. I'd appreciate any thoughts on how to get the host back online.

auto lo
iface lo inet loopback

iface enp0s31f6 inet manual

auto vmbr0
iface vmbr0 inet manual
    bridge-ports enp0s31f6
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

auto vmbr0.50
iface vmbr0.50 inet static
    address 10.7.5.6/24
    gateway 10.7.5.1
#Used for ProxMox console access. Do not remove.

iface wlp0s20f3 inet manual

source /etc/network/interfaces.d/*

127.0.0.1 localhost.localdomain localhost
10.7.5.6 prox.mydomain.com prox

 

[fabian]

please post the output of "journalctl -b -u networking"

 

[jimmycav]

I'm having the same issue with very similar /etc/network/interfaces configuration. Photo of logs attached,

The error seems to be "cannot create vlan vmbr0.30 30: operation failed with 'Operation not supported' (95)

I've upgraded 3 other nodes in the cluster although this is a different mini pc model. This is a Lenovo m720q

 

[shanreich]

I'm having the same issue with very similar /etc/network/interfaces configuration. Photo of logs attached,

The error seems to be "cannot create vlan vmbr0.30 30: operation failed with 'Operation not supported' (95)

I've upgraded 3 other nodes in the cluster although this is a different mini pc model. This is a Lenovo m720q

Can you post the the contents of your /etc/network/interfaces ?

Also, for both, the output of:

ip a
lspci -nnk
lsmod | grep -E '8021q|bridge'

 

[jimmycav]

See attached.

lsmod | grep -E '8021q|bridge' didn't return anything

 

[shanreich]

Does loading the driver manually work for you?

modprobe 8021q

If that succeeds, can you try reloading the network configuration via ifreload -avd

 

[jimmycav]

No it doesn't:

modprobe: FATAL: Module 8021q not found in directory /lib/modules/6.14.8-2-pve


If it's relevant, this was also the only node where pve8to9 suggested removing the systemd-boot package

 

[fabian]

please run "update-initramfs -u -k all", and then retry the modprobe

 

[jimmycav]

Ok yes, that worked. I could then reload networking. Do I need to do anything else to permanently fix this?

And do you know what caused this? Wondering if I made a mistake

 

[shanreich]

Ok yes, that worked. I could then reload networking. Do I need to do anything else to permanently fix this?

And do you know what caused this? Wondering if I made a mistake

This is caused by the following bug [1] - so you didn't do anything wrong.
The command supplied by fabian should fix this permanently.

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1110914

 

[fabian]

updated systemd-boot package will hit the repositories shortly on our end, so that upgrading before removing systemd-boot should avoid triggering this issue.

 

[jimmycav]

Awesome, thanks so much for your quick response. I hope the OP gets his issue resolved the same way.

 

[julsssark]

Yes! The posted commands fixed the network problem. Thank you so much Proxmox team and jimmycav.