No Networking After Upgrade to 8.2

[jt-socal]

I upgraded to 8.2 and now have no networking connections. I still have the same IP, but cannot ping out from the machine to the network or any VM. I am also unable to ping the host. Obviously cannot connect to web interface or reach any VM. Any suggestions please?

I tried disabling pve-firewall and proxmox-firewall services and rebooting, did not help. I was not using the firewall previously.

I rebooted into the prior kernel and networking is working again. Any suggestions on debugging the kernel upgrade?
root@pve:~# lspci | grep -i eth
67:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
67:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
67:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
67:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
b7:00.0 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GBASE-T (rev 04)
b7:00.1 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GBASE-T (rev 04)
b7:00.2 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ (rev 04)
b7:00.3 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ (rev 04)

 

[shanreich]

How does your current network configuration look like?

ip a
cat /etc/network/interfaces

Can you attach the output of the journal since the upgrade?

journalctl --since '2024-04-24' > journal.txt

 

[jt-socal]

Thanks, redacted so would not be displayed publically. Happy to provide directly.

 

[shanreich]

It looks like the identifier of your network card changed with the new kernel causing configuring your networking to fail:

Apr 24 06:32:29 pve networking[1065]: error: vmbr0: bridge port eno8 does not exist
Apr 24 06:32:29 pve /usr/sbin/ifup[1065]: error: vmbr0: bridge port eno8 does not exist
Apr 24 06:32:29 pve networking[1065]: warning: vmbr0: apply bridge ports settings: bridge configuration failed (missing ports)
Apr 24 06:32:29 pve /usr/sbin/ifup[1065]: warning: vmbr0: apply bridge ports settings: bridge configuration failed (missing ports)
Apr 24 06:32:30 pve networking[1065]: error: >>> Full logs available in: /var/log/ifupdown2/network_config_ifupdown2_163_Apr-24-2024_06:32:29.579878 <<<
Apr 24 06:32:30 pve /usr/sbin/ifup[1065]: >>> Full logs available in: /var/log/ifupdown2/network_config_ifupdown2_163_Apr-24-2024_06:32:29.579878 <<<

You will need to adapt your network configuration to use the proper identifier for the new kernel. Sadly this can happen with new kernel upgrades. Try to use the more stable enp183s0f3 'Predictable Network Interface Names' format. The probability of this happening is lower when using those names (albeit still not zero..)

 

[jt-socal]

Thanks, deleted prior post with info I would prefer not be public. Can provide directly if needed. I'll test out new names and new kernels this weekend. Thanks again.

 

[qwertbert]

We have just had the same problem.
The inferface names were renamed after the update.
Compare your /etc/network/interfaces with the output of ip a.

 

[jt-socal]

Thanks, so, like this?
1. boot into new kernel
2. ip a
3. cd /etc/network
4. cp interfaces interfaces.backup.just.in.case
5. nano -w interfaces
a. replace interface names without output of ip a, save and exit
6. service networking restart

 

[qwertbert]

Yes, I would use Vim, but that's up to you.

 

[badbroccoli]

Had the same issue. Resolved by modifying /etc/network/interfaces to reflect updated interface names.

 

[leeh]

Also had this happen. Really pissed me off. Some warning in the upgrade notes would have been nice—I could have prepared for it.

edit - there is a caveat in the upgrade notes. i'm dumb.

 

[Thomas Stone]

In my case the built-in network in my Asus PN50 kept working but my secondary NIC (a USB running ASIX AX88179) stopped working. I assumed either that cheapo nic finally died or we lost the driver with the update somehow. I swapped it for another USB nic I had lying around (after removing the bridge port over it, and deleting the non-working nic) and that set up just fine. Reading this thread now makes me think I can reverse the process and go back to the one I had.

My main Proxmox machine that has Intel NICs (enp and ens) did not have issues.

 

[krog]

Is there any way to determine/predict the interface name that will be used after the reboot to the new kernel?

 

[bluemax]

Hello,

The igb driver of the 6.x kernels seems more peeky about the nic nvm and refuse to enable it.

Maybe are affected by the same issue I have (see my Thread 'PVE8 Intel I350-T4 NIC driver issues' https://forum.proxmox.com/threads/pve8-intel-i350-t4-nic-driver-issues.145422/).

Check if the kernel created the interface for your I350 nic:

# ip link

Also check what the kernel thought of the I350 nic during boot:

# dmesg | grep igb

See what I posted.

Hope you do not have my issue.

Max

 

[TomFIT]

To avoid nic renaminig issues, you could override the device names as advised in
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#network_override_device_names

 

[ahayes]

Also had this happen. Really pissed me off. Some warning in the upgrade notes would have been nice—I could have prepared for it.

edit - there is a caveat in the upgrade notes. i'm dumb.

Nah. You are not the only one who missed that.

Also, the linked docs from that warning say that one could "[...] pin the version v252, which is the latest naming scheme version for a fresh Proxmox VE 8.0 installation [...]" which made me wonder why I didn't encounter the interface naming change when I upgraded to PVE 8.0 but did encounter the issue today. Running `udevadm test-builtin net_id /sys/class/net/eno1` on my remaining PVE 8.1 node reports "Using default interface naming scheme 'v252'."

Running `udevadm test-builtin net_id /sys/class/net/eno1np0` (the new name) on one of my PVE 8.2 nodes reports the same "Using default interface naming scheme 'v252'."

I think I found the answer in the SYSTEMD-UDEVD.SERVICE(8) manual page:

Note that selecting a specific scheme is not sufficient to fully stabilize interface naming: the naming is generally derived from driver attributes exposed by the kernel. As the kernel is updated, previously missing attributes systemd-udevd.service is checking might appear, which affects older name derivation algorithms, too.

So I have decided to follow the documentation for overriding network device names and use MAC addresses to set custom interface names.

Predictable Network Interface Names is a bit of a misnomer once you start digging in. But to be fair, it is a tough nut to crack.

I hope that helps.

 

[Binary Bandit]

Also had this happen. Really pissed me off. Some warning in the upgrade notes would have been nice—I could have prepared for it.

edit - there is a caveat in the upgrade notes. i'm dumb.

When I upgraded my PVE 7 to 8 cluster, I read and re-read the instructions over several days. I've never felt the need to do this for any sub-version (7.x, for example) upgrade. I'll now be reading upgrade notes. I'm grateful to have found this thread, but I'm definitely frustrated.