[SOLVED] Intel NIC (I225-V Rev 3) link down after switch reset

[kwat]

My system has PVE 9.0.10 with 2 NICs. One Realtek RTL8111 and the other Intel I225-V. The Realtek is on a dedicated VLAN used for management only and the I255 for everything else. Both NICs work as expected after reboots.

My problem is when the managed switched they are connected to is disconnected or rebooted. If that happens the Realtek reconnects but the I255 does not. The I255 has status DOWN (output from ip a command). Commands ifup, “systemctl restart networking” and “ip link set” fail to bring it up. Only a system reboot resolves the problem. My interfaces file is attached. The I255 is enp4s0 on bridge vmbr1.

Any help on what is going on would be greatly appreciated.

***************************************************************************
auto lo
iface lo inet loopback

iface enp3s0 inet manual
#1GB NIC

iface enp4s0 inet manual
#2.5GB NIC

auto vmbr0
iface vmbr0 inet manual
bridge-ports enp3s0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
#Uplink to Gateway

auto vmbr1
iface vmbr1 inet manual
bridge-ports enp4s0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
#Virtual pfSense LAN

auto vmbr0.2
iface vmbr0.2 inet dhcp
#Management interface

 

[kwat]

I have more information I can provide. The problem can be reproduced by simply unplugging the I-225 from the managed switch. While unplugged the status of the link is DOWN with NO-Carrier and the status does NOT change when the network cable is plugged back in. However, if I enter "ethtool -i enp4s0" the link immediately comes back up!

Any thoughts on what is going on and how I make the link come back up without human intervention?

 

[Onslow]

I can see there are many reports about issues with Intel I225(-V). One of the reasons may be power-saving...
Is there a possibility to disable it?

Intel Support Knowledge Base https://www.intel.com/content/www/u...igabit-ethernet-controllers-up-to-2-5gbe.html advices booting to Windows (ehh...) and installing "latest LAN driver"...

Edit: if executing ethtool -i enp4s0 is the only viable workaround, maybe you manage to use something similar to a method mentioned in https://forum.proxmox.com/threads/e1000e-reset-adapter-unexpectedly.87769/

The problem there is probably other (offload functions), but I think on putting
post-up /usr/sbin/ethtool -i $IFACE
at the end of iface enp4s0 stanza.

I wonder if it helps.

 

[kwat]

I can see there are many reports about issues with Intel I225(-V). One of the reasons may be power-saving...
Is there a possibility to disable it?

Intel Support Knowledge Base https://www.intel.com/content/www/u...igabit-ethernet-controllers-up-to-2-5gbe.html advices booting to Windows (ehh...) and installing "latest LAN driver"...

Edit: if executing ethtool -i enp4s0 is the only viable workaround, maybe you manage to use something similar to a method mentioned in https://forum.proxmox.com/threads/e1000e-reset-adapter-unexpectedly.87769/

The problem there is probably other (offload functions), but I think on putting
post-up /usr/sbin/ethtool -i $IFACE
at the end of iface enp4s0 stanza.

I wonder if it helps.

Thank you for your suggestion, it pointed me in the right direction. The problem stems from me running "powetop --auto-tune" as a service. This puts the I-225 into a state that does not respond when the link is reconnected.

Here is a good explanation I found in the internet:
the command "echo auto > /sys/bus/pci/devices/0000:01:00.0/power/control" (run by powertop) enables runtime power management for a specific PCI device. The transition to low-power states like D3 is typically triggered by inactivity, such as loss of link or lack of device activity.

When I added "ExecStart=/bin/sh -c "echo on > /sys/bus/pci/devices/0000:01:00.0/power/control"" to the service, the problem was resolved.