e1000 driver hang

Well, i am getting sick with the "e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang" issue.

Couple days ago it started happening, tried ethtool -K eno1 gso off gro off tso off tx off rx off rxvlan off txvlan off sg off but it seems that it didn't work..
My Ethernet controller is Intel Corporation Ethernet Connection (11) I219-V

Proxmox Version Linux 6.8.12-10-pve (2025-04-18T07:39Z)


Bash: 
ethtool -k eno1
Features for eno1:
rx-checksumming: on
tx-checksumming: on
        tx-checksum-ipv4: off [fixed]
        tx-checksum-ip-generic: on
        tx-checksum-ipv6: off [fixed]
        tx-checksum-fcoe-crc: off [fixed]
        tx-checksum-sctp: off [fixed]
scatter-gather: on
        tx-scatter-gather: on
        tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
        tx-tcp-segmentation: on
        tx-tcp-ecn-segmentation: off [fixed]
        tx-tcp-mangleid-segmentation: off
        tx-tcp6-segmentation: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-tunnel-remcsum-segmentation: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
tx-gso-list: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
rx-gro-list: off
macsec-hw-offload: off [fixed]
rx-udp-gro-forwarding: off
hsr-tag-ins-offload: off [fixed]
hsr-tag-rm-offload: off [fixed]
hsr-fwd-offload: off [fixed]
hsr-dup-offload: off [fixed]


Bash: 
  GNU nano 7.2                                                          /etc/network/interfaces                                                                  
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.0.101/24
        gateway 192.168.0.1
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

source /etc/network/interfaces.d/*


Can someone give me any suggestions, i cannot stand this anymore.
 
Last edited:
ang434 said:
Well, i am getting sick with the "e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang" issue.

Couple days ago it started happening, tried ethtool -K eno1 gso off gro off tso off tx off rx off rxvlan off txvlan off sg off but it seems that it didn't work..
My Ethernet controller is Intel Corporation Ethernet Connection (11) I219-V

Proxmox Version Linux 6.8.12-10-pve (2025-04-18T07:39Z)


Bash: 
ethtool -k eno1
Features for eno1:
rx-checksumming: on
tx-checksumming: on
        tx-checksum-ipv4: off [fixed]
        tx-checksum-ip-generic: on
        tx-checksum-ipv6: off [fixed]
        tx-checksum-fcoe-crc: off [fixed]
        tx-checksum-sctp: off [fixed]
scatter-gather: on
        tx-scatter-gather: on
        tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
        tx-tcp-segmentation: on
        tx-tcp-ecn-segmentation: off [fixed]
        tx-tcp-mangleid-segmentation: off
        tx-tcp6-segmentation: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-tunnel-remcsum-segmentation: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
tx-gso-list: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
rx-gro-list: off
macsec-hw-offload: off [fixed]
rx-udp-gro-forwarding: off
hsr-tag-ins-offload: off [fixed]
hsr-tag-rm-offload: off [fixed]
hsr-fwd-offload: off [fixed]
hsr-dup-offload: off [fixed]


Bash: 
  GNU nano 7.2                                                          /etc/network/interfaces                                                                 
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.0.101/24
        gateway 192.168.0.1
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

source /etc/network/interfaces.d/*


Can someone give me any suggestions, i cannot stand this anymore.
Click to expand...
The solution (for now) is to use a previous kernel that does not have this problem. The kernel I use is 6.8.12-8-pve. Use the following command: proxmox-boot-tool kernel pin 6.8.12-8-pve --next-boot
 
  • Like
Reactions: TomFIT, JTX2023 and ang434
I seem to be another one affected by this. Surprised it has gone on for so long.

pve-manager/8.4.1/2a5fa54a8503f96d (running kernel: 6.8.12-10-pve)
Ethernet controller [0200]: Intel Corporation Ethernet Connection (7) I219-LM [8086:15bb] (rev 10)

I'll try the line below and see it it helps, thanks:

Code: 
post-up /usr/bin/logger -p debug -t ifup "Disabling offload for eno1" && /sbin/ethtool -K $IFACE tso off gso off gro off && /usr/bin/logger -p debug -t ifup "Disabled offload for eno1"
 
Last edited:
sylsyl said:
I seem to be another one affected by this. Surprised it has gone on for so long.

pve-manager/8.4.1/2a5fa54a8503f96d (running kernel: 6.8.12-10-pve)
Ethernet controller [0200]: Intel Corporation Ethernet Connection (7) I219-LM [8086:15bb] (rev 10)

I'll try the line below and see it it helps, thanks:

Code: 
post-up /usr/bin/logger -p debug -t ifup "Disabling offload for eno1" && /sbin/ethtool -K $IFACE tso off gso off gro off && /usr/bin/logger -p debug -t ifup "Disabled offload for eno1"
Click to expand...
Currently what i did is create this https://gist.githubusercontent.com/crypt0rr/60aaabd4a5c29a256b4f276122765237 and also added things inside /etc/network/interfaces, check my post just above this reply and you should see what else i added.

Also i wrote ethtool -K eno1 gso off tso off rxvlan off txvlan off gro off tx off rx off sg off and ethtool -k eno1 | grep -E 'rx-checksum|tx-checksum|tso|gro|gso|sg|lro|rxvlan|txvlan|ufo'

I will report here if this happens again.
Worst case scenario i will just buy a ethernet card and plug it into pcie and it should solve this problem, hopefully.
Now i understand why many people are against Intel NICs and shit..
 
sylsyl said:
I seem to be another one affected by this. Surprised it has gone on for so long.

pve-manager/8.4.1/2a5fa54a8503f96d (running kernel: 6.8.12-10-pve)
Ethernet controller [0200]: Intel Corporation Ethernet Connection (7) I219-LM [8086:15bb] (rev 10)

I'll try the line below and see it it helps, thanks:

Code: 
post-up /usr/bin/logger -p debug -t ifup "Disabling offload for eno1" && /sbin/ethtool -K $IFACE tso off gso off gro off && /usr/bin/logger -p debug -t ifup "Disabled offload for eno1"
Click to expand...

I don't want to jinx it, but this seems to have made a difference - I had been crashing every 1-3 hours previously and I have now been up for 12 hours (and have even been able to increase the cores given to the VM) straight.Screenshot 2025-05-14 104636.png
 
Hello all,

I was suffering from this bug and in trying to solve it lead me to this thread.

In my case it seems to was a OpenVPN tunnel established from a VM on a VLAN tag ID network with a wrong MTU defined (bigger first and equal second try than the physical Ethernet definitions).

Lower the MTU (1500 on the ethernet and 1400 on the OpenVPN tun tunnel) seems helped... no issues since the change!

Let us see in the following days.

Greetings, Pedro
 
ang434 said:
Currently what i did is create this https://gist.githubusercontent.com/crypt0rr/60aaabd4a5c29a256b4f276122765237 and also added things inside /etc/network/interfaces, check my post just above this reply and you should see what else i added.

Also i wrote ethtool -K eno1 gso off tso off rxvlan off txvlan off gro off tx off rx off sg off and ethtool -k eno1 | grep -E 'rx-checksum|tx-checksum|tso|gro|gso|sg|lro|rxvlan|txvlan|ufo'

I will report here if this happens again.
Worst case scenario i will just buy a ethernet card and plug it into pcie and it should solve this problem, hopefully.
Now i understand why many people are against Intel NICs and shit..
Click to expand...
Just want to update, this indeed solved my problem it seems. So if anyone has similar issue feel free to apply this patch.
 
This problem happened to me first time yesterday.

Kernel version - Linux 6.8.12-9-pve (2025-03-16T19:18Z)
Interface - 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (17) I219-LM (rev 11)

Code: 
# journalctl |grep "Detected Hardware Unit Hang"|cut -c1-6|uniq -c
    308 May 18

My machine was unreachable and had to manually restart. I wanted to ask first if anyone can confirm that if you have two interfaces all of them stop working or just the problematic one, thinking about adding a second interface in case the main one crashed I can still remotely restart.

Since for me it happened when I was demanding a lot from the interface, I wanted to know if people already checked the temperature of the NIC when the crash happens? In case anyone can confirm the temperature to be high can you put the hardware under a fan an see the temperature again when/if it crashes?

I see people got it working as well by reducing MTU size can anyone share more about this experience?
 
Last edited:
6.8.12-11-pve was released for both PVE and PBS, there were some changes to the ABI.
https://git.proxmox.com/?p=pve-kernel.git;a=shortlog;h=refs/heads/bookworm-6.8

ABI stands for Application Binary Interface. It defines the low-level interface between the kernel and its modules (such as device drivers), specifying how compiled code interacts with the kernel at the binary level. This includes details like register usage, memory layout, calling conventions, and symbol versions of exported kernel functions and variables
Click to expand...
 
Good News! I have upgraded to kernel 6.8.12-11-pve about 12 hours ago and have experienced no network problems!

** Update - unfortunately the interface hung again! It took longer this time. Before it would fail during backups quite quickly. This time it failed without much network activity.:(
 
Last edited:
I am seeing similar issues with my Intel X550 since two days. Every time that my Proxmox host does its backups to PBS.

Bash: 
lspci -v | grep Ethernet
04:00.0 Ethernet controller: Intel Corporation Ethernet Controller X550 (rev 01)
        Subsystem: Intel Corporation Ethernet Converged Network Adapter X550-T2
04:00.1 Ethernet controller: Intel Corporation Ethernet Controller X550 (rev 01)
        Subsystem: Intel Corporation Ethernet Converged Network Adapter X550-T2

Bash: 
journalctl |grep "Detected Tx Unit Hang"|cut -c1-6|uniq -c
   8439 May 28
    575 May 29

Proxmox and all VMs keep running, but I can‘t reach Proxmox or any VM anymore.
 
I've just started noticing this problem recently on my NUC10i7FNH after starting to handle more throughput; I upgraded the BIOS and that seems to have solved it.

I went from FNCML357.0059.2022.1019.1055 10/19/2022 to FNCML357.0066.2024.1011.0925

I don't seem to get the full gig throughput, but thats another problem for another day - at least its now stable.
 
Hi,
I just want to confirm the problem and add my configuration in the hope the kernel can be fixed:
The problem started with upgrading to Kernel 6.8.12-9-pve and continued with 6.8.12-11-pve. I have the problem on two PVE 8.4.1 nodes.

Code: 
lspci -v | grep Ethernet
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (11) I219-LM
        Subsystem: Fujitsu Technology Solutions Ethernet Connection (11) I219-LM

uname -r
6.8.12-11-pve

journalctl --since=2025-05-20 |grep "Detected Hardware Unit Hang"|cut -c1-6|uniq -c
    2 May 21
   1168 May 27
  14283 May 29
  13593 May 30
Code: 
May 30 00:00:01 pve1 kernel: e1000e 0000:00:1f.6 enp0s31f6: Detected Hardware Unit Hang:
                               TDH                  <bf>
                               TDT                  <e6>
                               next_to_use          <e6>
                               next_to_clean        <be>
                             buffer_info[next_to_clean]:
                               time_stamp           <10b1acd0a>
                               next_to_watch        <bf>
                               jiffies              <10cceba80>
                               next_to_watch.status <0>
                             MAC Status             <80083>
                             PHY Status             <796d>
                             PHY 1000BASE-T Status  <3800>
                             PHY Extended Status    <3000>
                             PCI Status             <10>

I changed the kernel back to 6.8.12-8-pve as a workaround (hope it works):
apt install proxmox-kernel-6.8.12-8-pve
proxmox-boot-tool kernel pin 6.8.12-8-pve --next-boot
 
Last edited:
  • Like
Reactions: Adriankor
I've upgraded my system on Tuesday from 6.8.12-10-pve to 6.8.12-11-pve and it is still online, so yesterday I updated another system, which is technically identical. That one froze just few hours after the upgrade. The main difference is that the second one is also running PBS in a PVE VM.
 
I have the "Device Hang" issue too... Running on an older Intel NUC. Unfortunately i dont know after which update this problem came up. I realised it, as the NUC was not reachable anymore. I had to reconnect the ethernet cable to restore the network connection.
I tried downgrading to proxmox-kernel-6.8.12-8-pve and it seems to be better, but not gone.

root@pve:~# journalctl --since=2025-05-20 | grep "Detected Hardware Unit Hang" | cut -c1-6 | uniq -c
11663 May 30
689 May 31
8371 Jun 01
5 Jun 02
root@pve:~#

30.05, 31.05, and 01.06 were with the latest kernel... The 5 errors since 02.06. are with 6.8.12-8-pve.
 
Last edited:
You must log in or register to reply here.
Top Bottom