Server Crashes after high usage on network?

Chiren

Member
Jun 22, 2022
9
1
8
Server seemingly crashes when when using a lot of network this is the syslog

Code:
Jun 30 23:47:07 chiren kernel: e1000e 0000:00:1f.6 eno1: Reset adapter unexpectedly
Jun 30 23:47:07 chiren kernel: vmbr0: port 1(eno1) entered disabled state
Jun 30 23:47:08 chiren pvestatd[1414]: command '/sbin/vgscan --ignorelockingfailure --mknodes' failed: exit code 5
Jun 30 23:47:11 chiren kernel: e1000e 0000:00:1f.6 eno1: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Jun 30 23:47:11 chiren kernel: vmbr0: port 1(eno1) entered blocking state
Jun 30 23:47:11 chiren kernel: vmbr0: port 1(eno1) entered forwarding state
Jun 30 23:47:13 chiren kernel: e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
  TDH                  <0>
  TDT                  <1>
  next_to_use          <1>
  next_to_clean        <0>
buffer_info[next_to_clean]:
  time_stamp           <1025f58b9>
  next_to_watch        <0>
  jiffies              <1025f5a28>
  next_to_watch.status <0>
MAC Status             <40080083>
PHY Status             <796d>
PHY 1000BASE-T Status  <3c00>
PHY Extended Status    <3000>
PCI Status             <10>
Jun 30 23:47:15 chiren kernel: e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
  TDH                  <0>
  TDT                  <1>
  next_to_use          <1>
  next_to_clean        <0>
buffer_info[next_to_clean]:
  time_stamp           <1025f58b9>
  next_to_watch        <0>
  jiffies              <1025f5c18>
  next_to_watch.status <0>
MAC Status             <40080083>
PHY Status             <796d>
PHY 1000BASE-T Status  <3c00>
PHY Extended Status    <3000>
PCI Status             <10>
Jun 30 23:47:17 chiren kernel: e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
  TDH                  <0>
  TDT                  <1>
  next_to_use          <1>
  next_to_clean        <0>
buffer_info[next_to_clean]:
  time_stamp           <1025f58b9>
  next_to_watch        <0>
  jiffies              <1025f5e11>
  next_to_watch.status <0>
MAC Status             <40080083>
PHY Status             <796d>
PHY 1000BASE-T Status  <3c00>
PHY Extended Status    <3000>
PCI Status             <10>
Jun 30 23:47:19 chiren pvestatd[1414]: command '/sbin/vgscan --ignorelockingfailure --mknodes' failed: exit code 5
Jun 30 23:47:19 chiren kernel: e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
  TDH                  <0>
  TDT                  <1>
  next_to_use          <1>
  next_to_clean        <0>
buffer_info[next_to_clean]:
  time_stamp           <1025f58b9>
  next_to_watch        <0>
  jiffies              <1025f6000>
  next_to_watch.status <0>
MAC Status             <40080083>
PHY Status             <796d>
PHY 1000BASE-T Status  <3c00>
PHY Extended Status    <3000>
PCI Status             <10>
Jun 30 23:47:21 chiren kernel: e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
  TDH                  <0>
  TDT                  <1>
  next_to_use          <1>
  next_to_clean        <0>
buffer_info[next_to_clean]:
  time_stamp           <1025f58b9>
  next_to_watch        <0>
  jiffies              <1025f61f8>
  next_to_watch.status <0>
MAC Status             <40080083>
PHY Status             <796d>
PHY 1000BASE-T Status  <3c00>
PHY Extended Status    <3000>
PCI Status             <10>
Jun 30 23:47:21 chiren kernel: e1000e 0000:00:1f.6 eno1: Reset adapter unexpectedly
Jun 30 23:47:21 chiren kernel: vmbr0: port 1(eno1) entered disabled state
Jun 30 23:47:25 chiren kernel: e1000e 0000:00:1f.6 eno1: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Jun 30 23:47:25 chiren kernel: vmbr0: port 1(eno1) entered blocking state
Jun 30 23:47:25 chiren kernel: vmbr0: port 1(eno1) entered forwarding state
Jun 30 23:47:27 chiren kernel: e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
  TDH                  <0>
  TDT                  <1>
  next_to_use          <1>
  next_to_clean        <0>
buffer_info[next_to_clean]:
  time_stamp           <1025f6641>
  next_to_watch        <0>
  jiffies              <1025f6801>
  next_to_watch.status <0>
MAC Status             <40080083>
PHY Status             <796d>
PHY 1000BASE-T Status  <3c00>
PHY Extended Status    <3000>
PCI Status             <10>
Jun 30 23:47:28 chiren pvestatd[1414]: command '/sbin/vgscan --ignorelockingfailure --mknodes' failed: exit code 5
Jun 30 23:47:29 chiren kernel: e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
  TDH                  <0>
  TDT                  <1>
  next_to_use          <1>
  next_to_clean        <0>
buffer_info[next_to_clean]:
  time_stamp           <1025f6641>
  next_to_watch        <0>
  jiffies              <1025f69c8>
  next_to_watch.status <0>
MAC Status             <40080083>
PHY Status             <796d>
PHY 1000BASE-T Status  <3c00>
PHY Extended Status    <3000>
PCI Status             <10>
Jun 30 23:47:31 chiren kernel: e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
  TDH                  <0>
  TDT                  <1>
  next_to_use          <1>
  next_to_clean        <0>
buffer_info[next_to_clean]:
  time_stamp           <1025f6641>
  next_to_watch        <0>
  jiffies              <1025f6bb8>
  next_to_watch.status <0>
MAC Status             <40080083>
PHY Status             <796d>
PHY 1000BASE-T Status  <3c00>
PHY Extended Status    <3000>
PCI Status             <10>
Jun 30 23:47:33 chiren kernel: e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
  TDH                  <0>
  TDT                  <1>
  next_to_use          <1>
  next_to_clean        <0>
buffer_info[next_to_clean]:
  time_stamp           <1025f6641>
  next_to_watch        <0>
  jiffies              <1025f6db0>
  next_to_watch.status <0>
MAC Status             <40080083>
PHY Status             <796d>
PHY 1000BASE-T Status  <3c00>
PHY Extended Status    <3000>
PCI Status             <10>
Jun 30 23:47:35 chiren kernel: e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
  TDH                  <0>
  TDT                  <1>
  next_to_use          <1>
  next_to_clean        <0>
buffer_info[next_to_clean]:
  time_stamp           <1025f6641>
  next_to_watch        <0>
  jiffies              <1025f6fa0>
  next_to_watch.status <0>
MAC Status             <40080083>
PHY Status             <796d>
PHY 1000BASE-T Status  <3c00>
PHY Extended Status    <3000>
PCI Status             <10>
Jun 30 23:47:35 chiren kernel: e1000e 0000:00:1f.6 eno1: Reset adapter unexpectedly
Jun 30 23:47:35 chiren kernel: vmbr0: port 1(eno1) entered disabled state
Jun 30 23:47:38 chiren pvestatd[1414]: command '/sbin/vgscan --ignorelockingfailure --mknodes' failed: exit code 5
Jun 30 23:47:39 chiren kernel: e1000e 0000:00:1f.6 eno1: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Jun 30 23:47:39 chiren kernel: vmbr0: port 1(eno1) entered blocking state
Jun 30 23:47:39 chiren kernel: vmbr0: port 1(eno1) entered forwarding state
Jun 30 23:47:41 chiren kernel: e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
  TDH                  <0>
  TDT                  <1>
  next_to_use          <1>
  next_to_clean        <0>
buffer_info[next_to_clean]:
  time_stamp           <1025f7401>
  next_to_watch        <0>
  jiffies              <1025f7580>
  next_to_watch.status <0>
MAC Status             <40080083>
PHY Status             <796d>
PHY 1000BASE-T Status  <3c00>
PHY Extended Status    <3000>
PCI Status             <10>

And it goes on and on until i go and restart the server manually from the power button.


System Info:

System: Kernel: 6.1.2-1-pve x86_64 bits: 64 compiler: gcc v: 10.2.1
parameters: BOOT_IMAGE=/boot/vmlinuz-6.1.2-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on
iommu=pt pcie_acs_override=downstream,multifunction initcall_blacklist=sysfb_init video=simplefb:eek:ff
video=vesafb:eek:ff video=efifb:eek:ff video=vesa:eek:ff disable_vga=1 vfio_iommu_type1.allow_unsafe_interrupts=1
kvm.ignore_msrs=1
modprobe.blacklist=radeon,nouveau,nvidia,nvidiafb,nvidia-gpu,snd_hda_intel,snd_hda_codec_hdmi,i915
Console: tty 0 Distro: Debian GNU/Linux 11 (bullseye)
Machine: Type: Desktop System: ASUS product: N/A v: N/A serial: N/A
Mobo: ASUSTeK model: PRIME B560M-K v: Rev 1.xx serial: <filter> UEFI: American Megatrends v: 0820
date: 04/27/2021
CPU: Info: 6-Core model: 11th Gen Intel Core i5-11400 socket: LGA1200 bits: 64 type: MT MCP arch: N/A
family: 6 model-id: A7 (167) stepping: 1 microcode: 3C L1 cache: 480 KiB L2 cache: 12 MiB
L3 cache: 11.7 MiB
flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 62208
Speed: 4281 MHz min/max: 800/4400 MHz base/boost: 2574/4400 volts: 0.8 V ext-clock: 100 MHz
Core speeds (MHz): 1: 4281 2: 4205 3: 4295 4: 4294 5: 4176 6: 4292 7: 4194 8: 4290 9: 2600 10: 2600
11: 4284 12: 4291
Vulnerabilities: Type: itlb_multihit status: Not affected
Type: l1tf status: Not affected
Type: mds status: Not affected
Type: meltdown status: Not affected
Type: mmio_stale_data status: Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable
Type: retbleed mitigation: Enhanced IBRS
Type: spec_store_bypass mitigation: Speculative Store Bypass disabled via prctl
Type: spectre_v1 mitigation: usercopy/swapgs barriers and __user pointer sanitization
Type: spectre_v2 mitigation: Enhanced IBRS, IBPB: conditional, RSB filling, PBRSB-eIBRS: SW sequence
Type: srbds status: Not affected
Type: tsx_async_abort status: Not affected
Graphics: Device-1: NVIDIA GP106 [GeForce GTX 1060 6GB] driver: vfio-pci v: N/A alternate: nvidiafb,nouveau
bus ID: 01:00.0 chip ID: 10de:1c03 class ID: 0300
Display: server: No display server data found. Headless machine? tty: 120x30
Message: Advanced graphics data unavailable in console for root.
Audio: Device-1: Intel vendor: ASUSTeK driver: N/A alternate: snd_hda_intel, snd_sof_pci_intel_tgl
bus ID: 00:1f.3 chip ID: 8086:43c8 class ID: 0403
Device-2: NVIDIA GP106 High Definition Audio driver: vfio-pci alternate: snd_hda_intel bus ID: 01:00.1
chip ID: 10de:10f1 class ID: 0403
Sound Server: ALSA v: k6.1.2-1-pve
Network: Device-1: Intel Ethernet I219-V vendor: ASUSTeK driver: e1000e v: kernel port: efa0 bus ID: 00:1f.6
chip ID: 8086:15fa class ID: 0200
IF: eno1 state: up speed: 1000 Mbps duplex: full mac: <filter>
IF-ID-1: bonding_masters state: N/A speed: N/A duplex: N/A mac: N/A
IF-ID-2: fwbr100i0 state: up speed: 10000 Mbps duplex: unknown mac: <filter>
IF-ID-3: fwbr101i0 state: up speed: 10000 Mbps duplex: unknown mac: <filter>
IF-ID-4: fwbr102i0 state: up speed: 10000 Mbps duplex: unknown mac: <filter>
IF-ID-5: fwln100i0 state: up speed: 10000 Mbps duplex: full mac: <filter>
IF-ID-6: fwln101i0 state: up speed: 10000 Mbps duplex: full mac: <filter>
IF-ID-7: fwln102i0 state: up speed: 10000 Mbps duplex: full mac: <filter>
IF-ID-8: fwpr100p0 state: up speed: 10000 Mbps duplex: full mac: <filter>
IF-ID-9: fwpr101p0 state: up speed: 10000 Mbps duplex: full mac: <filter>
IF-ID-10: fwpr102p0 state: up speed: 10000 Mbps duplex: full mac: <filter>
IF-ID-11: tap100i0 state: unknown speed: 10 Mbps duplex: full mac: <filter>
IF-ID-12: tap101i0 state: unknown speed: 10 Mbps duplex: full mac: <filter>
IF-ID-13: veth102i0 state: up speed: 10000 Mbps duplex: full mac: <filter>
IF-ID-14: vmbr0 state: up speed: 10000 Mbps duplex: unknown mac: <filter>
Drives: Local Storage: total: 9.1 TiB lvm-free: 42.02 GiB used: 20.99 GiB (0.2%)
ID-1: /dev/sda maj-min: 8:0 vendor: Crucial model: CT500MX500SSD1 family: Micron Client SSDs
size: 465.76 GiB block size: physical: 4096 B logical: 512 B sata: 3.3 speed: 6.0 Gb/s rotation: SSD
serial: <filter> rev: 023 temp: 45 C scheme: GPT
SMART: yes state: enabled health: PASSED on: 354d 2h cycles: 2649 written: 117.47 TiB Old-Age:
UDMA CRC errors: 19182 alert: Bad cable/connection?
ID-2: /dev/sdb maj-min: 8:16 vendor: Seagate model: ST4000DM004-2CV104 family: BarraCuda 3.5
size: 3.64 TiB block size: physical: 4096 B logical: 512 B sata: 3.1 speed: 6.0 Gb/s rotation: 5425 rpm
serial: <filter> rev: 0001 temp: 41 C scheme: GPT
SMART: yes state: enabled health: PASSED on: 358d 0h cycles: 86 read: 41.06 TiB written: 11.08 TiB
Pre-Fail: attribute: Spin_Retry_Count value: 100 worst: 100 threshold: 97
ID-3: /dev/sdc maj-min: 8:32 vendor: Toshiba model: DT01ACA100 family: 3.5" DT01ACA... Desktop HDD
size: 931.51 GiB block size: physical: 4096 B logical: 512 B sata: 3.0 speed: 6.0 Gb/s rotation: 7200 rpm
serial: <filter> rev: A750 temp: 38 C scheme: GPT
SMART: yes state: enabled health: PASSED on: 3y 30d 16h cycles: 4369
ID-4: /dev/sdd maj-min: 8:48 vendor: Seagate model: ST500LM012 HN-M500MBB
family: Seagate SpinPoint M8 (AF) size: 465.76 GiB block size: physical: 4096 B logical: 512 B sata: 3.0
speed: 3.0 Gb/s rotation: 5400 rpm serial: <filter> rev: 0010 temp: 36 C scheme: GPT
SMART: yes state: enabled health: PASSED on: 2y 12d 16h cycles: 4134
ID-5: /dev/sde maj-min: 8:64 vendor: Toshiba model: HDWD240 size: 3.64 TiB block size: physical: 4096 B
logical: 512 B sata: 3.3 speed: 6.0 Gb/s rotation: 5400 rpm serial: <filter> rev: 0A temp: 35 C
scheme: GPT
SMART: yes state: enabled health: PASSED on: 1y 52d 19h cycles: 327
Partition: ID-1: / raw size: 96 GiB size: 93.93 GiB (97.85%) used: 19.15 GiB (20.4%) fs: ext4 block size: 4096 B
dev: /dev/dm-1 maj-min: 253:1 mapped: pve-root
ID-2: /boot/efi raw size: 512 MiB size: 511 MiB (99.80%) used: 328 KiB (0.1%) fs: vfat block size: 512 B
dev: /dev/sda2 maj-min: 8:2
Swap: Kernel: swappiness: 60 (default) cache pressure: 100 (default)
ID-1: swap-1 type: partition size: 8 GiB used: 1.84 GiB (22.9%) priority: -2 dev: /dev/dm-0
maj-min: 253:0 mapped: pve-swap
Sensors: System Temperatures: cpu: 27.8 C mobo: N/A
Fan Speeds (RPM): N/A
Info: Processes: 323 Uptime: 46m wakeups: 0 Memory: 15.47 GiB used: 15.29 GiB (98.9%) Init: systemd v: 247
runlevel: 5 Compilers: gcc: N/A Packages: apt: 886 lib: 498 Shell: Zsh v: 5.8 running in: tty 0 (SSH)
inxi: 3.3.01



Thanks in advance if anyone can help!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!