Proxmox VE 9.2.21 - BCM57412 bnxt_en network failure after upgrade to kernel 7.0.14-20-pve

loste87 · 2026-10-09T09:35:49+0200

fabian said:
-22 contains the initial version, -23 contains an updated one

Is it recommended to upgrade to -23? We updated our cluster to -22 yesterday and everything seems to be working fine.

fabian · 2026-10-09T10:01:07+0200

it contains a few other fixes in other parts of the kernel as well, but neither kernel is on enterprise yet.

doklages · 2026-10-09T16:52:00+0200

Hello again,
I can give a update:
With the .23 Kernel version the AMD-VI Error Messages are gone but the links is still going done after some hours but now with below error messages. The network card itself was already replaced to exclude an hardware isseu:

Code:

 bnxt_en 0000:10:00.0 enp16s0f0np0: Resp cmpl intr err msg: 0x51
Oct 09 16:45:16 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: hwrm_ring_free type 2 failed. rc:fffffff0 err:0
Oct 09 16:45:17 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: NETDEV WATCHDOG: CPU: 8: transmit queue 0 timed out 5189 ms
Oct 09 16:45:17 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: TX timeout detected, starting reset task!
Oct 09 16:45:17 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: Resp cmpl intr err msg: 0x51
Oct 09 16:45:17 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: hwrm_ring_free type 2 failed. rc:fffffff0 err:0
Oct 09 16:45:18 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: Resp cmpl intr err msg: 0x51
Oct 09 16:45:18 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: hwrm_ring_free type 2 failed. rc:fffffff0 err:0
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: Resp cmpl intr err msg: 0x51
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: hwrm_ring_free type 2 failed. rc:fffffff0 err:0
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: Invalid hwrm seq id 30802
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: Invalid hwrm seq id 30806
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: [0.0]: tx{fw_ring: 170 prod: 14 cons: 0}
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: [0]: rx{fw_ring: 242 prod: 1ff} rx_agg{fw_ring: 65535 agg_prod: 0 sw_agg_prod: 0}
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: [0]: cp{fw_ring: 1 raw_cons: 0}
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: [1.0]: tx{fw_ring: 171 prod: 0 cons: 0}
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: [1]: rx{fw_ring: 243 prod: 1ff} rx_agg{fw_ring: 65535 agg_prod: 0 sw_agg_prod: 0}
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: [1]: cp{fw_ring: 310 raw_cons: 0}
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: [2.0]: tx{fw_ring: 172 prod: 0 cons: 0}
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: [2]: rx{fw_ring: 244 prod: 1ff} rx_agg{fw_ring: 65535 agg_prod: 0 sw_agg_prod: 0}
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: [2]: cp{fw_ring: 311 raw_cons: 0}
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: [3.0]: tx{fw_ring: 173 prod: 2 cons: 0}
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: [3]: rx{fw_ring: 245 prod: 1ff} rx_agg{fw_ring: 65535 agg_prod: 0 sw_agg_prod: 0}
Oct 09 16:45:19 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: [3]: cp{fw_ring: 312 raw_cons: 0}
Oct 09 16:45:20 kvm1 kernel: net_ratelimit: 2207 callbacks suppressed
Oct 09 16:45:20 kvm1 kernel: bond0: (slave enp16s0f1np1): link status down for interface, disabling it in 200 ms
Oct 09 16:45:20 kvm1 kernel: bond0: (slave enp16s0f1np1): link status down for interface, disabling it in 200 ms
Oct 09 16:45:20 kvm1 kernel: bond0: (slave enp16s0f1np1): link status down for interface, disabling it in 200 ms
Oct 09 16:45:20 kvm1 kernel: bond0: (slave enp16s0f1np1): link status down for interface, disabling it in 200 ms
Oct 09 16:45:20 kvm1 kernel: bond0: (slave enp16s0f1np1): link status down for interface, disabling it in 200 ms
Oct 09 16:45:20 kvm1 kernel: bond0: (slave enp16s0f1np1): link status down for interface, disabling it in 200 ms
Oct 09 16:45:20 kvm1 kernel: bond0: (slave enp16s0f1np1): link status down for interface, disabling it in 200 ms
Oct 09 16:45:20 kvm1 kernel: bond0: (slave enp16s0f1np1): link status down for interface, disabling it in 200 ms
Oct 09 16:45:20 kvm1 kernel: bond0: (slave enp16s0f1np1): link status down for interface, disabling it in 200 ms
Oct 09 16:45:20 kvm1 kernel: bond0: (slave enp16s0f1np1): link status down for interface, disabling it in 200 ms
Oct 09 16:45:20 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: Resp cmpl intr err msg: 0x51
Oct 09 16:45:20 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: hwrm_ring_free type 2 failed. rc:fffffff0 err:0
Oct 09 16:45:21 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: Resp cmpl intr err msg: 0x51
Oct 09 16:45:21 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: hwrm_ring_free type 2 failed. rc:fffffff0

The current Firmware on the card is:

Code:

root@kvm1:~# ethtool -i enp16s0f0np0
driver: bnxt_en
version: 7.0.14-22-pve
firmware-version: 214.0.207.0/pkg 214.0.207.0
expansion-rom-version:
bus-info: 0000:10:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no

Update:

To just make sure that there is no deep firmeware bug I updated the controller to a more recent firmware bersion (year 2025):

Code:

root@kvm1:~/firmware# ethtool -i enp16s0f0np0
driver: bnxt_en
version: 7.0.14-23-pve
firmware-version: 233.0.151.0/pkg 233.1.135.7
expansion-rom-version:
bus-info: 0000:10:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no

I will test if this makes a difference

doklages · 2026-10-09T22:00:55+0200

With the new firmware the card also get locked up but now with even different error messages:

Code:

Oct 09 21:42:11 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: Abandoning msg {0x37 0x7fce} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:11 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: Abandoning msg {0x23 0x7e2e} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:11 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: Abandoning msg {0xb4 0x7e2f} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:12 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: Abandoning msg {0x23 0x7e30} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:12 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: Abandoning msg {0xb4 0x7e31} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:12 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: Abandoning msg {0x23 0x7fcf} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:12 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: Abandoning msg {0xb4 0x7fd0} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:13 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: Abandoning msg {0x23 0x7e32} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:13 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: Abandoning msg {0xb4 0x7e33} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:13 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: Abandoning msg {0x23 0x7fd1} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:13 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: Abandoning msg {0xb4 0x7fd2} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:14 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: Abandoning msg {0x23 0x7e34} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:14 kvm1 kernel: bnxt_en 0000:10:00.1 enp16s0f1np1: Abandoning msg {0xb4 0x7e35} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:14 kvm1 kernel: bnxt_en 0000:10:00.0 enp16s0f0np0: Abandoning msg {0x23 0x7fd3} len: 0 due to firmware status: 0xffffffff
Oct 09 21:42:14 kvm1 kern

EDIT: I have now updated to the most recent firmware for BCM57412 network chip 238.1.168.0 from 08/05/2026. Let's see whats happens

Code:

root@kvm1:~/firmware# ./bnxtnvm verify


Device #1
---------
Device: enp16s0f0np0
Verifying the NVM components
All the NVM components are verified successfully
root@kvm1:~/firmware# ./bnxtnvm device_info

Device Interface Name       : enp16s0f0np0
MACAddress                  : b0:26:28:d6:a1:80
Base MACAddress             : B0:26:28:D6:A1:80
Device Serial Number        : N/A
Chip Number                 : BCM57412
Part Number                 : BCM957412A4120AC
Description                 : Broadcom BCM57412 2x10G PCIe Ethernet NIC
PCI Vendor Id               : 14e4
PCI Device Id               : 16d6
PCI Subsys Vendor Id        : 14e4
PCI Subsys Device Id        : 4124
PCI Device Name             : 0000:10:00.0
Adapter Rev                 : 01
Active Package version      : 238.1.169.0
Package version on NVM      : 238.1.169.0
Firmware version            : 238.1.168.0
Active NVM config version   : N/A
NVM config version          : 0.0.0
HCRM Profile ID             : 1
HCRM Profile Version        : 1.0.8
Firmware Reset Counter      : 4
Error Recovery Counter      : 0
Crash Dump Timestamp        : N/A
Reboot Required             : No
Secure Boot                 : N/A
Secure Firmware Update      : N/A

Proxmox VE 9.2.21 - BCM57412 bnxt_en network failure after upgrade to kernel 7.0.14-20-pve

loste87

New Member

fabian

Proxmox Staff Member

doklages

Renowned Member

doklages

Renowned Member

We value your privacy