Machine looses 10G network devices

[hjdt]

Hey all,

since two weeks our new Proxmox machine is running but every 2-3 days the host is offline because it has taken the NIC offline. We use a 10GB Intel X550T2. The card runs with the igxbe module. Otherwise we have the latest Proxmox Enterprise version. The specific error message is:

Jun 15 04:51:29 pve kernel: [237072.155524] ixgbe 0000:b3:00.1: Adapter removed
Jun 15 04:51:29 pve kernel: [237072.155549] ixgbe 0000:b3:00.1: Warning firmware error detected FWSM: 0xFFFFFFFF
Jun 15 04:51:29 pve kernel: [237072.155551] ixgbe 0000:b3:00.1: Firmware recovery mode detected. Limiting functionality. Refer to the Intel(R) Ethernet Adapters and Devices User Guide for d
etails on firmware recovery mode.
Jun 15 04:51:29 pve kernel: [237072.156607] ixgbe 0000:b3:00.1: removed PHC on enp179s0f1
Jun 15 04:51:29 pve kernel: [237072.365167] bond0: (slave enp179s0f1): Releasing backup interface
Jun 15 04:51:29 pve kernel: [237072.424326] ixgbe 0000:b3:00.0: Adapter removed
Jun 15 04:51:29 pve kernel: [237072.424352] ixgbe 0000:b3:00.0: Warning firmware error detected FWSM: 0xFFFFFFFF
Jun 15 04:51:29 pve kernel: [237072.424354] ixgbe 0000:b3:00.0: Firmware recovery mode detected. Limiting functionality. Refer to the Intel(R) Ethernet Adapters and Devices User Guide for details on firmware recovery mode.

Any thoughts how to solve this issue?

Boris

 

[Neobin]

Is the firmware [1] up-to-date?

[1] https://www.intel.com/content/www/u...intel-ethernet-adapters-550-series-linux.html

 

[hjdt]

No the firmware isn't update to date. But I can't flush it, I always get the error "Flash failed" with no further information. Do I need to flash it in rescue mode or so?

 

[Neobin]

In which state did you try it?
Was it in the "broken" state with the error you posted above or was it in a (again) functional state?

Edit: If it is a branded card (HP, Dell, ...) you might need to look for a firmware update from them.

You could try different things like:

• Completely power-cycle the host and try the update again.
• Put the card in another machine and see if the update works there. (Maybe even a Windows machine with the latest NIC drivers from Intel.)

But since it is a new machine, it might make more sense to simply contact the vendor/distributor of it and see what he has to say and probably exchange the card (optimally with a already up-to-date firmware), at least for testing.

I can only say, that my X550-T2's are running absolutely fine since ~2 years (PVE 6.x up to the actual PVE 7.2) with all the different kernels and (most of the time up-to-date) firmware versions.

 

[bobmc]

Is the card in a server chassis - like dell/hp or supermicro? those cards do run hot and need adequate cooling to be reliable

 

[hjdt]

Yes the card is in a supermicro chassis 825TQC−R802LPB. There are different coolers but what can I do more?

 

[hjdt]

In which state did you try it?
Was it in the "broken" state with the error you posted above or was it in a (again) functional state?

Different times I tested it in functional state after a reboot.

Edit: If it is a branded card (HP, Dell, ...) you might need to look for a firmware update from them.

No it's a Intel version

You could try different things like:

• Completely power-cycle the host and try the update again.

Done but without success

• Put the card in another machine and see if the update works there. (Maybe even a Windows machine with the latest NIC drivers from Intel.)

But since it is a new machine, it might make more sense to simply contact the vendor/distributor of it and see what he has to say and probably exchange the card (optimally with a already up-to-date firmware), at least for testing.

I try to contact Intel with my issue