Proxmox VE and Intel i210-AT NIC: Repeating communication losses

[TheExpert]

Hi all,

I'm new to Proxmox VE. I'm testing it to see if I can switch over from VMware Hypervisor (free ESXi 8.0.2). So installed Proxmox VE 8.1 on a server with a Supermicro X10SLM-F mainboard. There are 2 onboard NICs from Intel: i210-AT and i217-LM.

The i210-AT onboard NICs worked with VMware ESXi without any issues but with Proxmox VE I've repeating communication losses to the management address when connected to the i210-AT. On the same interface there's the IPMI connected and this has no communication losses. Both IP adresses are on the same network. Pinging the IPMI has no packet losses but pinging the management of Proxmox VE has.

Now I switched the management interface to one of the separate HP NC-360T PCIe cards and once again to the onboard i217-LM and there aren't any communication losses.

The driver for i210-AT is igb, for the other ones it's e1000e.

lspci | grep Ethernet
00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-LM (rev 05)
01:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller D0/D1 (copper applications) (rev 06)
01:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller D0/D1 (copper applications) (rev 06)
02:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller D0/D1 (copper applications) (rev 06)
02:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller D0/D1 (copper applications) (rev 06)
05:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

Does anybody knows this issue? Did you resolved this and how?

Kind Regards

 

[zzz09700]

My bet is on Supermicro messed up again. We also experienced unexplainable connectivity issues on setups similar to what OP has so we stay away from shared IPMI LAN ports now.

 

[TheExpert]

There's a dedicated IPMI LAN port, too. I'm surprised that the IPMI also uses the i210-AT. So, you solved the issue by connecting the IPMI to the dedicated LAN port and the i210-AT then worked without issues? This is how I have connected my running ESXi hosts. Proxmox is installed on a test hardware and so I didn't connect all LAN lines.

 

[zzz09700]

So, you solved the issue by connecting the IPMI to the dedicated LAN port and the i210-AT then worked without issues?

Yes. We do have another configuration that IPMI uses the one of the onboard NIC and the host OS uses an add-on network card. Both works without issue.

 

[TheExpert]

OK, so I tested now with using the dedicated LAN port for the IPMI. But this doesn't help. I'm losing packets on the i210-AT when pinging the management address of the PVE server. I never had these issues with ESXi! The i210-AT is a failover interface for the IPMI. I can't change this in BIOS.

 

[TheExpert]

And even after setting the LAN port in the IPMI configuration to dedicated only I'm losing ping packets. I think there's an issue with the igb driver.

 

[zzz09700]

And even after setting the LAN port in IPMI to dedicated only I'm losing ping packets. I think there's an issue with the igb driver.

Or you could have hardware problems. Try another OS to see if anything changes maybe?

 

[TheExpert]

As already said: With VMware ESXi I never had this issue on this hardware.

 

[TheExpert]

And I've another system with the same mainboard. On this system I have XPEnology running (based on ARC) which doesn't have any issues regarding the i210-AT network card.

 

[TheExpert]

How to proceed to solve this issue with the onboard i210-AT? I don't want to by new additional network cards because of no possibility to use the onboard NICs, which are working fine with XPEnology and VMware ESXi.

 

[TheExpert]

No ideas?

 

[TheExpert]

Are there any news regarding this issue?

 

[WaaromZoMoeilijk]

Same issue here with 3x supermicro server, all the 2x onboard I210's are flakey and drop at random or on load.
I'm using a dedicated IPMI port (onboard, also I210) different from the 2x I210's on each server and each I210 port has issues.

No fix yet, if I get one, I'll post here.

 

[TheExpert]

I don't use the i210 as IPMI port. But even if I set up IPMI to not to use the i210 port as failover there are packet drops. I don't undestand why Proxmox isn't solving this issue. Supermicro boards and i210 NICs are often used. And other OS don't have this issue.

 

[WaaromZoMoeilijk]

What I did was set the IPMI port to dedicated, not failover or shared, even then it won't work.

Now trying the drivers from intels website and/or kernel downgrade.

 

[TheExpert]

Can you update this thread if you have new findings, please?

 

[WaaromZoMoeilijk]

Hey, so there's been some progress here, changed

/etc/default/grub

with the parameters

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash pcie_port_pm=off pcie_aspm.policy=performance"

Then execute

proxmox-boot-tool refresh && reboot

and where I normally straight away had console output of NIC's going up and down and changing speed from 1000M to 10M I now have a sanitized console login prompt.
This however is by no mains a confirmed fix/workaround, will let it hum along for a couple of days and report back.

Source:
https://old.reddit.com/r/buildapc/comments/xypn1m/network_card_intel_ethernet_controller_i225v_igc/

Edit after day 1, so far so good!

 

[TheExpert]

Is everything still so far so good?

 

[WaaromZoMoeilijk]

Hey,

So far I had one kernel panic related but other than that it's been solid.
Can't say what initiated it though...

 

[TheExpert]

Hm, today I tried your workaround but it doesn't seem to work for me. I've still network connection timeouts .