[Solved] Intel X540-T2 - second port not sending or receiving packets

Aterfax

New Member
Mar 21, 2021
16
7
3
30
Hi all,

I have 3 boxes, 2x Proxmox 6.3-6 (1x Xeon E3-1220V2 1x FX 8350) latest intel ixgbe driver and 1x Unraid 6.9.1 built in ixgbe driver (1x Xeon E3-1220V2) all of which seem to have the same issue with the 2nd port being detected, the link state being detected and negotiated to 10GBase correctly but the second ports on all 3 do not appear to send or recieve packets despite having static IPs set.

I'm running with PCIE passthrough of other components on Proxmox but not these 10GBase cards though I doubt IOMMU state is having an effect on things and I cannot turn passthrough off on the Proxmox boxes.

I've installed the latest Intel X540 driver and I have tried connecting various ports from each machine to the other with straight through and cross over cables - makes no difference - the first ports work but none of the second ports do.

I suspect something is wrong with the ixgbe drivers as I have a forth card running in a Windows 10 machine which is fully functional on both ports (and I have swapped cards through the Windows 10 machine and they work fully.)

Anyone got any ideas? I have no idea how to debug this as from the lspci and networking perspective the second ports appear to be fully functional. And the Unraid machine having the same issue is making me think ixgbe is probably the base cause.
 
Last edited:

Stoiko Ivanov

Proxmox Staff Member
Staff member
May 2, 2018
6,982
1,082
164
Anything interesting pointing to what causes the issue in the journal or `dmesg` output after booting up?

Regarding using the latest driver - I assume you mean the out-of-tree driver provided by Intel - I have had mixed experiences with those (meaning that they helped in some situation, but did break things in others) - Maybe try to use the ixgbe module shipped by the PVE kernel.

do the packets you sent out show up in a tcpdump on the machine?

I hope this helps!
 

Aterfax

New Member
Mar 21, 2021
16
7
3
30
There's nothing in the dmesg output or journal files which gives a hint.

Yes I do mean the latest out of tree by Intel - initially I was trying with the built in driver but was seeing the same thing.

Going to have an explore with tcpdump now.

Thanks!
 

Aterfax

New Member
Mar 21, 2021
16
7
3
30
Admittedly I haven't had enough time to dig into this during the week but it looks like the second interfaces were failing to set their IPs correctly (doh) - forcing static IPs again on the command line and I have now got symmetric 9.4Gbit going between all devices.

Thanks for the idea with tcpdump as the NICs were clearly getting something which made me look at the NICs in a bit more detail (they had not got the IPs I had told them to have.)

Much appreciated!
 

Stoiko Ivanov

Proxmox Staff Member
Staff member
May 2, 2018
6,982
1,082
164
Glad that helped :)

Please edit the thread and mark it as 'SOLVED'
Thanks!
 

snpz

Active Member
Mar 18, 2013
33
4
28
Admittedly I haven't had enough time to dig into this during the week but it looks like the second interfaces were failing to set their IPs correctly (doh) - forcing static IPs again on the command line and I have now got symmetric 9.4Gbit going between all devices.

Thanks for the idea with tcpdump as the NICs were clearly getting something which made me look at the NICs in a bit more detail (they had not got the IPs I had told them to have.)

Much appreciated!
And it works as expected after a server reboot?
I still cant find the reason why randomly one or more interfaces are not detected correctly during bootup!
https://forum.proxmox.com/threads/no-network-after-proxmox-kernel-upgrade.86216/
 

Aterfax

New Member
Mar 21, 2021
16
7
3
30
I haven't rebooted yet, will respond again if I do encounter that issue.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!