[SOLVED] Adding HBA card kills my ethernet?

[nertskull]

I have a fresh install of 6.2. It installed fine and I can access it over the network in my home. It's running on a ryzen 3700x and a x570 motherboard.

I have a Dell H200E HBA card. Once I install that on the motherboard and turn on my system. During boot I get flashing lights on my ethernet port like it connects just fine. But by the time proxmox gets to the login screen, the lights are gone and its like no cable is plugged in. And I have zero ethernet connection.

ip add still shows I get an ip address.

If I then take out the HBA card and boot up again, I have ethernet just as expected.

I see no errors in dmesg. But when I do NOT have the HBA card, dmesg shows me

igb XXX:XX:X... enp4so: igb: enp4s0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
vmbr0: port 1(enps40) entered blocking state
vmbr0: port 1(enps40) entered forwarding state
IPv6: ADDRCONF(NETDEV_CHANGE): vmbr0: link becomes ready

And there are a few other lines that look similar. But when I have the HBA card in, I get none of that. And no internet.

How can I trouble shoot this more? Has anyone seen this before?

 

[apoc]

Have you used different slots for the hba? Are slots sharing pcie lanes through a bridge?
Do they perhaps differ in pcie generation? E.g. I think the hba should be gen2 - which is quite old

 

[Stoiko Ivanov]

ip add still shows I get an ip address.

If I then take out the HBA card and boot up again, I have ethernet just as expected.

compare the outputs of `ip link` with and without the HBA card - it can happen that the network device gets renamed due to (not so) predicatable network interface names - https://www.freedesktop.org/wiki/So...Interface Names,but should fix real problems.

I hope this helps!

 

[nertskull]

compare the outputs of `ip link` with and without the HBA card - it can happen that the network device gets renamed due to (not so) predicatable network interface names - https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/#:~:text=Predictable Network Interface Names,but should fix real problems.

I hope this helps!

Yes, it's enp4s0 when the card is plugged in. And enp5s0 when it's not plugged in.

Have you used different slots for the hba? Are slots sharing pcie lanes through a bridge?
Do they perhaps differ in pcie generation? E.g. I think the hba should be gen2 - which is quite old

Same here. When in slot two, it wasn't working. It was enp4s0. But I originally installed proxmox without the card installed at all. Then put the card in slot 3 after install, and I got no ethernet. But if I put it into slot 2, I DID get ethernet.

I re-installed proxmox (since it was a brand new install anyway) with the card inserted into slot pcie16_3. This time it all works as expected.

I'm sure there is a way to have fixed it without re-installing. But this worked.

Thanks so much!!!

 

[drie_keer_ruk]

Just wanted to add that I just solved this problem without reinstalling. With and without the HBA card using command:

ip link

I was able to find out that the network card that the nic was renamed. In my case from enp3s0 to enp4s0. Using:

nano /etc/network/interfaces

and changing the nic name to the new ones everything started working after a:

/etc/init.d/networking restart

 

[MisterY]

I have a similar problem:

I installed Proxmox 7.4 on my Supermicro X9SRI-F (8.1 resulted in "frequency out of range" on the video output). and everything worked fine. Then I took my HBA (Dell Perc H310 flashed to IT mode) from another server and placed it in the lower-most (PCI-E 3.0 8x) slot. Now my NICs aren't detected anymore, except IPMI. Linux can't see them (when I try to install proxmox again it tells me "no network interface found"); they even don't show up under lspci. When I remove the HBA then it works fine again.

Any ideas what the problem is?

Something I found out:

lsipci -v gives me this:

04:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2008 PCI-Express Fusion
        DeviceName: Onboard Intel i350 Ethernet

I think there is something wrong with the IOMMU Group.

 

[apoc]

I'd try to use a different PCIe-Slot.
Also try to disable VT-d or similar virtualization technologies and all unneccessary IO-ports (e.g. serial connectors).

 

[MisterY]

I found a solution: enable bifurcation to 8x8x on this particular slot.