Another X520-DA2 thread, but maybe different...

[h2o]

Dell T350 server, ProxMox 8.4.0. It's been working fine. Today I added an eBay purchased Dell OEM Intel X520-DA2 NIC to the server and interfaced it to a Cisco CBS250-24P-4X SPF+ port switch with DAC cable that should be Cisco and Intel compatible. In ProxMox the NIC lists both SPF+ ports as Active=No. I'm somewhat new to ProxMox but I think that means no link. There are lots of Googles about similar topics but they are related to incompatible SPF+ modules or the NIC being locked down to specific ones. Thanks for any help.

I don't think the DAC cable is the issue here for the following reasons.

• dmsg does not contain "unsupported SFP+ module" errors
• The switch shows a 10G link
• The NIC link LED is illuminated
• It works fine when booted from Linux USB (Mint 22)

Some things I tried...

• A different X520-DA2 card (purchased at the same time)
• Updated NIC firmware (Dell OEM latest)
• Other X520-DA2 port
• Different DAC cable
• Different switch ports
• allow_unsupported_sfp=1 (grasping at straws)

Here's some output (ens4f0 and ens4f1 are the X520-DA2).

root@pve:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno8303: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether c4:5a:b1:a7:97:af brd ff:ff:ff:ff:ff:ff
    altname enp6s0f0
3: eno8403: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether c4:5a:b1:a7:97:b0 brd ff:ff:ff:ff:ff:ff
    altname enp6s0f1
4: ens4f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether b4:96:91:39:ca:a0 brd ff:ff:ff:ff:ff:ff
    altname enp7s0f0
5: ens4f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether b4:96:91:39:ca:a2 brd ff:ff:ff:ff:ff:ff
    altname enp7s0f1
6: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether c4:5a:b1:a7:97:af brd ff:ff:ff:ff:ff:ff

root@pve:~# dmesg |grep ixgbe
[    0.853127] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver
[    0.853129] ixgbe: Copyright (c) 1999-2016 Intel Corporation.
[    1.032502] ixgbe 0000:07:00.0: Multiqueue Enabled: Rx Queue count = 12, Tx Queue count = 12 XDP Queue count = 0
[    1.032807] ixgbe 0000:07:00.0: 16.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x4 link at 0000:00:1d.0 (capable of 32.000 Gb/s with 5.0 GT/s PCIe x8 link)
[    1.033140] ixgbe 0000:07:00.0: MAC: 2, PHY: 14, SFP+: 3, PBA No: G73131-009
[    1.033141] ixgbe 0000:07:00.0: b4:96:91:39:ca:a0
[    1.035541] ixgbe 0000:07:00.0: Intel(R) 10 Gigabit Network Connection
[    2.183596] ixgbe 0000:07:00.1: Multiqueue Enabled: Rx Queue count = 12, Tx Queue count = 12 XDP Queue count = 0
[    2.183901] ixgbe 0000:07:00.1: 16.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x4 link at 0000:00:1d.0 (capable of 32.000 Gb/s with 5.0 GT/s PCIe x8 link)
[    2.184232] ixgbe 0000:07:00.1: MAC: 2, PHY: 1, PBA No: G73131-009
[    2.184233] ixgbe 0000:07:00.1: b4:96:91:39:ca:a2
[    2.186626] ixgbe 0000:07:00.1: Intel(R) 10 Gigabit Network Connection
[    2.205497] ixgbe 0000:07:00.1 ens4f1: renamed from eth0
[    2.214649] ixgbe 0000:07:00.0 ens4f0: renamed from eth2

root@pve:~# lspci -k | sed -n '/Ethernet/,/driver in use/p'
06:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe
    DeviceName: Embedded NIC 1
    Subsystem: Dell NetXtreme BCM5720 Gigabit Ethernet PCIe
    Kernel driver in use: tg3
06:00.1 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe
    DeviceName: Embedded NIC 2
    Subsystem: Dell NetXtreme BCM5720 Gigabit Ethernet PCIe
    Kernel driver in use: tg3
07:00.0 Ethernet controller: Intel Corporation Ethernet 10G 2P X520 Adapter (rev 01)
    Subsystem: Intel Corporation 10GbE 2P X520 Adapter
    Kernel driver in use: ixgbe
07:00.1 Ethernet controller: Intel Corporation Ethernet 10G 2P X520 Adapter (rev 01)
    Subsystem: Intel Corporation 10GbE 2P X520 Adapter
    Kernel driver in use: ixgbe

 

[h2o]

One bump in case anyone has thoughts on this. Thanks.

 

[MarkusKo]

What's your current network config (/etc/network/interfaces) ?
Are the Intel NIC ports not part of a bridge?
I only see one Broadcom NIC port part of your vmbr0 bridge

 

[PwrBank]

This may or may not apply here, but on TrueNAS a similar thing will happen if you don't assign an IP address to the NIC, either via a bridge or on the NIC itself. Try adding an IP to it, or add it to a bridge?

 

[h2o]

Fixed, that was it. I created a Linux Bridge, assigned NIC, added an IP, applied changes. I was stuck because I thought "active" reflected the link status. Thank you.