[SOLVED] Adding ixgbe after install with udev

[sudoblue]

I just installed a SUPERMICRO AOC-STGN-i2S in my 5018D-FN4T.

PCI passthrough is working fine for a VM.

1. How do I add this interface to pve with the system already setup?

I know I can init the lines for a new iface in /network/interfaces, but there has to be more to it than that.

I'm not savvy about udevadm, not sure what the systemd predictable name would be for this card.

Anyone have any experience with adding intel x520 cards after install?

 

[mvs]

Check with lsmod | grep ixgbe command that ixgbe kernel module is loaded.
If it is not loaded, try to load it manually with modprobe ixgbe
Then use ip a command to list NICs

If you get unsupported transceiver error, detach SFP module / DAC cable and reload ixgbe kernel module.
Supermicro card should not have a vendor lock for SFP modules and DACs, but your SFP module / DAC may still be incompatible with ixgbe driver.

 

[sudoblue]

Thanks for the reply @mvs.

lsmod | grep ixgbe
results:
ixgbe 348160 0 xfrm_algo 16384 1 ixgbe dca 16384 3 igb,ioatdma,ixgbe mdio 16384 1 ixgbe


modprobe ixgbe --first-time
results:
modprobe: ERROR: could not insert 'ixgbe': Module already in kernel


ip a

results: device not listed

Regarding my transceiver: I'm using a Juniper EX3300 switch with a Juniper DAC. The network card works fine for a pfSense VM when passing it through. Just not seeing it in pve for some reason.

 

[mvs]

New versions of intel drivers are very picky about transceivers. pfSense is based on FreeBSD and it has probably an older driver.
If you run dmesg | grep ixgbe you will probably see unsupported SFP+ message.

 

[sudoblue]

Thanks again @mvs you're a sanity saver.

There isn't an unsupported SFP+ message below. I don't know what to call the interface so the system understands my reference.

OMG the answer is in the output below. The ixgbe module calls these ens2f0 & 1. I added the iface line and it's working. Lesson Learned

dmesg | grep ixgbe
results:
[ 2.164055] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver [ 2.164058] ixgbe: Copyright (c) 1999-2016 Intel Corporation. [ 3.570361] ixgbe 0000:03:00.0: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16 XDP Queue count = 0 [ 3.696410] ixgbe 0000:03:00.0: MAC: 5, PHY: 7, PBA No: 023B00-000 [ 3.696413] ixgbe 0000:03:00.0: redacted MAC [ 3.750083] ixgbe 0000:03:00.0: Intel(R) 10 Gigabit Network Connection [ 3.750201] libphy: ixgbe-mdio: probed [ 5.522356] ixgbe 0000:03:00.1: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16 XDP Queue count = 0 [ 5.648356] ixgbe 0000:03:00.1: MAC: 5, PHY: 7, PBA No: 023B00-000 [ 5.648366] ixgbe 0000:03:00.1: redacted MAC [ 5.702063] ixgbe 0000:03:00.1: Intel(R) 10 Gigabit Network Connection [ 5.702275] libphy: ixgbe-mdio: probed [5.868682] ixgbe 0000:04:00.0: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16 XDP Queue count = 0 [ 5.868975] ixgbe 0000:04:00.0: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link) [ 5.869058] ixgbe 0000:04:00.0: MAC: 2, PHY: 14, SFP+: 3, PBA No: 400900-000 [ 5.869060] ixgbe 0000:04:00.0: redacted MAC [ 5.870237] ixgbe 0000:04:00.0: Intel(R) 10 Gigabit Network Connection [ 5.870349] libphy: ixgbe-mdio: probed [ 7.052555] ixgbe 0000:04:00.1: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16 XDP Queue count = 0 [ 7.052846] ixgbe 0000:04:00.1: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link) [ 7.052928] ixgbe 0000:04:00.1: MAC: 2, PHY: 1, PBA No: 400900-000 [ 7.052930] ixgbe 0000:04:00.1: redacted MAC [ 7.054060] ixgbe 0000:04:00.1: Intel(R) 10 Gigabit Network Connection [ 7.054126] libphy: ixgbe-mdio: probed [ 7.055389] ixgbe 0000:03:00.1 eno4: renamed from eth1 [ 7.076064] ixgbe 0000:03:00.0 eno3: renamed from eth0 [ 7.096371] ixgbe 0000:04:00.0 ens2f0: renamed from eth2 [ 7.152253] ixgbe 0000:04:00.1 ens2f1: renamed from eth3 [ 11.386494] ixgbe 0000:03:00.0: registered PHC device on eno3 [ 11.689231] ixgbe 0000:03:00.1: registered PHC device on eno4 [ 17.656095] ixgbe 0000:04:00.0: complete [ 17.836179] ixgbe 0000:04:00.1: complete [82823.192171] ixgbe 0000:03:00.1 eno4: NIC Link is Up 1 Gbps, Flow Control: RX/TX

0000:04:00.0 & 1 is the SFP+ card, 0000:03:00.0 & 1 is the 10GBase-T from the SoC.