ixgbe initialize fails

[Stoiko Ivanov]

here still failed!

# lspci |grep Eth
......
06:00.0 Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF Dual Port Network Connection (rev 01)
06:00.1 Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF Dual Port Network Connection (rev 01)

[    1.149071] Intel(R) 10GbE PCI Express Linux Network Driver - version 5.3.7
[    1.149071] Copyright(c) 1999 - 2018 Intel Corporation.
[    1.149377] ixgbe 0000:06:00.0: can't disable ASPM; OS doesn't have ASPM control
[    1.233292] ixgbe 0000:06:00.0: failed to load because an unsupported SFP+ or QSFP module type was detected.
[    1.233356] ixgbe 0000:06:00.0: Reload the driver after installing a supported module.

The problem could be related to that message - unsupported sftp(+) modules.
The OOT driver still has the option (I think it's not present anymore in the intree version).

adding the option to the module's parameters might help:

echo "options ixgbe allow_unsupported_sfp=1,1" > /etc/modprobe.d/ixgbe-options.conf

(one 1 for each interface in the host)

 

[lynn_yudi]

echo "options ixgbe allow_unsupported_sfp=1,1" > /etc/modprobe.d/ixgbe-options.conf
[/CODE]
(one 1 for each interface in the host)

No success! Thanks

 

[Radapompa]

Tested the allow_unsupported approach aswell on my end. But it didn't work for me either.

I have four interfaces, so I tried the string 1,1,1,1. But got that it was an invalid value. Tried with only 1, and then it loaded. But still got the probe failed with error -5 in dmesg.

Also according to this bugreport https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1245938. There are three reasons that why this issue could occur. And I know these interfaces worked on these servers before. So I would assume that it fails to io map (ioremap).



Below is output for one of the interfaces with lspci -vvv.

26:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
        Subsystem: Intel Corporation Ethernet Server Adapter X520-2
        Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin B routed to IRQ 47
        NUMA node: 6
        Region 0: Memory at <ignored> (64-bit, non-prefetchable)
        Region 2: I/O ports at 8ce0 [size=32]
        Region 4: Memory at <ignored> (64-bit, non-prefetchable)
        Expansion ROM at <ignored> [disabled]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
                Address: 0000000000000000  Data: 0000
                Masking: 00000000  Pending: 00000000
        Capabilities: [70] MSI-X: Enable- Count=64 Masked-
                Vector table: BAR=4 offset=00000000
                PBA: BAR=4 offset=00002000
        Capabilities: [a0] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
                DevCtl: Report errors: Correctable- Non-Fatal+ Fatal+ Unsupported+
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s, Exit Latency L0s <1us, L1 <8us
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
                DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [e0] Vital Product Data
                Product Name: Intel(R) Ethernet Server Adapter X520-2
                Read-only fields:
                        [MN] Manufacture ID: 31 30 32 38
                        [V0] Vendor specific: FFV16.5.20
                        [RV] Reserved: checksum good, 1 byte(s) reserved
                End
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
        Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-9c-f7-70
        Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
                ARICap: MFVC- ACS-, Next Function: 0
                ARICtl: MFVC- ACS-, Function Group: 0
        Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
                IOVCap: Migration-, Interrupt Message Number: 000
                IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy-
                IOVSta: Migration-
                Initial VFs: 64, Total VFs: 64, Number of VFs: 0, Function Dependency Link: 01
                VF offset: 128, stride: 2, Device ID: 10ed
                Supported Page Size: 00000553, System Page Size: 00000001
                Region 0: Memory at 0000000000000000 (64-bit, non-prefetchable)
                Region 3: Memory at 0000000000000000 (64-bit, non-prefetchable)
                VF Migration: offset: 00000000, BIR: 0
        Kernel modules: ixgbe

 

[Radapompa]

Finally found a workaround for this.

On these servers. We had one embedded 4x 1Gbit RJ45 network, one Intel 4x 1Gbit RJ45 network and two 2x Intel 10Gbit SFP+ cards. So the embedded RJ45 and the Intel RJ45 card worked previously, but not the two SFP+ cards.

But after removing the Intel RJ45 card from a server. The two SFP+ cards started to work. So apparently the Intel RJ45 card disrupted the two SFP+ cards.

I'm not sure if this could be an issue with having different Intel network cards which are using different drivers that caused this. But we didn't need to remove the RJ45 card in the previous virtualization platform that we used. So I think, which also Intel developers pointed towards, is that there is an issue in Proxmox kernel for this.

I noticed that there was another forum post with a similar issue with a similar solution. So might be worth for developers to investigate more if possible.
https://forum.proxmox.com/threads/problem-with-ixgbe-sfp-module-for-networking.52419/#post-242961