[SOLVED] How to enable SR-IOV on Emulex oce14102

[SunTsu]

I tried to enable SR-IOV on emulex NIC oce14102.

x370-pro, ryzen 1700x

ARI enable, sr-iov enable

In bios oce14102 sr-iov enable

root@plrg:~# cat /sys/class/net/enp4s0f0/device/sriov_totalvfs

31

root@plrg:~# cat /sys/class/net/enp4s0f0/device/sriov_numvfs

0

I can create 31 vfs/nic



But when I do:

echo 2 > /sys/class/net/enp4s0f0/device/sriov_numvfs

I get dmesg:

[ 119.333371] pci 0000:04:04.0: unknown header type 7f, ignoring device

[ 120.352982] be2net 0000:04:00.0: SRIOV enable failed

[ 120.352986] be2net 0000:04:00.0: VF setup failed

[ 120.352998] be2net 0000:04:00.0: 2 VFs requested; only 0 enabled

I can't enable SR-IOV. I always get error.



My lspci output:

04:00.1 Ethernet controller: Emulex Corporation OneConnect NIC (Skyhawk) (rev 10)

Subsystem: Emulex Corporation OneConnect NIC (Skyhawk)

Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+

Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

Latency: 0, Cache Line Size: 64 bytes

Interrupt: pin B routed to IRQ 51

Region 0: Memory at f0580000 (64-bit, prefetchable) [size=16K]

Region 2: Memory at f0320000 (64-bit, prefetchable) [size=128K]

Region 4: Memory at f0300000 (64-bit, prefetchable) [size=128K]

Expansion ROM at f6400000 [disabled] [size=512K]

Capabilities: [40] Power Management version 3

Flags: PMEClk- DSI+ D1- D2- AuxCurrent=375mA PME(D0-,D1-,D2-,D3hot+,D3cold+)

Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-

Capabilities: [48] MSI-X: Enable+ Count=32 Masked-

Vector table: BAR=0 offset=00002000

PBA: BAR=0 offset=00003000

Capabilities: [c0] Express (v2) Endpoint, MSI 00

DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <1us, L1 <16us

ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 26.000W

DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+

RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-

MaxPayload 512 bytes, MaxReadReq 512 bytes

DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-

LnkCap: Port #0, Speed 8GT/s, Width x8, ASPM L0s L1, Exit Latency L0s <2us, L1 <16us

ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+

LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+

ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-

LnkSta: Speed 5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported

DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled

LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-

EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-

Capabilities: [b8] Vital Product Data

Product Name: Emulex VFA5 2x10 GbE SFP+ PCIe Adapter for IBM System x, NIC PF

Read-only fields:

[PN] Part number: 00JY820

[SN] Serial number: 11S00JY822Y652HY58M0KL

[V0] Vendor specific: FC53379011

[EC] Engineering changes: N33982

[FC] Unknown: 59 4f 59 4f

[VJ] Vendor specific: 001A

[VL] Vendor specific: 00JY823

[VN] Vendor specific: Y652HY58M0KL

[VR] Vendor specific: 22

[VS] Vendor specific: 32773

[VT] Vendor specific: 9

[VU] Vendor specific: 597E35CF322240CCADD30090FAB64572

[V1] Vendor specific: IBM 00JY822 Emulex VFA5 2x10 GbE SFP+ PCIe Adapter

[V2] Vendor specific: 00JY822

[V5] Vendor specific: OCm14102-UX-X

[V7] Vendor specific: ID:0242,V:00.00.01.01

[RV] Reserved: checksum good, 59 byte(s) reserved

End

Capabilities: [100 v1] Advanced Error Reporting

UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-

UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-

UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-

CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+

CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+

AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-

Capabilities: [180 v1] Single Root I/O Virtualization (SR-IOV)

IOVCap: Migration-, Interrupt Message Number: 000

IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy-

IOVSta: Migration-

Initial VFs: 32, Total VFs: 32, Number of VFs: 0, Function Dependency Link: 01

VF offset: 63, stride: 1, Device ID: 0720

Supported Page Size: 00000557, System Page Size: 00000001

Region 0: Memory at 00000000f0380000 (64-bit, prefetchable)

VF Migration: offset: 00000000, BIR: 0

Capabilities: [160 v1] Alternative Routing-ID Interpretation (ARI)

ARICap: MFVC- ACS-, Next Function: 0

ARICtl: MFVC- ACS-, Function Group: 0

Capabilities: [168 v1] Device Serial Number 00-90-fa-ff-fe-b6-45-72

Kernel driver in use: be2net

Kernel modules: be2net

Grub output :

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt pcie_acs_override=downstream,multifunction transparent_hugepage=madvise video=efifb:off rcu_nocbs=8-15 nohz_full=8-15 pcie_aspm=off "

 

[Stoiko Ivanov]

Sadly don't have one of those cards around to test this - but according to https://www.slideshare.net/juet-y/sr-iovemulexdebian10 ( a presentation from 2014) the sysfs for this driver does not work - you need to set the number of VFs when loading the module (with modprobe, and if this works with a file in /etc/modprobe.d)

I hope this helps!

 

[SunTsu]

It didn't work dmesg give info: command unknown - ignoring.
Also if I use sysfs -> "echo 0 > /sys/class/net/enp11s0f1/device/sriov_numvfs" than echo N+1 > /sys/class/net/enp11s0f1/device/sriov_numvfs it's ingnoring first command (since it's already 0)....

Problem was my motherboard third pci-e slot. It has 4 lines only - NIC need 8.
I fixed by swap NIC to second slot and quadro P400 to third slot.
Now everything is working.

 

[Stoiko Ivanov]

Glad that you found the culprit! Thanks for marking the thread as SOLVED!

 

[afrojewelz]

It didn't work dmesg give info: command unknown - ignoring.
Also if I use sysfs -> "echo 0 > /sys/class/net/enp11s0f1/device/sriov_numvfs" than echo N+1 > /sys/class/net/enp11s0f1/device/sriov_numvfs it's ingnoring first command (since it's already 0)....

Problem was my motherboard third pci-e slot. It has 4 lines only - NIC need 8.
I fixed by swap NIC to second slot and quadro P400 to third slot.
Now everything is working.

is it sriov really working?
my skyhawk NIC same brand same model is not working either on my z170 motherboard even when it plugin in PCIE X16 slot. i wonder what kind of board should this NIC will apply sriov smoothly?