I recently installed the latest proxmox (8.3.0) on a Supermicro X10SDV-TLN4F board
The motherboard has 4 ethernet ports: 2 x 1Gb (Intel i350) and 2 x 10Gb (X552/X557-AT)
ip addr only shows the 2 x 1Gb cards
but the cards are visible with lspci
I've looked at several of the threads here, with this one being the most relevant: [SOLVED] intel x553 SFP+ ixgbe no go on PVE8
As part of the things discussed in the thread, I've manually installed the latest drivers as written in this answer
the driver seems to load fine (i believe the message on module verification can be discarded)
I've also tried to manually create the interfaces in
I do not know what else I can/should explore. From the thread above, this issue should have been resolved in the latest kernel, which I'm already running.
Any help/pointers will be appreciated
Bash:
root@pve:~# pveversion
pve-manager/8.3.0/c1689ccb1065a83b (running kernel: 6.8.12-4-pve)The motherboard has 4 ethernet ports: 2 x 1Gb (Intel i350) and 2 x 10Gb (X552/X557-AT)
ip addr only shows the 2 x 1Gb cards
Bash:
root@pve:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
link/ether ac:1f:6b:6a:5a:c8 brd ff:ff:ff:ff:ff:ff
altname enp5s0f0
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether ac:1f:6b:6a:5a:c9 brd ff:ff:ff:ff:ff:ff
altname enp5s0f1
4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether ac:1f:6b:6a:5a:c8 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.99/24 scope global vmbr0
valid_lft forever preferred_lft forever
inet6 fe80::ae1f:6bff:fe6a:5ac8/64 scope link
valid_lft forever preferred_lft foreverbut the cards are visible with lspci
Bash:
03:00.1 Ethernet controller: Intel Corporation Ethernet Connection X552/X557-AT 10GBASE-T
DeviceName: Intel X557-AT2 Ethernet #2
Subsystem: Super Micro Computer Inc Ethernet Connection X552/X557-AT 10GBASE-T
Physical Slot: 0-1
Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin B routed to IRQ 17
IOMMU group: 28
Region 0: Memory at fba00000 (64-bit, prefetchable) [size=2M]
Region 4: Memory at fbe00000 (64-bit, prefetchable) [size=16K]
Expansion ROM at 90180000 [disabled] [size=512K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] MSI-X: Enable- Count=64 Masked-
Vector table: BAR=4 offset=00000000
PBA: BAR=4 offset=00002000
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- FLReset-
MaxPayload 256 bytes, MaxReadReq 4096 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [140 v1] Device Serial Number 00-00-c9-ff-ff-00-00-00
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 0
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration- 10BitTagReq- Interrupt Message Number: 000
IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy- 10BitTagReq-
IOVSta: Migration-
Initial VFs: 64, Total VFs: 64, Number of VFs: 0, Function Dependency Link: 01
VF offset: 128, stride: 2, Device ID: 15a8
Supported Page Size: 00000553, System Page Size: 00000001
Region 0: Memory at 0000000090400000 (64-bit, non-prefetchable)
Region 3: Memory at 0000000090500000 (64-bit, non-prefetchable)
VF Migration: offset: 00000000, BIR: 0
Capabilities: [1b0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel modules: ixgbeI've looked at several of the threads here, with this one being the most relevant: [SOLVED] intel x553 SFP+ ixgbe no go on PVE8
As part of the things discussed in the thread, I've manually installed the latest drivers as written in this answer
Bash:
root@pve:~# dmesg | grep ixgbe
[ 1.888827] ixgbe: loading out-of-tree module taints kernel.
[ 1.888848] ixgbe: module verification failed: signature and/or required key missing - tainting kernel
[ 1.915861] ixgbe 0000:03:00.0 0000:03:00.0 (uninitialized): ixgbe_validate_register_read: register 15f74 read unchanged
[ 1.928484] ixgbe 0000:03:00.0: HW Init failed: -17
[ 1.928647] ixgbe: probe of 0000:03:00.0 failed with error -17
[ 1.928770] ixgbe 0000:03:00.1 0000:03:00.1 (uninitialized): ixgbe_validate_register_read: register 15f74 read unchanged
[ 1.940490] ixgbe 0000:03:00.1: HW Init failed: -17
[ 1.940623] ixgbe: probe of 0000:03:00.1 failed with error -17
root@pve:~# modinfo ixgbe
filename: /lib/modules/6.8.12-4-pve/updates/drivers/net/ethernet/intel/ixgbe/ixgbe.ko
version: 5.21.5
license: GPL
description: Intel(R) 10GbE PCI Express Linux Network Driver
author: Intel Corporation, <linux.nics@intel.com>
srcversion: FBDC6774AD96393E4A46447
alias: pci:v00008086d000057B2sv*sd*bc*sc*i*
alias: pci:v00008086d000057B1sv*sd*bc*sc*i*
alias: pci:v00008086d000057B0sv*sd*bc*sc*i*
alias: pci:v00008086d000057AFsv*sd*bc*sc*i*
alias: pci:v00008086d000057AEsv*sd*bc*sc*i*
alias: pci:v00008086d000015E5sv*sd*bc*sc*i*
alias: pci:v00008086d000015E4sv*sd*bc*sc*i*
alias: pci:v00008086d000015CEsv*sd*bc*sc*i*
alias: pci:v00008086d000015CCsv*sd*bc*sc*i*
alias: pci:v00008086d000015CAsv*sd*bc*sc*i*
alias: pci:v00008086d000015C8sv*sd*bc*sc*i*
alias: pci:v00008086d000015C7sv*sd*bc*sc*i*
alias: pci:v00008086d000015C6sv*sd*bc*sc*i*
alias: pci:v00008086d000015C4sv*sd*bc*sc*i*
alias: pci:v00008086d000015C3sv*sd*bc*sc*i*
alias: pci:v00008086d000015C2sv*sd*bc*sc*i*
alias: pci:v00008086d000015AEsv*sd*bc*sc*i*
alias: pci:v00008086d000015ADsv*sd*bc*sc*i*
alias: pci:v00008086d000015ACsv*sd*bc*sc*i*
alias: pci:v00008086d000015ABsv*sd*bc*sc*i*
alias: pci:v00008086d000015B0sv*sd*bc*sc*i*
alias: pci:v00008086d000015AAsv*sd*bc*sc*i*
alias: pci:v00008086d000015D1sv*sd*bc*sc*i*
alias: pci:v00008086d00001563sv*sd*bc*sc*i*
alias: pci:v00008086d00001560sv*sd*bc*sc*i*
alias: pci:v00008086d00001558sv*sd*bc*sc*i*
alias: pci:v00008086d0000154Asv*sd*bc*sc*i*
alias: pci:v00008086d00001557sv*sd*bc*sc*i*
alias: pci:v00008086d0000154Dsv*sd*bc*sc*i*
alias: pci:v00008086d00001528sv*sd*bc*sc*i*
alias: pci:v00008086d000010F8sv*sd*bc*sc*i*
alias: pci:v00008086d0000151Csv*sd*bc*sc*i*
alias: pci:v00008086d00001529sv*sd*bc*sc*i*
alias: pci:v00008086d0000152Asv*sd*bc*sc*i*
alias: pci:v00008086d000010F9sv*sd*bc*sc*i*
alias: pci:v00008086d00001514sv*sd*bc*sc*i*
alias: pci:v00008086d00001507sv*sd*bc*sc*i*
alias: pci:v00008086d000010FBsv*sd*bc*sc*i*
alias: pci:v00008086d00001517sv*sd*bc*sc*i*
alias: pci:v00008086d000010FCsv*sd*bc*sc*i*
alias: pci:v00008086d000010F7sv*sd*bc*sc*i*
alias: pci:v00008086d00001508sv*sd*bc*sc*i*
alias: pci:v00008086d000010DBsv*sd*bc*sc*i*
alias: pci:v00008086d000010F4sv*sd*bc*sc*i*
alias: pci:v00008086d000010E1sv*sd*bc*sc*i*
alias: pci:v00008086d000010F1sv*sd*bc*sc*i*
alias: pci:v00008086d000010ECsv*sd*bc*sc*i*
alias: pci:v00008086d000010DDsv*sd*bc*sc*i*
alias: pci:v00008086d0000150Bsv*sd*bc*sc*i*
alias: pci:v00008086d000010C8sv*sd*bc*sc*i*
alias: pci:v00008086d000010C7sv*sd*bc*sc*i*
alias: pci:v00008086d000010C6sv*sd*bc*sc*i*
alias: pci:v00008086d000010B6sv*sd*bc*sc*i*
depends: dca
retpoline: Y
name: ixgbe
vermagic: 6.8.12-4-pve SMP preempt mod_unload modversions
parm: IntMode:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-X), default 2 (array of int)
parm: InterruptType:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-X), default IntMode (deprecated) (array of int)
parm: MQ:Disable or enable Multiple Queues, default 1 (array of int)
parm: DCA:Disable or enable Direct Cache Access, 0=disabled, 1=descriptor only, 2=descriptor and data (array of int)
parm: RSS:Number of Receive-Side Scaling Descriptor Queues, default 0=number of cpus (array of int)
parm: VMDQ:Number of Virtual Machine Device Queues: 0/1 = disable (1 queue) 2-16 enable (default=8) (array of int)
parm: max_vfs:Number of Virtual Functions: 0 = disable (default), 1-63 = enable this many VFs (array of int)
parm: VEPA:VEPA Bridge Mode: 0 = VEB (default), 1 = VEPA (array of int)
parm: InterruptThrottleRate:Maximum interrupts per second, per vector, (0,1,956-488281), default 1 (array of int)
parm: LLIPort:Low Latency Interrupt TCP Port (0-65535) (array of int)
parm: LLIPush:Low Latency Interrupt on TCP Push flag (0,1) (array of int)
parm: LLISize:Low Latency Interrupt on Packet Size (0-1500) (array of int)
parm: LLIEType:Low Latency Interrupt Ethernet Protocol Type (array of int)
parm: LLIVLANP:Low Latency Interrupt on VLAN priority threshold (array of int)
parm: FdirPballoc:Flow Director packet buffer allocation level:
1 = 8k hash filters or 2k perfect filters
2 = 16k hash filters or 4k perfect filters
3 = 32k hash filters or 8k perfect filters (array of int)
parm: AtrSampleRate:Software ATR Tx packet sample rate (array of int)
parm: FCoE:Disable or enable FCoE Offload, default 1 (array of int)
parm: MDD:Malicious Driver Detection: (0,1), default 1 = on (array of int)
parm: LRO:Large Receive Offload (0,1), default 0 = off (array of int)
parm: allow_unsupported_sfp:Allow unsupported and untested SFP+ modules on 82599 based adapters, default 0 = Disable (array of int)
parm: dmac_watchdog:DMA coalescing watchdog in microseconds (0,41-10000), default 0 = off (array of int)
parm: vxlan_rx:VXLAN receive checksum offload (0,1), default 1 = Enable (array of int)
parm: fwlog_method:FW event logging method. 0=ARQ event logging, 1=UART event logging
(ushort)
root@pve:~# /sbin/modinfo ixgbe | grep version
version: 5.21.5
srcversion: FBDC6774AD96393E4A46447
vermagic: 6.8.12-4-pve SMP preempt mod_unload modversionsthe driver seems to load fine (i believe the message on module verification can be discarded)
Bash:
root@pve:~# dmesg | grep ixgbe
[ 1.888827] ixgbe: loading out-of-tree module taints kernel.
[ 1.888848] ixgbe: module verification failed: signature and/or required key missing - tainting kernel
[ 1.915861] ixgbe 0000:03:00.0 0000:03:00.0 (uninitialized): ixgbe_validate_register_read: register 15f74 read unchanged
[ 1.928484] ixgbe 0000:03:00.0: HW Init failed: -17
[ 1.928647] ixgbe: probe of 0000:03:00.0 failed with error -17
[ 1.928770] ixgbe 0000:03:00.1 0000:03:00.1 (uninitialized): ixgbe_validate_register_read: register 15f74 read unchanged
[ 1.940490] ixgbe 0000:03:00.1: HW Init failed: -17
[ 1.940623] ixgbe: probe of 0000:03:00.1 failed with error -17I've also tried to manually create the interfaces in
/etc/network/interfaces, but it still fails to recognize the network interfaces.
Bash:
root@pve:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback
iface eno1 inet manual
auto vmbr0
iface vmbr0 inet static
address 192.168.1.99/24
gateway 192.168.1.1
bridge-ports eno1
bridge-stp off
bridge-fd 0
iface eno2 inet manual
iface eno3 inet manual
iface eno4 inet manual
source /etc/network/interfaces.d/*I do not know what else I can/should explore. From the thread above, this issue should have been resolved in the latest kernel, which I'm already running.
Any help/pointers will be appreciated
Last edited: