Hi all,
Long time reader, first time poster here. I'm running a homelab node on a HP Prodesk 600 G6. I've plugged an Intel X540-AT2 dual 10G card to it to use pfSense and homelab firewall.
Port 0 is connected to my ISP router's 10G port (Zyxel AX7501-B0), the other port is connected to a switch with a TP Link 10G RJ45 SFP+ transceiver. Card is connected to the PCIe x16 slot since I don't have a graphics card
The situation I have is that only one of the ports would negotiate 10G, with 1 other running at 1G. If I test the ports individually, each of them can link at 10G but when both are plugged, typically port 0 will run at 1G and port 1 at 10G.
So far I have tried to update to the latest ixgbe driver and enable IOMMU groups to make sure they don't share one. I'm running out of ideas therefore recurring to the community for some hints on what to try next. Any help is much appreciated.
Best regards!
These are some outputs:
ethtool:
ethtool -i:
dmesg
modinfo latest driver:
IOMMU groups:
lspci:
Long time reader, first time poster here. I'm running a homelab node on a HP Prodesk 600 G6. I've plugged an Intel X540-AT2 dual 10G card to it to use pfSense and homelab firewall.
Port 0 is connected to my ISP router's 10G port (Zyxel AX7501-B0), the other port is connected to a switch with a TP Link 10G RJ45 SFP+ transceiver. Card is connected to the PCIe x16 slot since I don't have a graphics card
The situation I have is that only one of the ports would negotiate 10G, with 1 other running at 1G. If I test the ports individually, each of them can link at 10G but when both are plugged, typically port 0 will run at 1G and port 1 at 10G.
So far I have tried to update to the latest ixgbe driver and enable IOMMU groups to make sure they don't share one. I'm running out of ideas therefore recurring to the community for some hints on what to try next. Any help is much appreciated.
Best regards!
These are some outputs:
ethtool:
Code:
root@pve:~# ethtool enp1s0f0
Settings for enp1s0f0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: 1000Mb/s
Duplex: Full
Auto-negotiation: on
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
MDI-X: Unknown
Supports Wake-on: umbg
Wake-on: g
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
root@pve:~# ethtool enp1s0f1
Settings for enp1s0f1:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Auto-negotiation: on
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
MDI-X: Unknown
Supports Wake-on: umbg
Wake-on: g
Current message level: 0x00000007 (7)
drv probe link
Link detected: yesethtool -i:
Code:
root@pve:~# ethtool -i enp1s0f1
driver: ixgbe
version: 6.5.11-7-pve
firmware-version: 0x8000052f
expansion-rom-version:
bus-info: 0000:01:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
root@pve:~# ethtool -i enp1s0f0
driver: ixgbe
version: 6.5.11-7-pve
firmware-version: 0x8000052f
expansion-rom-version:
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yesdmesg
Code:
[ 1.293509] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver
[ 1.293510] ixgbe: Copyright (c) 1999-2016 Intel Corporation.
[ 1.293558] ixgbe 0000:01:00.0: enabling device (0000 -> 0002)
[ 1.595779] ixgbe 0000:01:00.0: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16 XDP Queue count = 0
[ 1.679543] ixgbe 0000:01:00.0: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link)
[ 1.704009] ixgbe 0000:01:00.0: MAC: 3, PHY: 0, PBA No: 000000-000
[ 1.704012] ixgbe 0000:01:00.0: 6c:92:bf:5e:c8:e3
[ 1.850976] ixgbe 0000:01:00.0: Intel(R) 10 Gigabit Network Connection
[ 1.851054] ixgbe 0000:01:00.1: enabling device (0000 -> 0002)
[ 2.127312] ixgbe 0000:01:00.1: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16 XDP Queue count = 0
[ 2.211424] ixgbe 0000:01:00.1: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link)
[ 2.235610] ixgbe 0000:01:00.1: MAC: 3, PHY: 0, PBA No: 000000-000
[ 2.235612] ixgbe 0000:01:00.1: 6c:92:bf:5e:c8:e4
[ 2.382946] ixgbe 0000:01:00.1: Intel(R) 10 Gigabit Network Connection
[ 2.412958] ixgbe 0000:01:00.0 enp1s0f0: renamed from eth0
[ 2.456883] ixgbe 0000:01:00.1 enp1s0f1: renamed from eth1
[ 8.868295] ixgbe 0000:01:00.0 enp1s0f0: entered allmulticast mode
[ 8.868315] ixgbe 0000:01:00.0 enp1s0f0: entered promiscuous mode
[ 9.129915] ixgbe 0000:01:00.0: registered PHC device on enp1s0f0
[ 9.285110] ixgbe 0000:01:00.1 enp1s0f1: entered allmulticast mode
[ 9.285160] ixgbe 0000:01:00.1 enp1s0f1: entered promiscuous mode
[ 9.360594] ixgbe 0000:01:00.1: registered PHC device on enp1s0f1
[ 14.203023] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[ 31.754225] ixgbe 0000:01:00.0 enp1s0f0: NIC Link is Up 1 Gbps, Flow Control: RX/TXmodinfo latest driver:
Code:
filename: /lib/modules/6.5.11-7-pve/updates/dkms/ixgbe.ko
version: 5.19.9
license: GPL
description: Intel(R) 10GbE PCI Express Linux Network Driver
author: Intel Corporation, <linux.nics@intel.com>
srcversion: 3AA6B39C75EB10BF0E7186C
alias: pci:v00008086d000015E5sv*sd*bc*sc*i*
alias: pci:v00008086d000015E4sv*sd*bc*sc*i*
alias: pci:v00008086d000015CEsv*sd*bc*sc*i*
alias: pci:v00008086d000015CCsv*sd*bc*sc*i*
alias: pci:v00008086d000015CAsv*sd*bc*sc*i*
alias: pci:v00008086d000015C8sv*sd*bc*sc*i*
alias: pci:v00008086d000015C7sv*sd*bc*sc*i*
alias: pci:v00008086d000015C6sv*sd*bc*sc*i*
alias: pci:v00008086d000015C4sv*sd*bc*sc*i*
alias: pci:v00008086d000015C3sv*sd*bc*sc*i*
alias: pci:v00008086d000015C2sv*sd*bc*sc*i*
alias: pci:v00008086d000015AEsv*sd*bc*sc*i*
alias: pci:v00008086d000015ADsv*sd*bc*sc*i*
alias: pci:v00008086d000015ACsv*sd*bc*sc*i*
alias: pci:v00008086d000015ABsv*sd*bc*sc*i*
alias: pci:v00008086d000015B0sv*sd*bc*sc*i*
alias: pci:v00008086d000015AAsv*sd*bc*sc*i*
alias: pci:v00008086d000015D1sv*sd*bc*sc*i*
alias: pci:v00008086d00001563sv*sd*bc*sc*i*
alias: pci:v00008086d00001560sv*sd*bc*sc*i*
alias: pci:v00008086d00001558sv*sd*bc*sc*i*
alias: pci:v00008086d0000154Asv*sd*bc*sc*i*
alias: pci:v00008086d00001557sv*sd*bc*sc*i*
alias: pci:v00008086d0000154Dsv*sd*bc*sc*i*
alias: pci:v00008086d00001528sv*sd*bc*sc*i*
alias: pci:v00008086d000010F8sv*sd*bc*sc*i*
alias: pci:v00008086d0000151Csv*sd*bc*sc*i*
alias: pci:v00008086d00001529sv*sd*bc*sc*i*
alias: pci:v00008086d0000152Asv*sd*bc*sc*i*
alias: pci:v00008086d000010F9sv*sd*bc*sc*i*
alias: pci:v00008086d00001514sv*sd*bc*sc*i*
alias: pci:v00008086d00001507sv*sd*bc*sc*i*
alias: pci:v00008086d000010FBsv*sd*bc*sc*i*
alias: pci:v00008086d00001517sv*sd*bc*sc*i*
alias: pci:v00008086d000010FCsv*sd*bc*sc*i*
alias: pci:v00008086d000010F7sv*sd*bc*sc*i*
alias: pci:v00008086d00001508sv*sd*bc*sc*i*
alias: pci:v00008086d000010DBsv*sd*bc*sc*i*
alias: pci:v00008086d000010F4sv*sd*bc*sc*i*
alias: pci:v00008086d000010E1sv*sd*bc*sc*i*
alias: pci:v00008086d000010F1sv*sd*bc*sc*i*
alias: pci:v00008086d000010ECsv*sd*bc*sc*i*
alias: pci:v00008086d000010DDsv*sd*bc*sc*i*
alias: pci:v00008086d0000150Bsv*sd*bc*sc*i*
alias: pci:v00008086d000010C8sv*sd*bc*sc*i*
alias: pci:v00008086d000010C7sv*sd*bc*sc*i*
alias: pci:v00008086d000010C6sv*sd*bc*sc*i*
alias: pci:v00008086d000010B6sv*sd*bc*sc*i*
depends: dca
retpoline: Y
name: ixgbe
vermagic: 6.5.11-7-pve SMP preempt mod_unload modversions
sig_id: PKCS#7
signer: DKMS module signing key
sig_key: 06:6B:99:06:96:23:9F:1F:B5:08:9E:38:21:4D:14:14:AA:3F:77:07
sig_hashalgo: sha512
signature: 13:2A:72:4D:BE:01:B8:8D:22:EF:40:82:A0:32:FA:65:A9:7F:BC:46:
ED:4D:16:87:B9:D4:79:1F:93:1A:7C:B1:D0:71:62:AF:62:44:21:EE:
AD:36:B8:A1:0E:FF:5D:11:98:1E:F4:12:36:68:87:59:93:4E:DD:B7:
36:C6:0D:94:1D:FF:B7:FE:F4:9E:88:85:90:83:70:77:F5:4A:36:53:
31:68:10:0C:A1:62:20:FC:D7:82:52:8E:58:A5:21:94:59:54:30:E7:
61:C8:A6:18:FA:95:0B:4C:3A:BC:35:4D:68:2A:A2:FB:F3:8A:E9:CB:
C1:59:28:58:DD:BA:B3:7C:0C:A4:18:50:D2:62:71:25:FE:78:26:60:
18:5F:2C:58:3B:E4:F6:56:BA:43:44:CD:4E:99:68:6D:02:DB:F2:4F:
FA:AF:77:C4:97:5C:0C:F8:AC:D0:51:CD:B0:AC:AE:1E:8A:93:0D:5D:
A1:CC:BD:25:77:07:EC:CB:C3:21:0B:56:19:52:7A:2A:14:C2:A9:A9:
D3:17:32:5F:64:99:45:67:35:D5:8B:3A:B5:3A:4C:F6:1A:69:AE:BE:
D6:EC:41:66:CA:C5:44:06:D9:BB:02:79:C8:86:E6:55:A7:5C:FD:9B:
0E:DA:25:3B:7B:56:CF:32:BA:2E:E9:4D:DB:08:8A:9E
parm: IntMode:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-X), default 2 (array of int)
parm: InterruptType:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-X), default IntMode (deprecated) (array of int)
parm: MQ:Disable or enable Multiple Queues, default 1 (array of int)
parm: DCA:Disable or enable Direct Cache Access, 0=disabled, 1=descriptor only, 2=descriptor and data (array of int)
parm: RSS:Number of Receive-Side Scaling Descriptor Queues, default 0=number of cpus (array of int)
parm: VMDQ:Number of Virtual Machine Device Queues: 0/1 = disable (1 queue) 2-16 enable (default=8) (array of int)
parm: max_vfs:Number of Virtual Functions: 0 = disable (default), 1-63 = enable this many VFs (array of int)
parm: VEPA:VEPA Bridge Mode: 0 = VEB (default), 1 = VEPA (array of int)
parm: InterruptThrottleRate:Maximum interrupts per second, per vector, (0,1,956-488281), default 1 (array of int)
parm: LLIPort:Low Latency Interrupt TCP Port (0-65535) (array of int)
parm: LLIPush:Low Latency Interrupt on TCP Push flag (0,1) (array of int)
parm: LLISize:Low Latency Interrupt on Packet Size (0-1500) (array of int)
parm: LLIEType:Low Latency Interrupt Ethernet Protocol Type (array of int)
parm: LLIVLANP:Low Latency Interrupt on VLAN priority threshold (array of int)
parm: FdirPballoc:Flow Director packet buffer allocation level:
1 = 8k hash filters or 2k perfect filters
2 = 16k hash filters or 4k perfect filters
3 = 32k hash filters or 8k perfect filters (array of int)
parm: AtrSampleRate:Software ATR Tx packet sample rate (array of int)
parm: FCoE:Disable or enable FCoE Offload, default 1 (array of int)
parm: MDD:Malicious Driver Detection: (0,1), default 1 = on (array of int)
parm: LRO:Large Receive Offload (0,1), default 0 = off (array of int)
parm: allow_unsupported_sfp:Allow unsupported and untested SFP+ modules on 82599 based adapters, default 0 = Disable (array of int)
parm: dmac_watchdog:DMA coalescing watchdog in microseconds (0,41-10000), default 0 = off (array of int)
parm: vxlan_rx:VXLAN receive checksum offload (0,1), default 1 = Enable (array of int)IOMMU groups:
Code:
IOMMU group 0 00:02.0 VGA compatible controller [0300]: Intel Corporation CometLake-S GT2 [UHD Graphics 630] [8086:9bc5] (rev 05)
IOMMU group 10 01:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 [8086:1528] (rev 01)
IOMMU group 11 01:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 [8086:1528] (rev 01)
IOMMU group 12 02:00.0 Non-Volatile memory controller [0108]: SK hynix Gold P31/PC711 NVMe Solid State Drive [1c5c:174a]
IOMMU group 1 00:00.0 Host bridge [0600]: Intel Corporation 10th Gen Core Processor Host Bridge/DRAM Registers [8086:9b43] (rev 05)
IOMMU group 2 00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 05)
IOMMU group 3 00:12.0 Signal processing controller [1180]: Intel Corporation Comet Lake PCH Thermal Controller [8086:06f9]
IOMMU group 4 00:14.0 USB controller [0c03]: Intel Corporation Comet Lake USB 3.1 xHCI Host Controller [8086:06ed]
IOMMU group 4 00:14.2 RAM memory [0500]: Intel Corporation Comet Lake PCH Shared SRAM [8086:06ef]
IOMMU group 5 00:14.3 Network controller [0280]: Intel Corporation Comet Lake PCH CNVi WiFi [8086:06f0]
IOMMU group 6 00:16.0 Communication controller [0780]: Intel Corporation Comet Lake HECI Controller [8086:06e0]
IOMMU group 6 00:16.3 Serial controller [0700]: Intel Corporation Comet Lake Keyboard and Text (KT) Redirection [8086:06e3]
IOMMU group 7 00:17.0 SATA controller [0106]: Intel Corporation Comet Lake SATA AHCI Controller [8086:06d2]
IOMMU group 8 00:1b.0 PCI bridge [0604]: Intel Corporation Comet Lake PCI Express Root Port #21 [8086:06ac] (rev f0)
IOMMU group 9 00:1f.0 ISA bridge [0601]: Intel Corporation Q470 Chipset LPC/eSPI Controller [8086:0687]
IOMMU group 9 00:1f.3 Audio device [0403]: Intel Corporation Comet Lake PCH cAVS [8086:06c8]
IOMMU group 9 00:1f.4 SMBus [0c05]: Intel Corporation Comet Lake PCH SMBus Controller [8086:06a3]
IOMMU group 9 00:1f.5 Serial bus controller [0c80]: Intel Corporation Comet Lake PCH SPI Controller [8086:06a4]
IOMMU group 9 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (11) I219-LM [8086:0d4c]lspci:
Code:
01:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
01:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)