I've recently installed a TP-LINK TX401 NIC into my proxmox homeserver. While running a few things I've noticed that they keep getting repeatedly disconnected. After digging into the logs, it looks like the speed on the NIC keeps changing. I'm not sure where to start looking.
I tried to do some digging on my own, but can't seem to find the root cause of why it keeps disconnecting, or rather, switching between 0 and 10000 speed.
I'm not sure what to post for logs, so here are what I have looked at so far. Please let me know what else would be helpful and I will update the post and reply to your comment
This is the closest I could find to the issue, but it wasn't resolved. https://github.com/Aquantia/AQtion/issues/5
Setup
I have changed the port link speed to 10Gbps FDX. I have also tried disabling autoneg on the NIC.
ethtool
Proxmox Logs
lspci -vvvvv
I tried to do some digging on my own, but can't seem to find the root cause of why it keeps disconnecting, or rather, switching between 0 and 10000 speed.
I'm not sure what to post for logs, so here are what I have looked at so far. Please let me know what else would be helpful and I will update the post and reply to your comment
This is the closest I could find to the issue, but it wasn't resolved. https://github.com/Aquantia/AQtion/issues/5
Setup
Code:
UDM Pro -> CM-RJ45-10G Adapter (Port 10) -> CAT6A -> RX401I have changed the port link speed to 10Gbps FDX. I have also tried disabling autoneg on the NIC.
ethtool
Code:
Settings for enp1s0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
2500baseT/Full
5000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Auto-negotiation: off
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
MDI-X: Unknown
Supports Wake-on: pg
Wake-on: g
Current message level: 0x00000005 (5)
drv link
Link detected: yes
driver: atlantic
version: 5.15.60-1-pve
firmware-version: 3.1.100
expansion-rom-version:
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: yes
ip address: I've tried the default of 1500 mtu and still had the same result
4: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 16000 qdisc mq master vmbr1 state UP group default qlen 1000
link/ether 1c:61:b4:20:8f:4c brd ff:ff:ff:ff:ff:ff
7: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 16000 qdisc noqueue state UP group default qlen 1000
link/ether 1c:61:b4:20:8f:4c brd ff:ff:ff:ff:ff:ff
inet6 fe80::1e61:b4ff:fe20:8f4c/64 scope link
valid_lft forever preferred_lft foreverProxmox Logs
Code:
Sep 29 23:07:11 guai-node kernel: atlantic 0000:01:00.0 enp1s0: atlantic: link change old 10000 new 0
Sep 29 23:07:11 guai-node kernel: vmbr1: port 1(enp1s0) entered disabled state
Sep 29 23:07:16 guai-node kernel: atlantic 0000:01:00.0 enp1s0: atlantic: link change old 0 new 10000
Sep 29 23:07:21 guai-node kernel: vmbr1: port 1(enp1s0) entered blocking state
Sep 29 23:07:21 guai-node kernel: vmbr1: port 1(enp1s0) entered forwarding state
Sep 29 23:07:26 guai-node kernel: atlantic 0000:01:00.0 enp1s0: atlantic: link change old 10000 new 0
Sep 29 23:07:26 guai-node kernel: vmbr1: port 1(enp1s0) entered disabled state
Sep 29 23:07:31 guai-node kernel: atlantic 0000:01:00.0 enp1s0: atlantic: link change old 0 new 10000
Sep 29 23:07:31 guai-node kernel: vmbr1: port 1(enp1s0) entered blocking state
Sep 29 23:07:31 guai-node kernel: vmbr1: port 1(enp1s0) entered forwarding state
Sep 29 23:07:57 guai-node kernel: atlantic 0000:01:00.0 enp1s0: atlantic: link change old 10000 new 0
Sep 29 23:07:57 guai-node kernel: vmbr1: port 1(enp1s0) entered disabled state
Sep 29 23:08:02 guai-node kernel: atlantic 0000:01:00.0 enp1s0: atlantic: link change old 0 new 10000
Sep 29 23:08:02 guai-node kernel: vmbr1: port 1(enp1s0) entered blocking state
Sep 29 23:08:02 guai-node kernel: vmbr1: port 1(enp1s0) entered forwarding state
Sep 29 23:09:25 guai-node kernel: atlantic 0000:01:00.0 enp1s0: atlantic: link change old 10000 new 0
Sep 29 23:09:25 guai-node kernel: vmbr1: port 1(enp1s0) entered disabled state
Sep 29 23:09:30 guai-node kernel: atlantic 0000:01:00.0 enp1s0: atlantic: link change old 0 new 10000
Sep 29 23:09:30 guai-node kernel: vmbr1: port 1(enp1s0) entered blocking state
Sep 29 23:09:30 guai-node kernel: vmbr1: port 1(enp1s0) entered forwarding statelspci -vvvvv
Code:
01:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02)
Subsystem: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 42
IOMMU group: 15
Region 0: Memory at fc800000 (64-bit, non-prefetchable) [size=64K]
Region 2: Memory at fc810000 (64-bit, non-prefetchable) [size=4K]
Region 4: Memory at fc400000 (64-bit, non-prefetchable) [size=4M]
Capabilities: [40] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 75.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
MaxPayload 512 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L0s L1, Exit Latency L0s unlimited, L1 unlimited
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 8GT/s (ok), Width x4 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink+ Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+ EqualizationPhase1+
EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [80] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [90] MSI-X: Enable+ Count=32 Masked-
Vector table: BAR=2 offset=00000000
PBA: BAR=2 offset=00000200
Capabilities: [a0] MSI: Enable- Count=1/32 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [c0] Vital Product Data
Product Name: Atlantic
Read-only fields:
[PN] Part number: 3290495095
[EC] Engineering changes: 0
[FG] Unknown: 61 62 63
[LC] Unknown: 64 65 66
[MN] Manufacture ID: AFDSWEWEBSFD
[PG] Unknown: 49 49 49
[SN] Serial number: CPL5938TLKMY
[V0] Vendor specific: wfewfe
[V1] Vendor specific: fwewfe
[V2] Vendor specific: SDFWI
[RV] Reserved: checksum good, 0 byte(s) reserved
Read/write fields:
[YA] Asset tag: 9495829
[V0] Vendor specific: f34ge4rsg
[V1] Vendor specific: ger35g5rthghgsa3
[Y0] System specific: bsdfvbxcz
[Y1] System specific: fwefewwfe
[RW] Read-write area: 11 byte(s) free
End
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [150 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
Capabilities: [180 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Kernel driver in use: atlantic
Kernel modules: atlantic