[SOLVED] RTL 8125B not tx/rx/broadcasting on Proxmox VE 8.0

[PSteinberg]

UPDATE: Layer 8 problem - i managed to extract the old card from the microserver and to put the same card in again . rtl8169 was the onboard device not the 2.5 Gbit device. After seeing this FAIL i plugged the right one in, and everything is working as supposed. So no 8125 driver problems but a big admin problem


Issue: r8169 for Realtek 8125B not Broadcasting - driver loaded but no communication
tested: direct link, switch, ethtool, VM PCIE pass through to ubuntu machine (nic works instandly)

so card is good, pci express slot is good, means the host driver again is not working properlly or something in between.

PS: manual driver installation failed because of non signed module

I have no clue anymore what to try - this card is running in the same micro server when the nic is pass through to a Ubuntu VM (23.04)
Same card is working on another Ubuntu 23.04 machine out of the box

root@pve:~# ip a

10: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr4 state UP group default qlen 1000
    link/ether 8c:ec:4b:c1:8a:4a brd ff:ff:ff:ff:ff:ff
12: vmbr4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 8c:ec:4b:c1:8a:4a brd ff:ff:ff:ff:ff:ff
    inet6 fe80::8eec:4bff:fec1:8a4a/64 scope link
       valid_lft forever preferred_lft forever

root@pve:~# modinfo r8169
filename:       /lib/modules/6.2.16-12-pve/kernel/drivers/net/ethernet/realtek/r8169.ko
firmware:       rtl_nic/rtl8125b-2.fw
firmware:       rtl_nic/rtl8125a-3.fw
firmware:       rtl_nic/rtl8107e-2.fw
firmware:       rtl_nic/rtl8168fp-3.fw
firmware:       rtl_nic/rtl8168h-2.fw
firmware:       rtl_nic/rtl8168g-3.fw
firmware:       rtl_nic/rtl8168g-2.fw
firmware:       rtl_nic/rtl8106e-2.fw
firmware:       rtl_nic/rtl8106e-1.fw
firmware:       rtl_nic/rtl8411-2.fw
firmware:       rtl_nic/rtl8411-1.fw
firmware:       rtl_nic/rtl8402-1.fw
firmware:       rtl_nic/rtl8168f-2.fw
firmware:       rtl_nic/rtl8168f-1.fw
firmware:       rtl_nic/rtl8105e-1.fw
firmware:       rtl_nic/rtl8168e-3.fw
firmware:       rtl_nic/rtl8168e-2.fw
firmware:       rtl_nic/rtl8168e-1.fw
firmware:       rtl_nic/rtl8168d-2.fw
firmware:       rtl_nic/rtl8168d-1.fw
license:        GPL
softdep:        pre: realtek
description:    RealTek RTL-8169 Gigabit Ethernet driver
author:         Realtek and the Linux r8169 crew <netdev@vger.kernel.org>
srcversion:     B1E506184A8B0F572D52508
alias:          pci:v000010ECd00003000sv*sd*bc*sc*i*
alias:          pci:v000010ECd00008125sv*sd*bc*sc*i*
alias:          pci:v00000001d00008168sv*sd00002410bc*sc*i*
alias:          pci:v00001737d00001032sv*sd00000024bc*sc*i*
alias:          pci:v000016ECd00000116sv*sd*bc*sc*i*
alias:          pci:v00001259d0000C107sv*sd*bc*sc*i*
alias:          pci:v00001186d00004302sv*sd*bc*sc*i*
alias:          pci:v00001186d00004300sv*sd*bc*sc*i*
alias:          pci:v00001186d00004300sv00001186sd00004B10bc*sc*i*
alias:          pci:v000010ECd00008169sv*sd*bc*sc*i*
alias:          pci:v000010FFd00008168sv*sd*bc*sc*i*
alias:          pci:v000010ECd00008168sv*sd*bc*sc*i*
alias:          pci:v000010ECd00008167sv*sd*bc*sc*i*
alias:          pci:v000010ECd00008162sv*sd*bc*sc*i*
alias:          pci:v000010ECd00008161sv*sd*bc*sc*i*
alias:          pci:v000010ECd00008136sv*sd*bc*sc*i*
alias:          pci:v000010ECd00008129sv*sd*bc*sc*i*
alias:          pci:v000010ECd00002600sv*sd*bc*sc*i*
alias:          pci:v000010ECd00002502sv*sd*bc*sc*i*
depends:     
retpoline:      Y
intree:         Y
name:           r8169
vermagic:       6.2.16-12-pve SMP preempt mod_unload modversions
sig_id:         PKCS#7
signer:         Build time autogenerated kernel key
sig_key:        3C:77:A0:CB:73:A7:08:28:ED:35:3F:65:C9:6B:95:4A:A6:7F:F4:DC
sig_hashalgo:   sha512
signature:      76:C7:B3:E8:F7:9E:6E:E6:AB:8D:CF:7F:7B:7F:68:53:7F:6D:AC:66:
                8A:F0:64:2A:FC:E6:1B:D2:68:0F:12:8A:F8:E0:39:5B:76:1C:1B:86:
                69:E5:16:83:44:5C:67:F3:8F:60:73:11:94:05:D1:68:78:FC:74:0C:
                4D:07:74:DC:FC:74:2C:1E:EC:62:68:0E:81:87:B0:33:AB:EC:F3:6A:
                C0:B6:F2:58:37:0D:D4:78:E5:52:89:18:08:8D:02:2A:0A:90:FB:A4:
                EC:E1:2F:7C:93:5C:20:C2:03:58:77:3B:9A:EC:FC:1B:88:3F:DC:2F:
                86:01:E9:01:63:AA:50:F3:52:5B:7E:53:4C:92:D6:83:D0:3A:7D:4F:
                DB:35:68:BD:B8:5F:5F:7A:1C:B8:8C:0B:3A:E5:53:F0:4C:6C:57:51:
                EC:71:2B:38:28:B8:36:4E:D1:89:CF:B0:E8:9A:77:33:FF:F8:D6:3B:
                CA:A0:AB:9F:EC:92:3E:67:B8:C9:4B:03:11:F3:0B:B1:5B:0A:F4:E7:
                8D:F3:6A:5C:95:0E:00:F8:BC:5D:5D:74:02:62:CD:E0:75:4F:59:E5:
                31:51:6B:74:2C:99:34:1D:15:86:06:BE:44:2B:B3:E6:E0:CC:69:FE:
                87:74:13:49:8F:CF:C1:F5:B9:B7:A6:D1:4C:A8:6F:07:13:A8:29:BE:
                9D:FB:36:35:87:A6:7E:EE:2B:38:77:51:34:97:E7:1D:A8:C3:6F:05:
                96:4C:9F:42:3D:14:EE:8F:8F:EE:00:3F:61:36:AA:7D:BC:1E:7A:CD:
                28:F5:7F:04:99:3B:7A:0B:8F:24:1E:A4:58:5C:9E:50:D4:03:B8:B4:
                E3:CC:22:02:80:CB:C6:63:DB:DF:74:E5:D3:80:DF:D0:15:64:23:43:
                75:D1:DE:D2:66:F0:2C:D3:DC:7B:BE:9A:9B:8E:26:ED:4E:8F:1F:7E:
                9C:ED:36:E7:5E:A0:4A:D7:28:7B:99:6E:1C:10:0D:76:25:3B:44:CF:
                0C:7A:F6:7F:A8:41:2F:50:AD:0D:E5:62:81:91:7D:AD:E2:51:95:17:
                1F:6D:D2:FB:DD:44:47:6D:A5:0C:A7:CB:8D:E6:81:A6:50:4E:C9:25:
                AD:CA:E4:4E:91:35:0E:1B:85:BB:B1:7F:3B:77:93:3F:CC:B2:09:8B:
                C0:8A:BA:30:8D:23:13:8A:AD:79:FB:41:DF:9C:82:5E:FE:A4:54:65:
                99:D9:A3:3B:3A:68:30:DC:07:02:84:41:3D:DD:9E:6C:52:33:D9:02:
                D0:44:F4:CB:5C:02:82:86:D1:60:A4:12:38:7A:82:D9:90:3B:8C:93:
                21:8D:1C:07:31:4D:D4:DE:4E:7C:D7:1F

root@pve:~# lscpi -v

04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
        Subsystem: Dell RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
        Flags: bus master, fast devsel, latency 0, IRQ 16, IOMMU group 14
        I/O ports at 3000 [size=256]
        Memory at d1104000 (64-bit, non-prefetchable) [size=4K]
        Memory at d1100000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Endpoint, MSI 01
        Capabilities: [b0] MSI-X: Enable+ Count=4 Masked-
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Virtual Channel
        Capabilities: [160] Device Serial Number 01-00-00-00-68-4c-e0-00
        Capabilities: [170] Latency Tolerance Reporting
        Capabilities: [178] L1 PM Substates
        Kernel driver in use: r8169
        Kernel modules: r8169

root@pve:~# dmesg -e | grep eth


[Sep10 09:20] r8169 0000:04:00.0 eth0: RTL8168h/8111h, 8c:ec:4b:c1:8a:4a, XID 541, IRQ 133
[  +0.000005] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[  +0.006180] r8169 0000:04:00.0 enp4s0: renamed from eth0

root@pve:~# cat /sys/bus/pci/devices/0000\:04\:00.0/modalias
pci:v000010ECd00008168sv00001028sd0000085Cbc02sc00i00

root@pve:~# ethtool enp4s0
Settings for enp4s0:
        Supported ports: [ TP    MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Auto-negotiation: on
        master-slave cfg: preferred slave
        master-slave status: unknown
        Port: Twisted Pair
        PHYAD: 0
        Transceiver: external
        MDI-X: Unknown
        Supports Wake-on: pumbg
        Wake-on: d
        Link detected: no

PVE 8.0.4

root@pve:~# uname -a
Linux pve 6.2.16-12-pve #1 SMP PREEMPT_DYNAMIC PMX 6.2.16-12 (2023-09-04T13:21Z) x86_64 GNU/Linux

This is how it should look like ​

root@pve:~# ethtool -i enp3s0  
driver: r8169
version: 6.2.16-12-pve
firmware-version: rtl8125b-2_0.0.2 07/13/20
expansion-rom-version:  
bus-info: 0000:03:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no

That is how it looks on Ubunut 23.04 as comparison​

ethtool -i enp3s0
driver: r8169
version: 6.2.0-32-generic
firmware-version: rtl8125b-2_0.0.2 07/13/20

Any help appriciated...