Mellanox ConnectX-5 working on one machine but not another

j4ys0n

Member
Jan 15, 2020
38
7
13
After some initial switch issues, I was able to get one NIC working, I then installed the other NIC in a separate machine and followed the same steps. Only the first NIC is working correctly. Details below. Anyone know what the issue could be?


Proxmox 7.4 on both machines
I'm able to assign an IP address on both machines (and I can see both in my networking interface, with the statically assigned IPs)
Only one machine responds to a ping.

Machine 1 (working)
Code:
auto vmbr2
iface vmbr2 inet static
        address 10.3.0.12/24
        gateway 10.3.0.1
        bridge-ports enp71s0f1np1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

Machine 2 (not working)
Code:
auto vmbr1
iface vmbr1 inet static
        address 10.3.0.8/24
        gateway 10.3.0.1
        bridge-ports ens19f1np1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

Ethtool output (identical for both):

Code:
Settings for ens19f1np1:
    Supported ports: [ FIBRE ]
    Supported link modes:   1000baseKX/Full
                            10000baseKR/Full
                            25000baseCR/Full
                            25000baseKR/Full
                            25000baseSR/Full
    Supported pause frame use: Symmetric
    Supports auto-negotiation: Yes
    Supported FEC modes: None     RS     BASER
    Advertised link modes:  1000baseKX/Full
                            10000baseKR/Full
                            25000baseCR/Full
                            25000baseKR/Full
                            25000baseSR/Full
    Advertised pause frame use: Symmetric
    Advertised auto-negotiation: Yes
    Advertised FEC modes: None     RS     BASER
    Link partner advertised link modes:  Not reported
    Link partner advertised pause frame use: No
    Link partner advertised auto-negotiation: Yes
    Link partner advertised FEC modes: Not reported
    Speed: 25000Mb/s
    Duplex: Full
    Auto-negotiation: on
    Port: FIBRE
    PHYAD: 0
    Transceiver: internal
    Supports Wake-on: d
    Wake-on: d
        Current message level: 0x00000004 (4)
                               link
    Link detected: yes
 
How does the output of the following commands look like on the host where the NIC is not working?

INI:
ip a

journalctl -b -u networking
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!