No network connectivity in fresh Proxmox VE 7.3-3 install

[morik_proxmox]

Issue: vmbr0 along with all interfaces show as DOWN. Only console login feasible

HW: Dell XR12 w/ UEFI BIOS
NIC1: PCIe Broadcom NetExtreme-E 57504 quad-port 10/25GbE
NIC2: PCIe Mellanox ConnectX-6 dual-port 100GbE

Install steps:

• 2 ports on NIC1 and 1 port on NIC2 are connected via supported DACs to a switch
• The connected interfaces have layer-2 connectivity switch-side (show as UP)
• Virtualization + SR-IOV enabled @ global level in BIOS
• NIC1: NPart (Partitions) + SR-IOV enabled for BRCM 57505. 4 Partitions per port
• NIC2: SR-IOV enabled. 2 VFS
• Installed Proxmox VE
• reboot
• Noticed that default install didn't virtualized CPU or any other hardware components (No IOMMU_GROUPS detected). Any else have this issue?
• Followed steps here to enable

  vfio

• reboot
• CPU shows as virtualized; 16 (4*4) instances of NIC1 show up but all DOWN, Only 2 (instead of 4) instances of NIC2 show and are down, vmbr0 is down

Since only console access is available, copy/paste of content as code wasn't feasible. Attached are some screenshots instead (hope that is okay)?

eno8304np0: hwrm req_type 0x90 seq id 0x63 error 0x4
eno8304np0: hwrm vnic filter failure rc: ffffffe4
eno8304np0: bnxt_init_nic err: ffffffe4

Any recommendations on how to go about troubleshooting next?

 

[bbgeek17]

Any recommendations on how to go about troubleshooting next?

start with the basics, disable SRIOV, try bare ports. There is a clear failure message from sriov subsystem in your boot. Its not practical to troubleshoot virtualized hardware, which often has bugs/compatibility issues, until you established that bare metal works.

P.S. there is always a way to get a nice text output - plug and mount a usb stick, dump command output to a file and move to another PC.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[morik_proxmox]

Thank you for the prompt reply! I forgot to mention that i tried both 'factory default' configuration (no SRIOV, no virtualization), with virtualization but no SR-IOV, and then with both enabled. Same end result. I will take your advice and go back to the basic. Reset the server to factory defaults and see what comes of it.

Thank you for the tip on output dump! In running circles around the mind-blowingly complex linux internals i forgot that re-directing output is an easy thing in linux