Network setup on Dell R610 Quad Broadcom NetXtreme BCM5709 (rev20)

[bazarte]

I have three Dell R610 servers with Broadcom NetXreme BCM 5709 quad NICs. The installation goes perfectly, but I'm unable to get a network connectivity to or from Promox

I tried to reinstall Promox on two of the three servers. Unable to ping to or form the server.

Promox can see the NICs but it will not bring them up
"ip a" shows the following output
root@pve:~# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eno1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether d4:be:d9:af:35:cb brd ff:ff:ff:ff:ff:ff altname enp1s0f0 3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether d4:be:d9:af:35:cd brd ff:ff:ff:ff:ff:ff altname enp1s0f1 4: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether d4:be:d9:af:35:cf brd ff:ff:ff:ff:ff:ff altname enp2s0f0 5: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop master vmbr0 state DOWN group default qlen 1000 link/ether d4:be:d9:af:35:d1 brd ff:ff:ff:ff:ff:ff altname enp2s0f 1 6: vmbr0: <NO-CARRIER, BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000 link/ether d4:be:d9:af:35:d1 brd ff:ff:ff:ff:ff:ff inet 10.1.0.19/23 scope global vmbr0 valid_lft forever preferred_lft forever root@pve:"#

Has anyone had luck with this issue?

 

[Moayad]

Hi,
Please provide us with the network configuration as well?

 

[bazarte]

I'm not sure if this is what you are requesing:

root@pve:/etc/network# cat interfaces
auto lo
iface lo inet loopback
iface eno4 inet manual
auto vmbr0
    iface vmbr0 inet static
    address 10.1.0.19/23
    gateway 10.1.0.1
    bridge-ports eno4
    bridge-stp off
    bridge-fd 0
iface eno1 inet manual
iface eno2 inet manual
iface en03 inet manual
root@pve:/etc/network#

I also tried configuring other network ports on the Quad NIC to be the primary. No success
My other R610 is set to use "eno1". The same thing is happening

 

[Axl_fzetafs]

I'm not sure if this is what you are requesing:

root@pve:/etc/network# cat interfaces
auto lo
iface lo inet loopback
iface eno4 inet manual
auto vmbr0
    iface vmbr0 inet static
    address 10.1.0.19/23
    gateway 10.1.0.1
    bridge-ports eno4
    bridge-stp off
    bridge-fd 0
iface eno1 inet manual
iface eno2 inet manual
iface en03 inet manual
root@pve:/etc/network#

I also tried configuring other network ports on the Quad NIC to be the primary. No success
My other R610 is set to use "eno1". The same thing is happening

I would prepend "auto eno4" before "iface eno4 inet manual"

From "man interfaces": lines beginning with the word "auto" are used to identify the physical interfaces to be brought up when ifup is run with the -a option.

 

[bazarte]

I changed the interfaces file to the following:

auto lo
iface lo inet loopback

auto eno4
iface eno4 inet manual

auto vmbr0
    iface vmbr0 inet static
    address 10.1.0.19/23
    gateway 10.1.0.1
    bridge-ports eno4
    bridge-stp off
    bridge-fd 0

iface eno1 inet manual
iface eno2 inet manual
iface en03 inet manual

The interface is still not comming up after rebooting.

ip a still shows:
5: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop master vmbr0 state DOWN group default qlen 1000   
link/ether d4:be:d9:af:35:d1 brd ff:ff:ff:ff:ff:ff   
altname enp2s0f1
6: vmbr0: <NO-CARRIER, BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000 
 link/ether d4:be:d9:af:35:d1 brd ff:ff:ff:ff:ff:ff   
inet 10.1.0.19/23 scope global vmbr0     
 valid_lft forever preferred_lft forever
root@pve:"#

when i try to force the interface up with: "ifup eno4" I get:
bnx2: fw sync timeout, reset code = 5020004

 

[bazarte]

ifup eno1 and eno2 work...

 

[Moayad]

Hi,


You can also use ifreload -a command.

 

[Axl_fzetafs]

Does the reset code change every time? Sounds like a drive issue.

"ifreload -a" is a must of course