Missing NIC

D

dpearceFL

Member
Jun 1, 2020
75
3
13
64
I have three Cisco C220 M4 servers all loaded with Proxmox 8.1.4 and configured into a cluster. The cluster is healthy.

Up until now I have been using one 1 GBe/sec NIC per node. Now I want to utilize another NIC (I have eight others to choose from). With the server at the POST screen, I can get link lights on more than 6 of them including the NIC I am trying to use. I have been all through the BIOS and see no way to enable/disable the NICs.

Once I let the server boot, only the original NIC works. The NIC that was getting a link light on POST, is not longer getting a link. The new NIC does not show in Proxmox. Hopefully some of the following information will lead to a solution.

Code: 
# dmesg | grep igb
[    3.742546] igb: Intel(R) Gigabit Ethernet Network Driver
[    3.742549] igb: Copyright (c) 2007-2014 Intel Corporation.
[    3.947650] igb 0000:01:00.0: added PHC on eth0
[    3.947665] igb 0000:01:00.0: Intel(R) Gigabit Ethernet Network Connection
[    3.947667] igb 0000:01:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 38:0e:4d:0b:91:a0
[    3.947742] igb 0000:01:00.0: eth0: PBA No: FFFFFF-0FF
[    3.947743] igb 0000:01:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
[    4.056526] igb 0000:01:00.1: added PHC on eth1
[    4.056540] igb 0000:01:00.1: Intel(R) Gigabit Ethernet Network Connection
[    4.056542] igb 0000:01:00.1: eth1: (PCIe:5.0Gb/s:Width x4) 38:0e:4d:0b:91:a1
[    4.056617] igb 0000:01:00.1: eth1: PBA No: FFFFFF-0FF
[    4.056619] igb 0000:01:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
[    4.111613] igb 0000:81:00.0: added PHC on eth2
[    4.111626] igb 0000:81:00.0: Intel(R) Gigabit Ethernet Network Connection
[    4.111628] igb 0000:81:00.0: eth2: (PCIe:5.0Gb/s:Width x4) b4:96:91:0a:32:44
[    4.111913] igb 0000:81:00.0: eth2: PBA No: G62139-000
[    4.111915] igb 0000:81:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
[    4.165639] igb 0000:81:00.1: added PHC on eth3
[    4.165651] igb 0000:81:00.1: Intel(R) Gigabit Ethernet Network Connection
[    4.165652] igb 0000:81:00.1: eth3: (PCIe:5.0Gb/s:Width x4) b4:96:91:0a:32:45
[    4.165937] igb 0000:81:00.1: eth3: PBA No: G62139-000
[    4.165939] igb 0000:81:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
[    4.220955] igb 0000:81:00.2: added PHC on eth4
[    4.220967] igb 0000:81:00.2: Intel(R) Gigabit Ethernet Network Connection
[    4.220969] igb 0000:81:00.2: eth4: (PCIe:5.0Gb/s:Width x4) b4:96:91:0a:32:46
[    4.221254] igb 0000:81:00.2: eth4: PBA No: G62139-000
[    4.221255] igb 0000:81:00.2: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
[    4.277650] igb 0000:81:00.3: added PHC on eth5
[    4.277663] igb 0000:81:00.3: Intel(R) Gigabit Ethernet Network Connection
[    4.277664] igb 0000:81:00.3: eth5: (PCIe:5.0Gb/s:Width x4) b4:96:91:0a:32:47
[    4.277950] igb 0000:81:00.3: eth5: PBA No: G62139-000
[    4.277951] igb 0000:81:00.3: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
[    4.327006] igb 0000:01:00.1 enp1s0f1: renamed from eth1
[    4.378958] igb 0000:81:00.0 enp129s0f0: renamed from eth2
[    4.514941] igb 0000:81:00.3 enp129s0f3: renamed from eth5
[    4.539094] igb 0000:01:00.0 enp1s0f0: renamed from eth0
[    4.582972] igb 0000:81:00.1 enp129s0f1: renamed from eth3
[    4.607254] igb 0000:81:00.2 enp129s0f2: renamed from eth4
[    8.787926] igb 0000:01:00.0 enp1s0f0: entered allmulticast mode
[    8.787977] igb 0000:01:00.0 enp1s0f0: entered promiscuous mode
[   12.211201] igb 0000:01:00.0 enp1s0f0: igb: enp1s0f0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX

Code: 
# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp9s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:a2:ee:f0:58:03 brd ff:ff:ff:ff:ff:ff
3: enp10s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:a2:ee:f0:58:04 brd ff:ff:ff:ff:ff:ff
4: enp19s0: <BROADCAST,MULTICAST> mtu 9000 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 38:0e:4d:16:d2:92 brd ff:ff:ff:ff:ff:ff
5: enp20s0: <BROADCAST,MULTICAST> mtu 9000 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 38:0e:4d:16:d2:93 brd ff:ff:ff:ff:ff:ff
6: enp1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 38:0e:4d:0b:91:a0 brd ff:ff:ff:ff:ff:ff
7: enp1s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 38:0e:4d:0b:91:a1 brd ff:ff:ff:ff:ff:ff
8: enp129s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether b4:96:91:0a:32:44 brd ff:ff:ff:ff:ff:ff
9: enp129s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether b4:96:91:0a:32:45 brd ff:ff:ff:ff:ff:ff
10: enp129s0f2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether b4:96:91:0a:32:46 brd ff:ff:ff:ff:ff:ff
11: enp129s0f3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether b4:96:91:0a:32:47 brd ff:ff:ff:ff:ff:ff
12: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 38:0e:4d:0b:91:a0 brd ff:ff:ff:ff:ff:ff
13: vmbr5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether a6:dc:10:7a:4f:76 brd ff:ff:ff:ff:ff:ff
14: vmbr4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 5a:6b:da:de:3a:c0 brd ff:ff:ff:ff:ff:ff
15: vxlan_vmlan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master vmlan state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 52:cf:7f:74:28:64 brd ff:ff:ff:ff:ff:ff
16: vmlan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 52:cf:7f:74:28:64 brd ff:ff:ff:ff:ff:ff
17: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr100i0 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 76:65:e8:99:b1:55 brd ff:ff:ff:ff:ff:ff
18: fwbr100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ed:c1:c9:63:16 brd ff:ff:ff:ff:ff:ff
19: fwpr100p0@fwln100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 3a:45:13:e7:a9:80 brd ff:ff:ff:ff:ff:ff
20: fwln100i0@fwpr100p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP mode DEFAULT group default qlen 1000
    link/ether aa:ed:c1:c9:63:16 brd ff:ff:ff:ff:ff:ff

Code: 
# lspci | grep -i "ethernet"
01:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
09:00.0 Ethernet controller: Cisco Systems Inc VIC Ethernet NIC (rev a2)
0a:00.0 Ethernet controller: Cisco Systems Inc VIC Ethernet NIC (rev a2)
13:00.0 Ethernet controller: Cisco Systems Inc VIC Ethernet NIC (rev a2)
14:00.0 Ethernet controller: Cisco Systems Inc VIC Ethernet NIC (rev a2)
81:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
81:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
81:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
81:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

Thanks to this community of givers. :D
 
Code: 
# cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface enp1s0f0 inet manual

iface enp129s0f2 inet manual

iface enp129s0f3 inet manual

iface enp9s0 inet manual

auto enp10s0
iface enp10s0 inet manual

auto enp19s0
iface enp19s0 inet manual

auto enp20s0
iface enp20s0 inet manual

iface enp1s0f1 inet manual

iface enp129s0f0 inet manual

iface enp129s0f1 inet manual

auto vmbr0
iface vmbr0 inet static
        address 172.20.21.238/24
        gateway 172.20.21.1
        bridge-ports enp1s0f0
        bridge-stp off
        bridge-fd 0

auto vmbr5
iface vmbr5 inet static
        address 10.1.1.0/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0
#Management

auto vmbr4
iface vmbr4 inet static
        address 10.1.2.0/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0
#Communications

source /etc/network/interfaces.d/*

Code: 
# ip route
default via 172.20.21.1 dev vmbr0 proto kernel onlink
10.1.1.0/24 dev vmbr5 proto kernel scope link src 10.1.1.0
10.1.2.0/24 dev vmbr4 proto kernel scope link src 10.1.2.0
172.20.21.0/24 dev vmbr0 proto kernel scope link src 172.20.21.238
 
Last edited:
dpearceFL said:
auto vmbr5 iface vmbr5 inet static address 10.1.1.0/24 bridge-ports none bridge-stp off bridge-fd 0 #Management
Click to expand...

dpearceFL said:
auto vmbr4 iface vmbr4 inet static address 10.1.2.0/24 bridge-ports none bridge-stp off bridge-fd 0 #Communications
Click to expand...
These two are missing bridge ports. Check vmbr0 for an example.

Can you provide more information please on what exactly "missing from Proxmox" means? The NICs seem to be present in "ip a" output.


Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
"Missing" was perhaps a bad choice of words. "ip a" always showed the NICs as "down". Then I found I needed to flag the new NIC as "autostart". At this the link lights light up. Now "ip a" shows them as "up".

I then created the bridge and it all works now.

Thanks for helping me!
 
  • Like
Reactions: Kingneutron
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back