PVE disabling one (of two) NIC port as it boots up

[Oblique]

I am having a problem I have spent about 6 hours on, research etc. and I am hopeful someone can help me out here.

So as the title notes, when I boot up the PVE server, it goes through all of the motions until it would be time to sign into a console session (password prompt...) and then it drops one of the two Ethernet ports available on the NIC.

So all that reading paid off some. I now know a temporary solution to the problem. After boot up, if I issue ifup eno3 the port and its lights are restored. - The GUI concures with a "Yes" in the active box. Sadly this does not persist through a reboot. So I need a way to do the equivalent of what ifup eno3 does.

I'll post some logs and such here as one may need them to find a permanent solution.

My cat /etc/network output

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno4 inet manual

iface eno3 inet manual

iface eno1 inet manual

iface eno2 inet manual

iface enp3s0f0 inet manual

iface enp3s0f1 inet manual

iface enp131s0f0 inet manual

iface enp131s0f1 inet manual

auto vmbr0
iface vmbr0 inet static
    address 10.188.20.4/24
    gateway 10.188.20.10
    bridge-ports eno4
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

Output of ip a

root@carbon5:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 24:6e:96:55:d1:c4 brd ff:ff:ff:ff:ff:ff
    altname enp6s0f0
    inet6 2002:a29b:b59a:0:266e:96ff:fe55:d1c4/64 scope global dynamic mngtmpaddr
       valid_lft 86391sec preferred_lft 291sec
    inet6 fe80::266e:96ff:fe55:d1c4/64 scope link
       valid_lft forever preferred_lft forever
3: eno4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
    link/ether 24:6e:96:55:d1:c5 brd ff:ff:ff:ff:ff:ff
    altname enp6s0f1
4: eno1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 24:6e:96:55:d1:c0 brd ff:ff:ff:ff:ff:ff
    altname enp1s0f0
5: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 24:6e:96:55:d1:c2 brd ff:ff:ff:ff:ff:ff
    altname enp1s0f1
6: enp3s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a0:36:9f:de:79:b4 brd ff:ff:ff:ff:ff:ff
7: enp3s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a0:36:9f:de:79:b6 brd ff:ff:ff:ff:ff:ff
8: enp131s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a0:36:9f:df:ca:c0 brd ff:ff:ff:ff:ff:ff
9: enp131s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a0:36:9f:df:ca:c2 brd ff:ff:ff:ff:ff:ff
10: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 24:6e:96:55:d1:c5 brd ff:ff:ff:ff:ff:ff
    inet 10.188.20.4/24 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::266e:96ff:fe55:d1c5/64 scope link
       valid_lft forever preferred_lft forever
11: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr100i0 state UNKNOWN group default qlen 1000
    link/ether ee:07:e3:6b:c7:a4 brd ff:ff:ff:ff:ff:ff
12: fwbr100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 56:52:68:0e:b1:e1 brd ff:ff:ff:ff:ff:ff
13: fwpr100p0@fwln100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether e6:fb:bc:f6:f6:8b brd ff:ff:ff:ff:ff:ff
14: fwln100i0@fwpr100p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP group default qlen 1000
    link/ether 4a:68:fa:8b:d8:23 brd ff:ff:ff:ff:ff:ff
15: veth101i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr101i0 state UP group default qlen 1000
    link/ether fe:8e:ac:20:1a:cc brd ff:ff:ff:ff:ff:ff link-netnsid 0
16: fwbr101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 5e:3a:c0:9b:47:e0 brd ff:ff:ff:ff:ff:ff
17: fwpr101p0@fwln101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether 72:e8:9f:a0:80:99 brd ff:ff:ff:ff:ff:ff
18: fwln101i0@fwpr101p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr101i0 state UP group default qlen 1000
    link/ether 4a:b7:c5:32:83:7b brd ff:ff:ff:ff:ff:ff
19: tap102i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr102i0 state UNKNOWN group default qlen 1000
    link/ether 82:13:89:ad:1e:f7 brd ff:ff:ff:ff:ff:ff
20: fwbr102i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 8a:e7:8a:f2:fc:0d brd ff:ff:ff:ff:ff:ff
21: fwpr102p0@fwln102i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether ba:d4:5b:b0:42:99 brd ff:ff:ff:ff:ff:ff
22: fwln102i0@fwpr102p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr102i0 state UP group default qlen 1000
    link/ether 3a:75:a6:62:14:c0 brd ff:ff:ff:ff:ff:ff
23: veth103i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether fe:96:e9:4d:73:36 brd ff:ff:ff:ff:ff:ff link-netnsid 1
24: veth104i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr104i0 state UP group default qlen 1000
    link/ether fe:f6:a3:5d:3b:d9 brd ff:ff:ff:ff:ff:ff link-netnsid 2
25: fwbr104i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 8a:79:0b:66:d2:d4 brd ff:ff:ff:ff:ff:ff
26: fwpr104p0@fwln104i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether a2:5c:4c:54:8f:1d brd ff:ff:ff:ff:ff:ff
27: fwln104i0@fwpr104p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr104i0 state UP group default qlen 1000
    link/ether 92:38:07:ce:c4:be brd ff:ff:ff:ff:ff:ff

Output of lspci | grep 'Ethernet'

root@carbon5:~#  lspci | grep 'Ethernet'
01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
03:00.0 Ethernet controller: Intel Corporation Ethernet 10G 2P X520 Adapter (rev 01)
03:00.1 Ethernet controller: Intel Corporation Ethernet 10G 2P X520 Adapter (rev 01)
06:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
06:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
83:00.0 Ethernet controller: Intel Corporation Ethernet 10G 2P X520 Adapter (rev 01)
83:00.1 Ethernet controller: Intel Corporation Ethernet 10G 2P X520 Adapter (rev 01)

I know, lots of connectivity, but only this card supports Ethernet.

Thanks in advance!

 

[Chris]

Hi,
you will have to add an auto eno3 for the interface to be brought up on boot, see also https://wiki.debian.org/NetworkConfiguration#Setting_up_an_Ethernet_Interface

 

[Oblique]

This is a hack if you ask me but until the root cause and a remidy come about, this will allow the second port to operate after boot time without using ifup after every reboot.

auto lo eno3 #    <--- add offending NIC interface here...
iface lo inet loopback

iface eno4 inet manual

iface eno3 inet manual

iface eno1 inet manual

iface eno2 inet manual

iface enp3s0f0 inet manual

iface enp3s0f1 inet manual

iface enp131s0f0 inet manual

iface enp131s0f1 inet manual

auto vmbr0
iface vmbr0 inet static
    address 10.188.20.4/24
    gateway 10.188.20.10
    bridge-ports eno4
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

 

[Oblique]

Hi,
you will have to add an auto eno3 for the interface to be brought up on boot, see also https://wiki.debian.org/NetworkConfiguration#Setting_up_an_Ethernet_Interface

lol, thanks. We replied at the same time. I hope there is a "real" fix for this in the future.

 

[Chris]

lol, thanks. We replied at the same time. I hope there is a "real" fix for this in the future.

What do you mean with "real" fix? This is the way network interfaces are configured, so if you configure it without auto, you tell the system it should not bring up the interface at boot.

 

[Oblique]

What do you mean with "real" fix? This is the way network interfaces are configured, so if you configure it without auto, you tell the system it should not bring up the interface at boot.

My apologies, I was just thinking that all ports on a card would naturally come up under normal circumstances. I mean why would I ever want to shut off a NIC port? Anyway I am new and returning to PVE, so much has changed. Thanks for posting.