NIC keeps changing interfaces on reboots

H

harmonyp

Member
Nov 26, 2020
196
4
23
46
It will either boot up as enp133s0f or enp129s0f, what would cause it to keep changing?

Code: 
4: enp133s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:10:18:c3:a1:80 brd ff:ff:ff:ff:ff:ff
5: enp133s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:10:18:c3:a1:82 brd ff:ff:ff:ff:ff:ff

4: enp129s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:10:18:c3:a1:80 brd ff:ff:ff:ff:ff:ff
5: enp129s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:10:18:c3:a1:82 brd ff:ff:ff:ff:ff:ff
 
Hi!

As you didn't specify anything about your system, this amounts to bit of guesswork.
Just to be sure, can you provide the output of pveversion -v?

But this is most likely due to the order of how the firmware probes the NICs on boot, which can differ between boots. This then results in that the kernel sees it at a different location. Not all that uncommon for large "enterprise" servers, where the firmware often does all sorts of funky things.

You can work around that by setting a fixed name for the interface using systemd.link(5)

Create a file named /etc/systemd/network/90-eth0.link with the following contents for the first NIC:
Code: 
[Match]
MACAddress=00:10:18:c3:a1:80

[Link]
Name=eth0

And for the second NIC /etc/systemd/network/91-eth1.link:
Code: 
[Match]
MACAddress=00:10:18:c3:a1:82

[Link]
Name=eth1

I used eth0/1 as an example you, although you freely choose the name the interfaces should have by replacing eth0 or eth1 as appropriate.
The important things here are that 1) the MACAddress property is correct and 2) that the files are placed under /etc/systemd/network and end in .link.
 
  • Like
Reactions: nett_hier and hainh
I am having a similar issue:

On my Lenovo P700 with two built in NICs (eth1 and eno1) and under a new installation of PVE8, the two built in NICs keep swapping their names at every boot.
cheiss said:
Create a file named /etc/systemd/network/90-eth0.link with the following contents for the first NIC:
Code: 
[Match]
MACAddress=00:10:18:c3:a1:80

[Link]
Name=eth0

And for the second NIC /etc/systemd/network/91-eth1.link:
Code: 
[Match]
MACAddress=00:10:18:c3:a1:82

[Link]
Name=eth1

I used eth0/1 as an example you, although you freely choose the name the interfaces should have by replacing eth0 or eth1 as appropriate.
The important things here are that 1) the MACAddress property is correct and 2) that the files are placed under /etc/systemd/network and end in .link.
Click to expand...
I tried this (naming the NICs eth0 and eth1) but it does not work consistently for me: Yesterday, for example, it did not work at first. I then rebooted and it did work. After that I turned off the PC. Today I turned it on again and eth0 is missing and I got eno1 back. In addition, ip link show presents yet another set of altnames for both interfaces:

Code: 
: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 12:34:56:78:90:ab brd ff:ff:ff:ff:ff:ff
    altname enp8s0
3: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 12:34:56:78:90:ac brd ff:ff:ff:ff:ff:ff
    altname enp0s25
5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 12:34:56:78:90:ac brd ff:ff:ff:ff:ff:ff
(the MAC addresses are not the actual ones)

pveversion -v:
Code: 
proxmox-ve: 8.1.0 (running kernel: 6.5.11-8-pve)
pve-manager: 8.1.4 (running version: 8.1.4/ec5affc9e41f1d79)
proxmox-kernel-helper: 8.1.0
proxmox-kernel-6.5: 6.5.11-8
proxmox-kernel-6.5.11-8-pve-signed: 6.5.11-8
proxmox-kernel-6.5.11-7-pve-signed: 6.5.11-7
proxmox-kernel-6.5.11-4-pve-signed: 6.5.11-4
ceph-fuse: 17.2.7-pve1
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx8
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.0
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.3
libpve-access-control: 8.0.7
libpve-apiclient-perl: 3.3.1
libpve-common-perl: 8.1.0
libpve-guest-common-perl: 5.0.6
libpve-http-server-perl: 5.0.5
libpve-network-perl: 0.9.5
libpve-rs-perl: 0.8.8
libpve-storage-perl: 8.0.5
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 5.0.2-4
lxcfs: 5.0.3-pve4
novnc-pve: 1.4.0-3
proxmox-backup-client: 3.1.3-1
proxmox-backup-file-restore: 3.1.3-1
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.2.3
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.4
proxmox-widget-toolkit: 4.1.3
pve-cluster: 8.0.5
pve-container: 5.0.8
pve-docs: 8.1.3
pve-edk2-firmware: 4.2023.08-3
pve-firewall: 5.0.3
pve-firmware: 3.9-1
pve-ha-manager: 4.0.3
pve-i18n: 3.2.0
pve-qemu-kvm: 8.1.2-6
pve-xtermjs: 5.3.0-3
qemu-server: 8.0.10
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.2-pve1

Is there anything else I could try?

Thanks!
 
mow said:
Does proxmox use systemd for network config?
Click to expand...
No, it uses ifupdown2.
But this configuration is not related to systemd-networkd, but rather is applied by systemd itself on boot.

proxwolfe said:
I tried this (naming the NICs eth0 and eth1) but it does not work consistently for me: Yesterday, for example, it did not work at first. I then rebooted and it did work. After that I turned off the PC. Today I turned it on again and eth0 is missing and I got eno1 back. In addition, ip link show presents yet another set of altnames for both interfaces:
Click to expand...
Are you sure the mac addresses match? Have you checked that whether they stay consistent after each reboot?
Because if they do, than your .link files are not configured correctly.

Also, have you tried updating the firmware and checking, if there is any odd setting in there? Maybe even contacting the firmware vendor, if interfaces disappear after reboots?
 
It happened again!

Last time I booted, eth0 (or was it eth1?) was renamed to eth2. Today, I have back eth0 and eth1.

mow said:
Search through `dmesg` and try to find where and by what the interfaces get renamed.
Click to expand...
Thank you for the suggestion. I did check dmesg but all I found was:

e1000e 0000:00:19.0 eth0: renamed from eth1

No hint as to who or what renamed it...
cheiss said:
Are you sure the mac addresses match? Have you checked that whether they stay consistent after each reboot?
Click to expand...
Yes, they don't appear to change. I just compared the output of 'ip a' with my .link files - and they match (but then, on this boot, the interfaces are named as expected. I should have checked the last time when the names were wrong...)
 
My current working hypothesis is that the interface (I think it happens to only one of the two) changes its name upon a reboot (as opposed to a shutdown and restart). I don't have enough data points to support the hypothesis yet but I will observe. And I don't know if it helps to know that is is how it goes.

But does that make sense? Is that a thing? Is there a mechanism that could cause such behaviour?
 
I exactly have the same situation.

my first boot is:
Code: 
[    8.317765] igb 0000:04:00.0: added PHC on eth0
[    8.317788] igb 0000:04:00.0: eth0: (PCIe:2.5Gb/s:Width x1) 6c:0b:84:67:03:79
[    8.317851] igb 0000:04:00.0: eth0: PBA No: 000300-000
[    8.409221] igb 0000:04:00.0 eno1: renamed from eth0
[    8.441103] e1000e 0000:00:19.0 eth0: (PCI Express:2.5GT/s:Width x1) 6c:0b:84:67:03:78
[    8.441109] e1000e 0000:00:19.0 eth0: Intel(R) PRO/1000 Network Connection
[    8.441147] e1000e 0000:00:19.0 eth0: MAC: 11, PHY: 12, PBA No: FFFFFF-0FF

my next boot is:
Code: 
[    8.358739] igb: Intel(R) Gigabit Ethernet Network Driver
[    8.387046] igb 0000:04:00.0: added PHC on eth0
[    8.387083] igb 0000:04:00.0: Intel(R) Gigabit Ethernet Network Connection
[    8.387085] igb 0000:04:00.0: eth0: (PCIe:2.5Gb/s:Width x1) 6c:0b:84:67:03:79
[    8.387132] igb 0000:04:00.0: eth0: PBA No: 000300-000
[    8.433337] e1000e 0000:00:19.0 eth1: (PCI Express:2.5GT/s:Width x1) 6c:0b:84:67:03:78
[    8.433344] e1000e 0000:00:19.0 eth1: Intel(R) PRO/1000 Network Connection
[    8.433382] e1000e 0000:00:19.0 eth1: MAC: 11, PHY: 12, PBA No: FFFFFF-0FF
[    8.901290] igb 0000:04:00.0 eno1: renamed from eth0

As you can see, I have two physical NICs. Let's say igb and e1000e. And my machine model is Lenovel ThinkStation P910.
1. At every boot, there always takes an action on igb to rename it from eth0 to eno1. While no more any action on e1000e to do things like this.
2. Seeing the time stamp at the first boot, if that rename action happend right after igb process, then e1000e will get eth0. I think it because the first eth0 has been replaced by eno1. So there will not be duplication.
3. At the second boot, if the rename action happened after e1000e process, then e1000e will get eth1.

My questions are:
1. Why the OS doing so? Is this a racing condition while initialization?
2. When I make vmbr and attach to NIC by its name, this mechanism will totally mess up.
3. How could we fix it or any workaround?

Thanks very much.
 
Last edited:
benxtong said:
1. Why the OS doing so?
Click to expand...
Because someone couldn't leave well enough alone, and thanks to those morons now we can't just plug a SSD into a different machine anymore because it won't come up with usable network. (Right now I have extra work due to this problem as I have to bring up a clone of a system that has been installed with ssh pubkey access only and disabled root password and OF COURSE network didn't come up so I can't get in via ssh ...)
benxtong said:
Is this a racing condition while initialization?
Click to expand...
Yes.
 
proxwolfe said:
While I need to test it more, I gave the .link files another chance (https://wiki.debian.org/NetworkInterfaceNames#CUSTOM_SCHEMES_USING_.LINK_FILES). Somewhere in the long article @mow linked above, it says to not use names that the system would use (like "eth0"). Before, I had used "eth0" and "eth1". So this time I used "lan0" and "lan1". So far, it has been working for me...
Click to expand...
This works for me now. And appreciated the reminder for NIC naming.
Code: 
# dmesg | grep -ie "igb" -e "e1000e"
[    8.295705] e1000e: Intel(R) PRO/1000 Network Driver
[    8.295711] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    8.299650] igb: Intel(R) Gigabit Ethernet Network Driver
[    8.299655] igb: Copyright (c) 2007-2014 Intel Corporation.
[    8.300025] e1000e 0000:00:19.0: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
[    8.328343] igb 0000:04:00.0: added PHC on eth0
[    8.328381] igb 0000:04:00.0: Intel(R) Gigabit Ethernet Network Connection
[    8.328384] igb 0000:04:00.0: eth0: (PCIe:2.5Gb/s:Width x1) 6c:0b:84:67:03:79
[    8.328430] igb 0000:04:00.0: eth0: PBA No: 000300-000
[    8.328433] igb 0000:04:00.0: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
[    8.369374] igb 0000:04:00.0 eno1: renamed from eth0
[    8.379131] e1000e 0000:00:19.0 0000:00:19.0 (uninitialized): registered PHC clock
[    8.445175] e1000e 0000:00:19.0 eth0: (PCI Express:2.5GT/s:Width x1) 6c:0b:84:67:03:78
[    8.445181] e1000e 0000:00:19.0 eth0: Intel(R) PRO/1000 Network Connection
[    8.445219] e1000e 0000:00:19.0 eth0: MAC: 11, PHY: 12, PBA No: FFFFFF-0FF
[   12.838671] e1000e 0000:00:19.0 lan0: renamed from eth0
[   19.062499] igb 0000:04:00.0 eno1: entered allmulticast mode
[   19.065360] igb 0000:04:00.0 eno1: entered promiscuous mode
[   19.130230] e1000e 0000:00:19.0 lan0: entered allmulticast mode
[   19.133124] e1000e 0000:00:19.0 lan0: entered promiscuous mode
[   22.073814] igb 0000:04:00.0 eno1: igb: eno1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[   22.135497] e1000e 0000:00:19.0 lan0: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None

I now realize that the NIC drivers started over at almost simultaneously time by maybe two threads. And compete to initialize their properties independently.
 
This drove me nuts.. Couldn't figure out why proxmox wasn't booting correctly since I built my P700 several weeks ago. Then rebooted and lost connectivity earlier tonight. Finally got out of my chair and hooked up a monitor, keyboard, and mouse to the server and couldn't ping out either. Realized then it was the dual nic.

My solution was going into the Lenovo P700 BIOS and disabling one nic.
 
  • Like
Reactions: theprez1980
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom