Wrong order of NIC links

[kv71h]

Hi, i installed a new network daughter card in my dell R730, the previous one had ports 1-4 assigned to links eno1-eno4 respectively. Linux now assigns the new card ports as such:

port 1=eno3
port 2=eno4
port 3=eno1
port 4=eno2

Does this have to do with IOmmu being on? I also don't know if the card is seated properly as it was very difficult to install, but i don't think it would function at all if even one pin wasn't connected. Is there a way to fix this at all?

 

[kv71h]

The altnames are as follows:

eno1=enp1s0f0
eno2=enp1s0f1
eno3=enp2s0f0
eno4=enp2s0f1

 

[kv71h]

I'm pretty sure linux changed from using ethX to predictable network interface device names, because ethX used to be a hardware race. Is this the same with enoX? Is there a way to use "predictable network interface device names" instead of enoX in proxmox?

 

[kv71h]

I see there is a guide to change the link names myself, but i don't see why proxmox automatically changes the link names in the first place?

 

[sw-omit]

Linux in general assigns names based on the hardware parameters it finds, and any hardware-change can mess with the things it finds and as such the order it receives.

A general example:

You have a 4 PCIe-slots, and in slot 1 and 3 you have a network-card, linux finds them and assigns the enp1 and enp2 ranges to it, numbering up, since because the slots are empty, the mother-board doesn't give the OS any information about them.
Now you buy another network-card and put it in slot 2, it will still number up, but since there is a device in-between, that device gets enp2, and the old enp2 card now has enp3.
Similarly, if a firmware or software update now allows the OS to view more info about a card, it might give it the name enp1p0 and enp2p0, since it is a card that it now knows can have more ports on it if you solder something and flip a switch.

Long story short, I've learned the hard way that names might change, which lets me to now start setting the names static to prevent future issues based on something I know is always static like the mac-address, till I do a hardware-change, and then I know that I only need to change the mac and that all other network-settings will just follow suit.

Why it kept the names but flipped the ordering though, that's more the fault of the card('s manufacturer) then the OS, most likely there are two sets of ports on it, and the order of the sets and the ports on each set does not go in the same direction, causing this strange counting. It could also have gone from 1>4 / 2>3 / 3>2 / 4>1 for example if they were in the same direction just a different direction then your old card.) I don't think a bad connection on 1 pin would cause this though.

 

[kv71h]

I think you understood me wrong, one card got replaced by one card, i still only have one network card. It's obvious that linux detects it as 2 individual network cards (that's why i asked if iommu had to do something with it) because of the names linux assigned it. I don't think i can do anything about the way the card presents itself, but i am still wondering why proxmox changes the names to enoX by default.

 

[_gabriel]

it's SystemD Linux that manage device and so the network device name, not Proxmox role.

https://forum.proxmox.com/threads/i...ernet-amongst-other-issues.150764/post-681743

 

[kv71h]

Yes, that's what i meant, i'm sorry for not being more clear. I meant to ask: "Does proxmox have any udev or systemd rules that change the way network devices are named?"

 

[kv71h]

As i explained before, the enoX links all have an altname that is a predictable network link name such as "enp1s0f0". That must mean something has renamed it from "enp1s0f0" to "eno1".

 

[_gabriel]

That must mean something has renamed it from "enp1s0f0" to "eno1".

Only :
1/ Hardware change like your new NIC
or
2/ Kernel update

 

[kv71h]

Only :
1/ Hardware change like your new NIC
or
2/ Kernel update

I still think you're understanding me wrong, this "renaming to enoX" goes for every proxmox node i have installed so far. You linked to this yourself here which takes me to this wiki page, that clearly states proxmox uses the "enoX" naming scheme for the first on-board NIC it detects.

I am asking why this happens, because usually, links get changed from eth0 to something like enp1s0f0. But in my current experience, they get changed from enp1s0f0 to eno1, with the altname being set to enp1s0f0. I want this the other way around, and i don't understand why this is happening. I have already fixed it in my system by changing the link names with .link systemd files, but i would still like to understand this issue.

 

[meyergru]

It is not up to Proxmox to decide how this is done, it is systemd, which Proxmox relies upon.

Historically, network device naming in the presence of multiple network devices was always a PITA. Then came systemd and changed how it was done - several times, see: https://manpages.debian.org/bookworm/systemd/systemd.net-naming-scheme.7.en.html

Currently, there are device names that are based on firmware, topology and other factors. What seems to be the case with your setup is that systemd thinks those devices are onboard and therefore get enoX names. While this assumption may be wrong anyway (since you could exchange those cards), the order will most probably depend on the PCIe topology. Potentially, a daughter board could be considered "on-board" and not a peripheral card.

If systemd thought the devices where on standard "peripheral" PCIe paths, it would probably name them enpXsY, but the order would still be dependend on what is detected in what order. Maybe the layout of the ports on your new NIC daughterboard is just different.

You can either use "link" entries in /etc/systemd/network that match on specific attributes like PCIe path or MAC address to have it otherwise like so (you did but it would probably not help the next time if the MACs change again):

[Match]
# This is the MAC address of the card to change.  You can also
# match against other properties like PCI/USB slot.
MACAddress=d8:5e:d3:11:22:33

[Link]
# This is the name to assign.  It must not conflict with any existing
# device, so be careful if you use a name like eth0 which can fail
# unexpectedly if you plug in another device that ends up on eth0 first.
Name=ens0

or you can use the pre-systemd scheme by adding "net.ifnames=0 biosdevname=0" to the Proxmox kernel commandline. In that case, the devices are numbered ethX in the order they are being detected, like in the olden days. I do that with all Linux VMs, BTW, should I recreate the VM and get another MAC or have more/less/different PCIe slots assigned.

Whatever the case, I doubt that Proxmox could easily "fix" this.