Proxmox 5.4 to 6.0 : Strange network issues

[t.lamprecht]

Yes, the Ethernet port is active; LED is blinking. After aborting the Proxmox Setup, i can also see that the Setup successfully obtained an Address via DHCP:

OK, so the NIC works after all with the kernel, which is a good starter.

But that one is not configured correctly anymore after the installation? It'd be interesting to have the perspective there..

 

[Lebedev]

Hi Thomas,

this is getting exciting! I found out that this issue has nothing todo with the realtek driver.
Earlier i said, that the Ethernet port is turned off as soon as Proxmox has boot up. This is correct, but it happens only, if an PCI Express SSD (M.2) is installed.

This is my storage configuration:

- SATA SSD (for Proxmox installation)
- M.2 SSD (Slot 1)
- M.2 SSD (Slot 2)

It doesn't matter if i put an M.2 SSD in the first Slot, but as soon as i put one into the second slot, the network link is down. It doesn't matter which SSD is used, i tried both. This reminds me of old times... hardware IRQ conflicts and so on.

So.. normally i would guess that this is an hardware issue (no fault of Proxmox). But interestingly Debian 10 does not have this issue. PartedMagic (Slackware) also has no problem.

 

[t.lamprecht]

Earlier i said, that the Ethernet port is turned off as soon as Proxmox has boot up. This is correct, but it happens only, if an PCI Express SSD (M.2) is installed.

holy moly, talk about weird things happening.. Nice for catching this!

So.. normally i would guess that this is an hardware issue (no fault of Proxmox). But interestingly Debian 10 does not have this issue. PartedMagic (Slackware) also has no problem.

Hmm, yeah, it is indeed strange. What does lspci -knn say? IT could be interesting to compare a call with an M.2 on the second slot and one without that M.2 device.

Also, what motherboard is the server using.

 

[Stoiko Ivanov]

Hmm - please take a look at the output of:
* `ip link show`
with and without the second M.2 installed

on a hunch I would guess that the interface does work but gets a new name (predictable network interface names sometimes are not really that predicatable, especially if you add another pci-device which changes the order of the NICs)
(see https://github.com/systemd/systemd/blob/master/src/udev/udev-builtin-net_id.c#L20
https://www.freedesktop.org/software/systemd/man/systemd.net-naming-scheme.html )

I hope this helps!

 

[Lebedev]

It's Desktop Hardware. The Motherboard is a Asus P8Z77-M Pro.
The PCI Express SSDs are two Samsung SM951.
The SSDs are installed via Asus Hyper M.2 X4 because the board has no M.2 Slots onboard.

If anyone would tell me this, i would answer him that this must be a hardware issue. Some compatiblity things regarding the PCIe Slots or whatever. But the fact that Debian 7, 8, 9, 10 and Slackware work well gnaws on me.

As far as i know, there are three different types of M.2 SSDs:

B-Key (SATA)
M-Key (PCIe AHCI)
M-Key (PCIe NVMe)

The Samsung SM951 are the AHCI-type. However, they are still regular PCI Express devices.

without SSD
klick

SSD in the first Slot
klick

SSD in the second Slot
klick

 

[Lebedev]

Hmm - please take a look at the output of:
* `ip link show`
with and without the second M.2 installed

on a hunch I would guess that the interface does work but gets a new name (predictable network interface names sometimes are not really that predicatable, especially if you add another pci-device which changes the order of the NICs)
(see https://github.com/systemd/systemd/blob/master/src/udev/udev-builtin-net_id.c#L20
https://www.freedesktop.org/software/systemd/man/systemd.net-naming-scheme.html )

I hope this helps!

I got used to disconnect all storage media when installing an operating system, except the one that the operating system is installed on.

I think you are right. The PCIe SSDs were not plugged in, when i've done the Proxmox VE Installation. So the Ethernet interface which i've selected as management interface is not available anymore (since it got a new name after i've plugged the PCIe SSDs in).

Oh boy... what's the easiest way to fix this?

Edit:

I have edited /etc/network/interfaces: enp3s0 changed to enp4s0
Everything works now as expected.

Thank you for your help, guys!
Now i can finally try out Proxmox VE

 

[t.lamprecht]

good hunch stoiko!

Oh boy... what's the easiest way to fix this?

edit /etc/network/interfaces and swap all old occurences of enpXY... with the id the NIC has now and reboot.

 

[Lebedev]

I have edited /etc/network/interfaces: enp3s0 changed to enp4s0
Everything works now as expected.

Thank you for your help, guys!
Now i can finally try out Proxmox VE

Link is now up, and i am able to ping the Proxmox VE Server. I have still no access to the WebGUI, not via HTTP nor HTTPS.
Do i have to change firewall rules or anything like that?

Edit: nevermind, i forgot that Port 8006 is used!

 

[iankun]

Just wanted to drop by this old thread to provide an update.
After revisiting migration to OVS on the Proxmox cluster post-migration to 7.0, I managed to solve the problem by enabling firewall on the network interface of the VMs. If you have "Firewall" checkbox "checked", the interface is converted into OVSInt instead of using tap. This solves all of the issues and network works as expected. So, just to kick the proverbial "dead horse", usage of tap interfaces by Proxmox is broken and is not supported by OVS (see: docs.openvswitch.org/en/latest/faq/issues/ and search for tap).

 

[webmastermt]

hi guys,

i solved problem change to harddisk. (For entered disable state and locking state)

i was using sata ssd

i changed nvme disk and solved.

i think sata not enough for 10 or 1 gigabit lan

sory for my bad english