Lost Management Interface

wdatkinson

New Member
Mar 16, 2021
14
1
3
49
I've been running proxmox on a DL380G7 for a couple of years. I've slowly been migrating machines from an old ESXI server. Yesterday, while troubleshooting an unrelated network issue, I noticed that I was significantly behind on updates so I upgraded using the option in the web gui. Upon a reboot, I had zero connectivity into the box either on vmbr0 (management, single VLAN access port) or vmbr1 (trunked VM bridge).

After grabbing a crash cart of sorts, I began to poke around. I tried rebooting, using systemctl to restart the networking service, and many other options that my internet searching turned up.

Eventually, I was able to get vmbr1 up by using the 'ip link set vmbr1 up' command. After a short while, I began to see VM's with network connectivity. Using the qm command to bring up additional VM's I was able to restore most functionality, But I still had/have no management interface. Performing the same ip link command on vmbr0 appeared to bring it up, but it did not have an IP, despite what was configured in /etc/network/interfaces.

i have searched and read so many different posts, but have not found an answer. At this point, I feel like my only recourse is to re-install proxmox and import the existing VM's, but I still feel that I should be able to get the management interface back up, if for nothing else than to help ease operation while I migrate a handful of ESXI VM's over and then wipe ESXI and rebuilt it into a new proxmox server.

For now, if I need to access management functions, I have to RDP into a Win10 box and then use HP iLO to access the console, or go sit in front of the machine. Neither option is ideal

Happy to provide any additional information.
 

aaron

Proxmox Staff Member
Staff member
Jun 3, 2019
3,152
536
118
Can you provide the output of the ip a command and the contents of /etc/network/interfaces?
 

wdatkinson

New Member
Mar 16, 2021
14
1
3
49
With no management interface, I'm stuck with taking pictures of the monitor either live or in ilo.

Attached is the /etc/network/interfaces

The output of ip a is ~136 entries, so not sure how to capture that in its entirety.
 

Attachments

  • interfaces.jpg
    interfaces.jpg
    530.3 KB · Views: 20

aaron

Proxmox Staff Member
Staff member
Jun 3, 2019
3,152
536
118
The output of ip a is ~136 entries, so not sure how to capture that in its entirety.
You could pipe it to a pager like less: ip a | less
You can scroll easily and exit it by pressing q.

Checking if the enp3s0f1 is actually present. You did not change anything on the hardware right? Like changing some PCI cards or placing them in other slots? That could cause the NIC to get another name.
 

wdatkinson

New Member
Mar 16, 2021
14
1
3
49
Ok, I can grep it out and take a pic of the interface, but it is there. And f1 is vmbr1, which is feeding the VM's, f0 is vmbr0 which is the management interface.

And no changes were made. I merely ran updates, which appeared to hang in the web interface. I came back everal hours later. So I rebooted the machine. That's it.
 

aaron

Proxmox Staff Member
Staff member
Jun 3, 2019
3,152
536
118
Hmm, so let me recap the situation. Right now you don't have any network connection. You cannot ping the server or ping something from the PVE server?

The status of the interfaces vmbr0 and enp3s0f0 from the ip a command would be very interesting.
 

wdatkinson

New Member
Mar 16, 2021
14
1
3
49
vmbr0 is toast. So no management via web, ssh, etc. Nothing I do seems to bring it back.

vmbr1 is up, so those VM's that 's either started from the physical console or via ilo are running normally. But I have no easy way to admin the box. I don't sit in front of the console and have a single Windows desktop from which to run ilo. Everything else is some flavor of linux or MacOS.

I'm heading downstairs this morning and in between meetings will grab you an 'ip a | grep enp3s0f8' and snap a pic with my phone to post here.
 

aaron

Proxmox Staff Member
Staff member
Jun 3, 2019
3,152
536
118
If you do a grep, do include the lines before and after, just to be sure: grep -B 6 -A 6 enp3s0f0. Also do it for vmbr0
 
Mar 23, 2021
2
0
1
Just had a very similar effect: after installing an additional network card, the numbering of interfaces has changed, enp3s0 no longer available, instead the first available interface is enp4s0. Once the vmbr0 got connected to enp3s0, it does not work, there is no web-gui connection.

How I solved it the dirty way:
in web-gui, without the new card, anticipate the correct interface after installation and connect it to the wanted Linux Bridge.
One may watch the process by "IP a" at the console.
Then install the netweok card, and the old bridge works with the new interface enp4s0, instead of enp3s0.

If the first interface is named eno1 (on the motherboard), then adding a network card works without problems.
My enp3s0 and enp4s0 are on the mortherboard too. Strange, all 3 boxes are identical and installed at the same day, same ISO.

I hope, there will be a more elegant way.

Yours, Wolf7
 
Last edited:

wdatkinson

New Member
Mar 16, 2021
14
1
3
49
I have no access to the managment interface, vmbr0, or it's associated physical interface. Vmbr1 is up (trunked) and VM's are running. If i need to do something with the VM's I either have to go to the console or ilo to the console and run qm from the shell.

Again, no hardware has changed. Nothing added, nothing removed. I merely attempted an upgrade which appeared to hang. After allowing several hours for it to complete, I gave up and rebooted and here i am.

Proxmox is installed in a USB drive and the 8 SAS drives in the DL380 are my storage. So I'm wondering if I should just re-install Proxmox from scratch and import the VM's from the storage array, however that might work.
 
Mar 23, 2021
2
0
1
Hi, well it came to my mind, that your GUI is lost, sorry. You may try to edit from console /etc/network/interfaces and "anticipate" the correct interface name. In my case, it is enp4s0, below is an excerpt of my working version, pls check vmbr0 mentioning the 2nd interface enp4s0
auto lo
iface lo inet loopback

iface enp3s0 inet manual

iface enp4s0 inet manual

iface enp5s0 inet manual

iface enp6s0 inet manual

iface enp7s0 inet manual

..... other stuff about the new network adapter

auto vmbr0
iface vmbr0 inet static
address xyz
gateway xyz
bridge-ports enp4s0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094

auto vmbr1
iface vmbr1 inet static
address xyz
bridge-ports enp5s0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
#cluster traffic
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!