[SOLVED] Upgrade to 7 - No Network after Reboot

Younex

Well-Known Member
Jan 30, 2018
92
21
48
43
Hi Guys.

After reboot network is lost on one of my nodes.

I see all NIC are in stat down.

I can simple bring back network by this command: "/etc/init.d/networking/ restart".

I wonder why it don´t boot at first place.

Maybe i´m missing the hw-mac on bridges?

Code:
iface eno1 inet manual


iface eno2 inet manual


iface eno3 inet manual


iface eno4 inet manual


iface ens6f0 inet manual


iface ens6f1 inet manual


#auto bond2

iface bond2 inet manual

        bond-slaves eno1 eno2

        bond-miimon 100

        bond-mode active-backup


auto vmbr0

iface vmbr0 inet static

        address  192.168.90.103/24

        gateway  192.168.90.1

        bridge-ports bond2

        bridge-stp off

        bridge-fd 0


As i have a bond inside vmbr0, would i need to name two hw-mac?

Thx!
 
Please provide the complete journal since the last boot (journalctl -b > journal.txt) and attach it here.
 
On one of my nodes the upgrade/change to ifupdown2 from ifupdown left the networking service no longer enabled at boot. May not be the case with you, but sudo systemctl is-enabled networking should return enabled. If not, sudo systemctl enable networking fixed it for me.
 
  • Like
Reactions: jamesharr
@vesalius.
Thanks, this was not the case.

Hi MIra,
thanks for you reply.
Attached are 2 files from 2 nodes of mine.
I suppose all nodes have the same issue. Want to restart them only after i fixed it on some :)
Thank you.
 

Attachments

  • journal_node4.txt
    269.8 KB · Views: 10
  • journal_node3.txt
    257.2 KB · Views: 4
I found this part in the journal.
1639413181086.png

Maybe it is related with this issue?
https://forum.proxmox.com/threads/proxmox-6-network-wont-start.56362/

But it seems the service do not exist:
1639413249932.png

Last try was to just have one network interface with no bond or bridge inside /etc/network/interfaces.
Same behaviour.
Just with manual restart of network-services it become online.
I read a thread where i found any hardware device can delay initilization which result in skipping bringing network up.
Not sure, maybe somebody have an idea about this.

Seems i´m not the only one:
https://forum.proxmox.com/threads/h...iling-to-reboot-pve-7-1-7.100820/#post-437050
 
Last edited:
The issue seems to be systemd-udev-settle.service as seen above.
Code:
Dec 13 17:13:39 prox-4-mb systemd-udevd[1625]: sdh1: Spawned process '/usr/bin/systemd-run /sbin/lvm pvscan --cache 8:113' [2002] is taking longer than 59s to complete
Dec 13 17:13:39 prox-4-mb systemd-udevd[1623]: sdg1: Spawned process '/usr/bin/systemd-run /sbin/lvm pvscan --cache 8:97' [2004] is taking longer than 59s to complete
Dec 13 17:13:39 prox-4-mb systemd-udevd[1609]: sdg1: Worker [1623] processing SEQNUM=4427 is taking a long time
Dec 13 17:13:39 prox-4-mb systemd-udevd[1609]: sdh1: Worker [1625] processing SEQNUM=4443 is taking a long time

The `ifupdown-pre` service depends on it, and as a result fails.
Maybe check your HBA/RAID controller and your disks.
 
Thank you Mira.

Finally this dirty hack fixed my issue:
Code:
systemctl mask systemd-udev-settle.service
 
Thank you Mira.

Finally this dirty hack fixed my issue:
Code:
systemctl mask systemd-udev-settle.service

Hi Younex,

I have the exact same issue.

I used the "dirty hack" for systemd-udev-settle.service, but my network is still not coming up. Did you do anything else to solve that issue?

Thanks
 
Hi eepohboy,

no, but it would be good to have more information on your issue.

Do you have any remote tool to monitor bootup sequence (ilo on hp servers for example)?
 
Hi eepohboy,

no, but it would be good to have more information on your issue.

Do you have any remote tool to monitor bootup sequence (ilo on hp servers for example)?

Hi Younex,

]Thanks for taking time to reply.

Trying to make a long story short:
  • I have a NUC like box (MinisForum HM80) which I can connect to my monitor
  • I had a Proxmox installation from v6. Updated to the latest version and kernel, and I think I ended up with the same issue discussed here and here
  • didn't know I had that issue, so after hitting a dead end, I nuked my installation and reinstalled Proxmox with the latest version (v7.1-2)
  • faced the same issue you posted here after the new installation, and tried the fixes posted above
  • what lead me to believe it was the kernel issue was that reboot/shutdown would hang at "kvm exiting hardware virtualization"
  • after hitting a wall again, I tried downloading an earlier version of the installer from the Proxmox website (v7.0-2)
  • now Proxmox boots perfectly
    • no udev service issue
    • no network issue
    • no reboot/shutdown issue
New problem is I can't resolve any DNS. I can ping 1.1.1.1, 8.8.8.8, etc. Can't even resolve "google.com". Looking through the forums to fix that issue now.
 
On one of my nodes the upgrade/change to ifupdown2 from ifupdown left the networking service no longer enabled at boot. May not be the case with you, but sudo systemctl is-enabled networking should return enabled. If not, sudo systemctl enable networking fixed it for me.

TY, enabling networking fixed it for me.

I likely created the situation myself when I tried to clean up some uninstalled packages. This was an in-place upgrade from 6.5 -> 7.1 and then I decided to go clean out some old kernels and purge some packages with an "rc" status. I'm not entirely sure where it was that I shot myself in the foot. Posting this in case anyone else does the same thing.
Bash:
# Me cleaning up
apt autoremove
dpkg -l | grep ^rc
dpkg -P pve-kernel-5.4.162-1-pve  linux-image-4.19.0-18-amd64 ifupdown ifenslave gsfonts bsdmainutils python
[FONT=Open Sans]apt remove pve-kernel-5.4.166-1-pve pve-kernel-5.4.73-1-pve[/FONT]
apt autoremove
dpkg -l | grep ^rc
dpkg -P pve-kernel-5.4 pve-kernel-5.4.166-1-pve pve-kernel-5.4.73-1-pve
reboot

# problem exists after boot
systemctl enable networking
reboot

# problem resolved
 
  • Like
Reactions: vesalius
I was in the same situation today. On 3 PVE hosts no more network, and 1 still worked.
It turned out, the the installation of ifupdown2 (necessary to change network configuration from the GUI without rebooting) seemed to have uninstalled ifupdown.
When I purged the remaining configuration from ifupdown, the networking was disabled
 
  • Like
Reactions: vesalius

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!