Boot hangs after Upgrade to Proxmox 8

ubu

Renowned Member
Nov 24, 2017
374
72
68
49
After the Upgrade booting hangs permanently at starting networking.service.

I have disabled networking, system bootsup, login via RMC works, starting networking by hand with systemctl start networking works.

If i reenable networking system hangs again
 

Attachments

  • boot-hangs.txt
    197.4 KB · Views: 13
  • interfaces.txt
    1.3 KB · Views: 6
Boot with networking disabled, started later by hand
 

Attachments

  • boot-works.txt
    551.8 KB · Views: 5
Already tried that, does not help. Since the network can be started after the boot, Interfaces and config seem to be ok
 
Good to hear that you could fix the issue!

So far the root cause of the hang was not clear. I took another look now, and might have a theory how/why the hang happens. It would be great if you could run some checks (see below) to find out whether this theory matches your experience.

In general, it seems like networking hangs at boot if both ntpsec and ntpsec-ntpdate are installed -- the reason seems to be a deadlock during network setup which only triggers if both these packages are present.

However, this seems to be the case for PVE 7.4 already (can be reproduced by installing both packages). But according to you and others [3], the hangs did not happen before the upgrade to PVE 8. One explanation would be:
  • before the upgrade to PVE 8, at least one of ntpsec and ntpsec-ntpdate was not installed, so the node boots fine
  • after the upgrade to PVE 8, both ntpsec and ntpsec-ntpdate are installed, so the node now hangs at boot
If this is the case, the question is why the upgrade installs those packages. I suspect this is due to changes in ntp-related packages between Debian Bullseye and Bookworm. For example, if I install ntp and ntpsec-ntpdate on PVE 7.4 and upgrade to 8, the upgrade also installs ntpsec (because ntp depends on ntpsec since Bookworm [4]), and the node hangs after reboot.

To find out what happened in your case, could you maybe post the following?
  • the output of dpkg -l | grep ntp
  • the entry of /var/log/apt/history.log corresponding to the upgrade to PVE 8. The entry should look as follows:
    Code:
    Start-Date: 2023-07-...
    Commandline: apt dist-upgrade
    Install: <long list of packages>
    Upgrade: <long list of packages>
    Remove: <list of packages>
    End-Date: 2023-07-...
    This would allow us to find out if/why ntpsec or ntpsec-ntpdate got installed during upgrade.
Bitte Hinweis in pve7to8 einbauen
Note that there is a note in the upgrade guide already [5]. But yeah, if we can find a reliable way to detect this situation before the upgrade, this check could potentially be added to pve7to8.

[1] https://pve.proxmox.com/wiki/Upgrade_from_7_to_8#Network_Fails_on_Boot_Due_to_NTPsec_Hook
[2] https://salsa.debian.org/debian/ntpsec/-/blob/debian/unstable/debian/ntpsec-ntpdate.if-up#L40
[3] https://forum.proxmox.com/threads/proxmox-ve-8-0-released.129320/page-6#post-567264
[4] https://packages.debian.org/bookworm/ntp
[5] https://pve.proxmox.com/wiki/Upgrade_from_7_to_8#Network_Fails_on_Boot_Due_to_NTPsec_Hook
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!