Proxmox does not start after attempted update from 7 to 8 to 9

L

Lars34

New Member
Oct 5, 2025
6
0
1
Good evening,

I think I messed something up. This morning I started an upgrade from Proxmox VE 7 to 8. This worked fine. All my LXCs and VMs started properly, Home Assistant OS was able to use the Zigbee stick. Then I ran pve8to9 and it indicated I needed to upgrade my intel-microcode package. To do that I needed to add `non-free-firmware` to the apt sources. I did this and then ran `apt update` and `apt upgrade`. This is when things stopped working. I now see these messages:

Code: 
Job for pvedaemon.service failed because the control process exited with error code.
See "systemctl status pvedaemon.service" and "journalctl -xeu pvedaemon.service" for details.
Job for pveproxy.service failed because the control process exited with error code.
See "systemctl status pveproxy.service" and "journalctl -xeu pveproxy.service" for details.
Job for pvestatd.service failed because the control process exited with error code.
See "systemctl status pvestatd.service" and "journalctl -xeu pvestatd.service" for details.
Job for pvescheduler.service failed because the control process exited with error code.
See "systemctl status pvescheduler.service" and "journalctl -xeu pvescheduler.service" for details.
.

I have tried to reboot, but this did not solve the issue.

Now, if I try to run a pve command like `pct list` I see this:
Code: 
> root@har-serv-01:/etc/pve# pct list
Attempt to reload PVE/HA/Config.pm aborted.
Compilation failed in require at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 20.
BEGIN failed--compilation aborted at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 20.
Compilation failed in require at /usr/share/perl5/PVE/API2/LXC/Status.pm line 24.
BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2/LXC/Status.pm line 29.
Compilation failed in require at /usr/share/perl5/PVE/CLI/pct.pm line 24.
BEGIN failed--compilation aborted at /usr/share/perl5/PVE/CLI/pct.pm line 24.
Compilation failed in require at /usr/sbin/pct line 6.
BEGIN failed--compilation aborted at /usr/sbin/pct line 6.



This is my /etc/apt/sources.list file:

Code: 
deb http://ftp.debian.org/debian bookworm main contrib non-free-firmware
deb http://ftp.debian.org/debian bookworm-updates main contrib non-free-firmware
deb http://security.debian.org/debian-security bookworm-security main contrib non-free-firmware
deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription
deb http://deb.debian.org/debian bookworm-backports main contrib non-free-firmware

Is there a path forward to fix this, or am I going to need to do a clean reinstall....
 
Last edited:
In Proxmox servers it's always recommended doing apt full-upgrade, not just apt upgrade.

What happens if you execute it now?
 
Ah I've tried that as well. It doesn't make any changes and it doesn't work, unfortunately
 
Onslow said:
In Proxmox servers it's always recommended doing apt full-upgrade, not just apt upgrade.
Click to expand...
Yes, apt dist-upgrade/apt full-upgrade must be used instead of apt upgrade as described in [0], because among other things the latter will make packages stay on a lower versioned package instead of resolving the package dependencies correctly.

Lars34 said:
Is there a path forward to fix this, or am I going to need to do a clean reinstall....
Click to expand...
On which versions were you on Proxmox VE 7, 8, and 9 (for the last a pveversion -v is sufficient)? If you're upgrading to PVE 9, the sources.list should also use trixie and there might be some added deb822 files in /etc/apt/sources.list.d/ which might conflict with the sources list in /etc/apt/sources.list.

What does the APT log in /var/log/apt/history.log show for the upgrade between PVE 7 to PVE 9 say (If it isn't in there, it is log-rotated in a gzipped archive, e.g. history.log.1.gz)?

[0] https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#system_software_updates
 
Thank you for helping me!

I was on the latest version of VE 7 before upgrading.

I haven't upgraded to version 9 yet, because I first tried to upgrade the intel microcode because the pve8to9 tool indicated I needed to do that before upgrading to VE 9.

However, now the pve8to9 tool isn't working anymore because of the same issue.

I have attached my /var/log/apt/history.log file.

The result of running pvevefsion -v is this:

Code: 
proxmox-ve: 8.4.0 (running kernel: 6.14.11-3-pve)
pve-manager: 8.4.14 (running version: 8.4.14/b502d23c55afcba1)
proxmox-kernel-helper: 9.0.4
pve-kernel-5.15: 7.4-15
proxmox-kernel-6.14.11-3-pve-signed: 6.14.11-3
proxmox-kernel-6.14: 6.14.11-3
proxmox-kernel-6.8: 6.8.12-15
proxmox-kernel-6.8.12-15-pve-signed: 6.8.12-15
pve-kernel-5.15.158-2-pve: 5.15.158-2
pve-kernel-5.15.102-1-pve: 5.15.102-1
ceph-fuse: 16.2.15+ds-0+deb12u1
corosync: 3.1.9-pve1
criu: 3.17.1-2+deb12u2
glusterfs-client: 10.3-5
ifupdown2: 3.3.0-1+pmx10
intel-microcode: 3.20250512.1~deb12u1
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libknet1: 1.30-pve2
libproxmox-acme-perl: 1.7.0
libproxmox-backup-qemu0: 1.5.2
libproxmox-rs-perl: 0.3.5
libpve-access-control: 9.0.3
libpve-apiclient-perl: 3.4.0
libpve-cluster-api-perl: 9.0.6
libpve-cluster-perl: 9.0.6
libpve-common-perl: 9.0.11
libpve-guest-common-perl: 6.0.2
libpve-http-server-perl: 6.0.4
libpve-network-perl: 0.11.2
libpve-rs-perl: 0.9.4
libpve-storage-perl: 9.0.13
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.6.0-3
proxmox-backup-client: 3.4.6-1
proxmox-backup-file-restore: 3.4.6-1
proxmox-backup-restore-image: 1.0.0
proxmox-firewall: 0.7.1
proxmox-kernel-helper: 9.0.4
proxmox-mail-forward: 0.3.3
proxmox-mini-journalreader: 1.5
proxmox-widget-toolkit: 5.0.6
pve-cluster: 8.1.2
pve-container: 5.3.3
pve-docs: 9.0.8
pve-edk2-firmware: 4.2025.02-4
pve-esxi-import-tools: 0.7.4
pve-firewall: 5.1.2
pve-firmware: 3.17-1
pve-ha-manager: 5.0.4
pve-i18n: 3.6.0
pve-qemu-kvm: 9.2.0-7
pve-xtermjs: 5.5.0-2
qemu-server: 8.4.3
smartmontools: 7.3-pve1
spiceterm: 3.3.1
swtpm: 0.8.0+pve1
vncterm: 1.8.1
zfsutils-linux: 2.2.8-pve1
 

Attachments

Last edited:
Lars34 said:
I was on the latest version of VE 7 before upgrading.

I haven't upgraded to version 9 yet, because I first tried to upgrade the intel microcode because the pve8to9 tool indicated I needed to do that before upgrading to VE 9.
Click to expand...
From the apt history it looks like there was an in-between upgrade to many packages from Proxmox VE 9:

Code: 
Start-Date: 2025-10-05  16:21:44
Commandline: apt upgrade
Install: [...] proxmox-kernel-6.14:amd64 (6.14.11-3, automatic), libjs-bootstrap5:amd64 (5.2.3+dfsg-8, automatic), javascript-common:amd64 (11+nmu1, automatic)
Upgrade: pve-docs:amd64 (8.4.1, 9.0.8), pve-edk2-firmware-ovmf:amd64 (4.2025.02-4~bpo12+1, 4.2025.02-4), proxmox-backup-restore-image:amd64 (0.7.0, 1.0.0), proxmox-default-kernel:amd64 (1.1.0, 2.0.0), proxmox-widget-toolkit:amd64 (4.3.13, 5.0.6), pve-firmware:amd64 (3.16-3, 3.17-1), libproxmox-acme-perl:amd64 (1.6.0, 1.7.0), libpve-cluster-api-perl:amd64 (8.1.2, 9.0.6), pve-edk2-firmware-legacy:amd64 (4.2025.02-4~bpo12+1, 4.2025.02-4), pve-ha-manager:amd64 (4.0.7, 5.0.4), libpve-apiclient-perl:amd64 (3.3.2, 3.4.0), libpve-storage-perl:amd64 (8.3.7, 9.0.13), libpve-guest-common-perl:amd64 (5.2.2, 6.0.2), [...], pve-i18n:amd64 (3.4.5, 3.6.0), proxmox-archive-keyring:amd64 (3.3, 4.0), libpve-http-server-perl:amd64 (5.2.2, 6.0.4), libpve-common-perl:amd64 (8.3.4, 9.0.11), libpve-notify-perl:amd64 (8.1.2, 9.0.6), libcorosync-common4:amd64 (3.1.9-pve1, 3.1.9-pve2), proxmox-kernel-helper:amd64 (8.1.4, 9.0.4), pve-edk2-firmware:amd64 (4.2025.02-4~bpo12+1, 4.2025.02-4), libpve-cluster-perl:amd64 (8.1.2, 9.0.6)
End-Date: 2025-10-05  16:22:43

But some of these were manually downgraded back to Proxmox VE 8, but not all of them (especially the core dependencies of those are on Proxmox VE 9 already with breaking changes). AFAIK there's no "revert upgrade" support here, so there isn't a way to directly revert back to the exact versions except doing something like apt install pve-docs=8.4.1 pve-edk2-firmware-ovmf=4.2025.02-4~bpo12+1 ..., which is a bit cumbersome when having no backup of the previous host's state, but would allow you to get pve8to9 running again to proceed the upgrade.
 
Last edited:
That's cumbersome, but I'm willing to try. Would creating a vm on my laptop with the Proxmox version 8 I need work? I would then look what the versions I need are and set those on the broken system
 
Do I understand right, that you want to know which versions had particular packages in PVE 8?
If so, here you are mine. This is up-to-date PVE 8.4.14.
 

Attachments

@dakralex @Onslow thank you so much for the help! I managed to downgrade the relevant packages and now Proxmox is starting again and everything is working! Now on to upgrade to PVE 9 (but I'll first make a backup this time :D)

For anyone else who has this problem, this is the command I executed. I don't recommend running it directly yourself, but maybe you can use it for information.
Code: 
apt install libcgi-pm-perl=4.55-1 libcorosync-common4=3.1.9-pve1 libcpg4=3.1.9-pve1 libmodule-find-perl=0.16-2 libmodule-scandeps-perl=1.31-2+deb12u1 libnbd0=1.14.2-1 libndctl6=76.1-1 libnet-cidr-perl=0.21-2 libnet-server-perl=2.013-2 libnet-snmp-perl=6.0.1-7 libpcre2-16-0=10.42-1 libpmem1=1.12.1-2 libpmemblk1=1.12.1-2 libproc-processtable-perl=0.634-1+b2 libproxmox-acme-perl=1.6.0 libproxmox-acme-plugins=1.6.0 libpve-access-control=8.2.2 libpve-apiclient-perl=3.3.2 libpve-cluster-api-perl=8.1.2 libpve-cluster-perl=8.1.2 libpve-common-perl=8.3.4 libpve-guest-common-perl=5.2.2 libpve-http-server-perl=5.2.2 libpve-notify-perl=8.1.2 libpve-storage-perl=8.3.7 libquorum5=3.1.9-pve1    libvirt0=9.0.0-4+deb12u2  libvotequorum8=3.1.9-pve1  libyajl2=2.1.0-3+deb12u2  lsscsi=0.31-1+b1  munin-common=2.0.73-1 munin-libvirt-plugins=0.0.7-1 munin-node=2.0.73-1 munin-plugins-core=2.0.73-1 munin-plugins-extra=2.0.73-1 novnc-pve=1.6.0-2 proxmox-archive-keyring=3.3 proxmox-backup-restore-image=0.7.0 proxmox-default-kernel=1.1.0 proxmox-kernel-6.8.12-14-pve-signed=6.8.12-14 proxmox-kernel-helper=8.1.4 proxmox-widget-toolkit=4.3.13 pve-docs=8.4.1 pve-edk2-firmware-legacy=4.2025.02-4~bpo12+1 pve-edk2-firmware-ovmf=4.2025.02-4~bpo12+1 pve-edk2-firmware=4.2025.02-4~bpo12+1 pve-firmware=3.16-3 pve-ha-manager=4.0.7 pve-i18n=3.4.5
 
You must log in or register to reply here.
Top Bottom