broke my install with PVE 9.0 upgrade attempt

A

alexbstl

New Member
Aug 6, 2025
4
0
1
Hi- I'm sure I did something stupid but I'm messing around on a homelab that was running PVE 8.4 and I attempted to upgrade to 9.0 this morning. After rebooting, the webinterface didn't start. I ssh'd in and after doing some quick googling, I tried
Code: 
systemctl restart pvedaemon pveproxy pvestatd pvescheduler
and was greeted with the following: Screenshot 2025-08-06 at 10.06.13 AM.png
(all of the other errors are similar)

I've gone through and disabled repos associated with Debian 13 and attempted to remove Proxmox 9 apt sources and downgrade but nothing has worked so far. Any help would be much appreciated.
 
Ok, for anyone who finds this, I was able to successfully downgrade and restore my previous environment by doing the following:
  • disabled/removed any references to debian trixie (13) and revert them to bookworm (12) in /etc/apt/sources.list.d
  • disabled the legacy repo file in pve-install-repo.list (the repos are in proxmox.list) in /etc/apt/sources.list.d AFTER THIS STEP AND THE PREVIOUS, YOU MUST RUN
    Code: 
    apt update
  • reinstalled various packages with
    Code: 
     apt install --reinstall proxmox-ve pve-manager pve-cluster pve-ha-manager
    Note: I specifically had to allow a downgrade for pve-ha-manager to 4.0.7 from 5.0.4 (run apt install --allow-downgrades pve-ha-manager=4.0.7
  • downgrade proxmox-widget-toolkit with
    Code: 
    apt install --allow-downgrades proxmox-widget-toolkit=4.3.13
    (if you don't do this, the webui will just be a black screen)
  • restarted all the services
    Code: 
    [/LIST]
systemctl restart pve-cluster
systemctl restart pvedaemon pveproxy pvestatd pvescheduler

    So far, this seems to have restored proxmox to 8.4.9, and it seems to be working fine now.
 
Last edited:
root@proxmox:~# pve8to9 --full
= CHECKING VERSION INFORMATION FOR PVE PACKAGES =

Checking for package updates..
PASS: all packages up-to-date

Checking proxmox-ve package version..
PASS: proxmox-ve package has version >= 8.4-0

Checking running kernel version..
PASS: running kernel '6.14.8-2-pve' is considered suitable for upgrade.

= CHECKING CLUSTER HEALTH/SETTINGS =

SKIP: standalone node.

= CHECKING HYPER-CONVERGED CEPH STATUS =

SKIP: no hyper-converged ceph setup detected!

= CHECKING CONFIGURED STORAGES =

PASS: storage 'Media' enabled and active.
PASS: storage 'Photos' enabled and active.
PASS: storage 'local' enabled and active.
PASS: storage 'local-lvm' enabled and active.
INFO: Checking storage content type configuration..
PASS: no storage content problems found
PASS: no storage re-uses a directory for multiple content types.
INFO: Check for usage of native GlusterFS storage plugin...
PASS: No GlusterFS storage found.
INFO: Checking whether all external RBD storages have the 'keyring' option configured
SKIP: No RBD storage configured.

= VIRTUAL GUEST CHECKS =

INFO: Checking for running guests..
WARN: 3 running guest(s) detected - consider migrating or stopping them.
INFO: Checking if LXCFS is running with FUSE3 library, if already upgraded..
SKIP: not yet upgraded, no need to check the FUSE library version LXCFS uses
INFO: Checking for VirtIO devices that would change their MTU...
PASS: All guest config descriptions fit in the new limit of 8 KiB
INFO: Checking container configs for deprecated lxc.cgroup entries
WARN: The following CTs have 'lxc.cgroup' keys configured, which will be ignored in the new default unified cgroupv2:
CT 101
Often it can be enough to change to the new 'lxc.cgroup2' prefix after the upgrade to Proxmox VE 7.x
INFO: Checking VM configurations for outdated machine versions
PASS: All VM machine versions are recent enough

= MISCELLANEOUS CHECKS =

INFO: Checking common daemon services..
PASS: systemd unit 'pveproxy.service' is in state 'active'
PASS: systemd unit 'pvedaemon.service' is in state 'active'
PASS: systemd unit 'pvescheduler.service' is in state 'active'
PASS: systemd unit 'pvestatd.service' is in state 'active'
INFO: Checking for supported & active NTP service..
PASS: Detected active time synchronisation unit 'chrony.service'
INFO: Checking if the local node's hostname 'proxmox' is resolvable..
INFO: Checking if resolved IP is configured on local node..
PASS: Resolved node IP '192.168.2.180' configured and active on single interface.
INFO: Check node certificate's RSA key size
PASS: Certificate 'pve-root-ca.pem' passed Debian Busters (and newer) security level for TLS connections (4096 >= 2048)
PASS: Certificate 'pve-ssl.pem' passed Debian Busters (and newer) security level for TLS connections (2048 >= 2048)
INFO: Checking backup retention settings..
PASS: no backup retention problems found.
INFO: checking CIFS credential location..
PASS: no CIFS credentials at outdated location found.
INFO: Checking permission system changes..
INFO: Checking custom role IDs
PASS: no custom roles defined
INFO: Checking node and guest description/note length..
PASS: All node config descriptions fit in the new limit of 64 KiB
INFO: Checking if the suite for the Debian security repository is correct..
PASS: found no suite mismatch
INFO: Checking for existence of NVIDIA vGPU Manager..
PASS: No NVIDIA vGPU Service found.
INFO: Checking bootloader configuration...
WARN: systemd-boot meta-package installed but the system does not seem to use it for booting. This can cause problems on upgrades of other boot-related packages. Consider removing 'systemd-boot'
WARN: System booted in uefi mode but grub-efi-amd64 meta-package not installed, new grub versions will not be installed to /boot/efi! Install grub-efi-amd64.
INFO: Check for dkms modules...
SKIP: could not get dkms status
INFO: Check for legacy 'filter' or 'group' sections in /etc/pve/notifications.cfg...
INFO: Check for legacy 'notification-policy' or 'notification-target' options in /etc/pve/jobs.cfg...
PASS: No legacy 'notification-policy' or 'notification-target' options found!
INFO: Check for LVM autoactivation settings on LVM and LVM-thin storages...
NOTICE: storage 'local-lvm' has guest volumes with autoactivation enabled
NOTICE: Starting with PVE 9, autoactivation will be disabled for new LVM/LVM-thin guest volumes. This system has some volumes that still have autoactivation enabled. All volumes with autoactivations reside on local storage, where this normally does not cause any issues.
You can run the following command to disable autoactivation for existing LVM/LVM-thin guest volumes:

/usr/share/pve-manager/migrations/pve-lvm-disable-autoactivation

INFO: Check space requirements for RRD migration...
PASS: Enough free disk space for increased RRD metric granularity requirements, which is roughly 7.64 MiB.
INFO: Checking for IPAM DB files that have not yet been migrated.
PASS: No legacy IPAM DB found.
PASS: No legacy MAC DB found.
INFO: Checking if the legacy sysctl file '/etc/sysctl.conf' needs to be migrated to new '/etc/sysctl.d/' path.
PASS: Legacy file '/etc/sysctl.conf' exists but does not contain any settings.
INFO: Checking if matching CPU microcode package is installed.
WARN: The matching CPU microcode package 'amd64-microcode' could not be found! Consider installing it to receive the latest security and bug fixes for your CPU.
apt install amd64-microcode

= SUMMARY =

TOTAL: 43
PASSED: 31
SKIPPED: 5
WARNINGS: 5
FAILURES: 0

ATTENTION: Please check the output for detailed information!
 
I don't see from your original post how this warnings could affect you but two of them stand out that could cause a problem

1) WARN: System booted in uefi mode but grub-efi-amd64 meta-package not installed, new grub versions will not be installed to /boot/efi! Install grub-efi-amd64.

and

WARN: systemd-boot meta-package installed but the system does not seem to use it for booting. This can cause problems on upgrades of other boot-related packages. Consider removing 'systemd-boot'

Not sure if this would affect you like it did for me but yesterday my system wasn't booting anymore. I was able to fix it. You may want to check this post here for my steps if you go ahead and you have boot issues : https://forum.proxmox.com/threads/u...recommended-to-mount-it-to-boot-or-efi.169274

2) WARN: The following CTs have 'lxc.cgroup' keys configured, which will be ignored in the new default unified cgroupv2:
CT 101

That one sounds like it could be a problem but... again doesn't explain the issues you faced.


3) WARN: 3 running guest(s) detected - consider migrating or stopping them.

Also, not sure if you upgraded with VMs running but generally it is best to shut them down. I had to keep mine on my last system because it is running OPNSEnse and without that, no network.


Sorry I couldn't be of more help.
 
  • Like
Reactions: alexbstl and Kingneutron
the upgrade attempt wasn't my best decision, of course, but i figured that it didn't matter much because this is just my homelab and a little downtime isn't really a big deal for me.

Since then, I've resolved all the package warnings and the only two that remain are the Guests and the lxc.cgroup ones.


As far as the warning I'm not sure if I need to keep the VMs running because I pass through a truenas VM to containers, for storage and that throws warnings about storage not being online when I run pve8to9 as well.

I have no idea what to do about the last warning; it specifically references upgrading to Pve7, so I assumed it didn't really apply to me here.
 
Last edited:
your original posts very much looks like what happens if you don't correctly change all repositories to the Trixie/ PVE 9 ones and do a partial upgrade.. please read the upgrade guide carefully and follow the steps!
 
Hi, Alexbstl.

You said you resolved the warnings and some other notifications the script returned.

Could you tell us how you resolved them? In my case, I have the warning for Microcode.

Thank you.
 
alexandre.costa said:
Could you tell us how you resolved them? In my case, I have the warning for Microcode.
Click to expand...
I also had the warning for microcode package missing (intel in my case). and everything went wrong after upgrade (nfs was not working, tons of modules not loaded...). Install amd64-microcode before upgrading. For this add this in /etc/apt/sources.list.d/debian.sources

Types: deb
URIs: https://deb.debian.org/debian/
Suites: trixie
Components: main non-free-firmware
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg

Then apt update && apt upgrade -y && apt install amd64-microcode

It seems [intel|amd64]-microcode is mandatory not just optional for many servers...
see here: https://forum.proxmox.com/threads/after-update-from-8-4-to-9-0-nfs-doesnt-work.169327/post-789621
 
Last edited:
  • Like
Reactions: alexbstl
titou10 said:
It seems [intel|amd64]-microcode is mandatory not just optional for many servers...
see here: https://forum.proxmox.com/threads/after-update-from-8-4-to-9-0-nfs-doesnt-work.169327/post-789621
Click to expand...
Not specific to ProxMox, but I would recommend having this package installed when you can on physical hosts (not needed on vms obviously). There have been too many security issues with microcode over the past few years to ignore it imo. I can see reasons why you might not want this, but it is all a risk calculation to consider.
 
Thanks, @titou10 !

It worked on proxmox 8 (Debian 12), but only after I added "non-free-firmware" to the other line:

Code: 
deb http://ftp.br.debian.org/debian bookworm main contrib non-free-firmware

I haven't upgraded yet, but the proxmox analysis script stopped complaining.

Thank you very much,
 
You must log in or register to reply here.
Top Bottom