[SOLVED] pve-manager update/install stuck

T

triplus

New Member
Nov 22, 2023
3
0
1
Hi

I tried doing a regular update on my single-node Proxmox homelab this morning (via GUI). The update got stuck during the "setting up" of pve-manager. I've tried doing some general troubleshooting (installing previous version, restarting, re-installing), but it keeps getting stuck on the below:

Code: 
root@proxmox:~# apt dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up pve-manager (8.1.4) ...
  LVM configuration valid.

When I manage to kill the update process (it's really stuck, the only way to get it to stop is to restart the host), it shows below error:
Code: 
Job for pvescheduler.service canceled.
dpkg: error processing package pve-manager (--configure):
 installed pve-manager package post-installation script subprocess returned error exit status 1

I tried rolling back to 8.1.3, but same place where it gets stuck. I let it "hang" for 1h+ on that spot already but nothing happened. Below some troubleshooting info:

Code: 
root@proxmox:~# pveversion -v
proxmox-ve: 8.1.0 (running kernel: 6.5.11-8-pve)
pve-manager: not correctly installed (running version: 8.1.4/ec5affc9e41f1d79)
proxmox-kernel-helper: 8.1.0
pve-kernel-6.2: 8.0.5
proxmox-kernel-6.5: 6.5.11-8
proxmox-kernel-6.5.11-8-pve-signed: 6.5.11-8
proxmox-kernel-6.5.11-7-pve-signed: 6.5.11-7
proxmox-kernel-6.2.16-20-pve: 6.2.16-20
proxmox-kernel-6.2: 6.2.16-20
pve-kernel-6.2.16-3-pve: 6.2.16-3
ceph-fuse: 17.2.6-pve1+3
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx8
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.0
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.3
libpve-access-control: 8.0.7
libpve-apiclient-perl: 3.3.1
libpve-common-perl: 8.1.0
libpve-guest-common-perl: 5.0.6
libpve-http-server-perl: 5.0.5
libpve-network-perl: 0.9.5
libpve-rs-perl: 0.8.8
libpve-storage-perl: 8.0.5
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 5.0.2-4
lxcfs: 5.0.3-pve4
novnc-pve: 1.4.0-3
proxmox-backup-client: 3.1.3-1
proxmox-backup-file-restore: 3.1.3-1
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.2.3
proxmox-mini-journalreader: 1.4.0
proxmox-widget-toolkit: 4.1.3
pve-cluster: 8.0.5
pve-container: 5.0.8
pve-docs: 8.1.3
pve-edk2-firmware: 4.2023.08-3
pve-firewall: 5.0.3
pve-firmware: 3.9-1
pve-ha-manager: 4.0.3
pve-i18n: 3.2.0
pve-qemu-kvm: 8.1.2-6
pve-xtermjs: 5.3.0-3
qemu-server: 8.0.10
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.2-pve1


Code: 
root@proxmox:~# cat /var/log/apt/history.log

Start-Date: 2024-02-02  10:04:41
Commandline: apt-get dist-upgrade
Install: proxmox-kernel-6.5.11-8-pve-signed:amd64 (6.5.11-8, automatic)
Upgrade: pve-edk2-firmware-ovmf:amd64 (4.2023.08-2, 4.2023.08-3), libpve-rs-perl:amd64 (0.8.7, 0.8.8), proxmox-mail-forward:amd64 (0.2.2, 0.2.3), pve-edk2-firmware-legacy:amd64 (4.2023.08-2, 4.2023.08-3), proxmox-kernel-6.5:amd64 (6.5.11-7, 6.5.11-8), proxmox-backup-file-restore:amd64 (3.1.2-1, 3.1.3-1), libc6:amd64 (2.36-9+deb12u3, 2.36-9+deb12u4), locales:amd64 (2.36-9+deb12u3, 2.36-9+deb12u4), ifupdown2:amd64 (3.2.0-1+pmx7, 3.2.0-1+pmx8), pve-i18n:amd64 (3.1.5, 3.2.0), proxmox-backup-client:amd64 (3.1.2-1, 3.1.3-1), pve-manager:amd64 (8.1.3, 8.1.4), libc-l10n:amd64 (2.36-9+deb12u3, 2.36-9+deb12u4), libc-bin:amd64 (2.36-9+deb12u3, 2.36-9+deb12u4), pve-edk2-firmware:amd64 (4.2023.08-2, 4.2023.08-3)
Error: Sub-process /usr/bin/dpkg returned an error code (1)
End-Date: 2024-02-02  10:53:48

Start-Date: 2024-02-02  10:55:48
Commandline: apt-get dist-upgrade
Error: Sub-process /usr/bin/dpkg returned an error code (1)
End-Date: 2024-02-02  11:26:12

Start-Date: 2024-02-02  11:27:40
Commandline: apt autoremove
Remove: proxmox-kernel-6.2.16-19-pve:amd64 (6.2.16-19), proxmox-kernel-6.5.11-6-pve-signed:amd64 (6.5.11-6)
Error: Sub-process /usr/bin/dpkg returned an error code (1)
End-Date: 2024-02-02  11:30:59

Start-Date: 2024-02-02  11:36:58
Commandline: apt install pve-manager pve-manager=8.1.3
Downgrade: pve-manager:amd64 (8.1.4, 8.1.3)

Start-Date: 2024-02-02  11:46:45
Commandline: apt install pve-manager=8.1.4
Upgrade: pve-manager:amd64 (8.1.3, 8.1.4)

Start-Date: 2024-02-02  12:22:22
Commandline: apt dist-upgrade

I tried to apt remove pve-manager, but it prompted to also remove proxmox-ve, which I wasn't comfortable with, without further guidance
 
Last edited:
I've run into this issue as well whilst trying to upgrade my Proxmox instance to the latest package versions. It appears to be hanging on deb-system-invoke reload-or-restart pvescheduler.service that is run as part of the pve-manager package post-install script. I have tried running dpkg --configure -a manually and this hangs in the same place as well.
 
Yes exactly, it seems pve-manager just doesn't want to "install" and it hangs on something. Let's hope we can get some support :)
 
I fixed my upgrade issue by downgrading all of the upgraded packages per the apt history, rebooting, then re-running apt dist-upgrade. Here are all of the steps that I followed:
  1. Downgrade all of the upgraded packages to the previously installed version using apt install <package>=<version> (the old versions on my system are listed below). Kill the stuck systemctl reload-or-restart pvescheduler.service process to allow the downgrade to finish
  2. Bulk stopped all running VMs and CTs, then ran systemctl mask pve-guests.service to stop guests from starting at boot
  3. Restart the Proxmox system
  4. Complete the pve-manager post-install script via dpkg --configure -a
  5. Re-run apt dist-upgrade
  6. Re-enabled guests to start at boot: systemctl unmask pve-guests.service then reboot again
I initially tried to downgrade specific packages, but none of the downgrades enabled the pve-manager post-install script to complete. So I'm still none the wiser about which package caused the pvescheduler systemd unit to hang on restart, but at least I have a working and upgraded system.

Code: 
Start-Date: 2024-02-04  09:55:04
Commandline: apt-get dist-upgrade
Upgrade: pve-edk2-firmware-ovmf:amd64 (4.2023.08-2, 4.2023.08-3), pve-qemu-kvm:amd64 (8.1.2-4, 8.1.2-6), pve-edk2-firmware-legacy:amd64 (4.2023.08-2, 4.2023.08-3), libproxmox-rs-perl:amd64 (0.3.1, 0.3.3), libc6:amd64 (2.36-9+deb12u3, 2.36-9+deb12u4), locales:amd64 (2.36-9+deb12u3, 2.36-9+deb12u4), ifupdown2:amd64 (3.2.0-1+pmx7, 3.2.0-1+pmx8), pve-xtermjs:amd64 (5.3.0-2, 5.3.0-3), pve-i18n:amd64 (3.1.4, 3.1.5), pve-manager:amd64 (8.1.3, 8.1.4), libc-l10n:amd64 (2.36-9+deb12u3, 2.36-9+deb12u4), libc-bin:amd64 (2.36-9+deb12u3, 2.36-9+deb12u4), pve-edk2-firmware:amd64 (4.2023.08-2, 4.2023.08-3)
 
Fun fact- I was about to do as you described.

I got the error

Code: 
root@proxmox:~# apt install pve-edk2-firmware-ovmf:amd64=4.2023.08-2
E: dpkg was interrupted, you must manually run 'dpkg --configure -a' to correct the problem.

Since something was still wrong from last time

So I ran dpkg --configure -a expecting the system to get stuck again. But it just went trough immediately...

Looking at the installed versions, it no longer shows up as pve-manager: not correctly installed (running version: 8.1.4/ec5affc9e41f1d79)

Code: 
root@proxmox:~# pveversion -v
proxmox-ve: 8.1.0 (running kernel: 6.5.11-8-pve)
pve-manager: 8.1.4 (running version: 8.1.4/ec5affc9e41f1d79)
proxmox-kernel-helper: 8.1.0
pve-kernel-6.2: 8.0.5
proxmox-kernel-6.5: 6.5.11-8
proxmox-kernel-6.5.11-8-pve-signed: 6.5.11-8
proxmox-kernel-6.5.11-7-pve-signed: 6.5.11-7
proxmox-kernel-6.2.16-20-pve: 6.2.16-20
proxmox-kernel-6.2: 6.2.16-20
pve-kernel-6.2.16-3-pve: 6.2.16-3
ceph-fuse: 17.2.6-pve1+3
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx8
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.0
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.3
libpve-access-control: 8.0.7
libpve-apiclient-perl: 3.3.1
libpve-common-perl: 8.1.0
libpve-guest-common-perl: 5.0.6
libpve-http-server-perl: 5.0.5
libpve-network-perl: 0.9.5
libpve-rs-perl: 0.8.8
libpve-storage-perl: 8.0.5
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 5.0.2-4
lxcfs: 5.0.3-pve4
novnc-pve: 1.4.0-3
proxmox-backup-client: 3.1.3-1
proxmox-backup-file-restore: 3.1.3-1
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.2.3
proxmox-mini-journalreader: 1.4.0
proxmox-widget-toolkit: 4.1.3
pve-cluster: 8.0.5
pve-container: 5.0.8
pve-docs: 8.1.3
pve-edk2-firmware: 4.2023.08-3
pve-firewall: 5.0.3
pve-firmware: 3.9-1
pve-ha-manager: 4.0.3
pve-i18n: 3.2.0
pve-qemu-kvm: 8.1.2-6
pve-xtermjs: 5.3.0-3
qemu-server: 8.0.10
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.2-pve1

It seems for me that "waiting" was what I needed to do. The only thing I did in parallel was upgrade my proxmox backup server too (running on different hardware), but I doubt that is related.

So it seems this is resloved. Your solution is probably what is required, but for me trying again 24-48h later was the solution :)

Thanks!
 
crowbarz said:
I fixed my upgrade issue by downgrading all of the upgraded packages per the apt history, rebooting, then re-running apt dist-upgrade. Here are all of the steps that I followed:
  1. Downgrade all of the upgraded packages to the previously installed version using apt install <package>=<version> (the old versions on my system are listed below). Kill the stuck systemctl reload-or-restart pvescheduler.service process to allow the downgrade to finish
  2. Bulk stopped all running VMs and CTs, then ran systemctl mask pve-guests.service to stop guests from starting at boot
  3. Restart the Proxmox system
  4. Complete the pve-manager post-install script via dpkg --configure -a
  5. Re-run apt dist-upgrade
  6. Re-enabled guests to start at boot: systemctl unmask pve-guests.service then reboot again
I initially tried to downgrade specific packages, but none of the downgrades enabled the pve-manager post-install script to complete. So I'm still none the wiser about which package caused the pvescheduler systemd unit to hang on restart, but at least I have a working and upgraded system.

Code: 
Start-Date: 2024-02-04  09:55:04
Commandline: apt-get dist-upgrade
Upgrade: pve-edk2-firmware-ovmf:amd64 (4.2023.08-2, 4.2023.08-3), pve-qemu-kvm:amd64 (8.1.2-4, 8.1.2-6), pve-edk2-firmware-legacy:amd64 (4.2023.08-2, 4.2023.08-3), libproxmox-rs-perl:amd64 (0.3.1, 0.3.3), libc6:amd64 (2.36-9+deb12u3, 2.36-9+deb12u4), locales:amd64 (2.36-9+deb12u3, 2.36-9+deb12u4), ifupdown2:amd64 (3.2.0-1+pmx7, 3.2.0-1+pmx8), pve-xtermjs:amd64 (5.3.0-2, 5.3.0-3), pve-i18n:amd64 (3.1.4, 3.1.5), pve-manager:amd64 (8.1.3, 8.1.4), libc-l10n:amd64 (2.36-9+deb12u3, 2.36-9+deb12u4), libc-bin:amd64 (2.36-9+deb12u3, 2.36-9+deb12u4), pve-edk2-firmware:amd64 (4.2023.08-2, 4.2023.08-3)
Click to expand...
I followed these instructions and got my system to upgrade the packages. However, pve-scheduler.service becomes dead shortly after a reboot. Upon attempting to install pve-manager again with --reinstall... it halts again...

I think something is seriously wrong with a package. I hope the proxmox devs get to looking at this soon as this is really bad.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back