Upgrade from 6.4 to 7.4 in-place, apt full-upgrade hangs on "Setting up pve-manager"

Also, what's the output of systemctl list-jobs while you have the hanging restart?

AFAICT from reading some more bug reports, the systemd-tty-ask-password-agent --watch is not necessarily the culprit here as it is spawned together with the systemctl but not blocking it https://github.com/systemd/systemd/issues/9507

So let's shift our focus on why the pvescheduler.service restart might fail. What's the output of systemctl status pvescheduler.service while the upgrade hangs? Please also attach the journal since boot, journalctl -b > journal.txt
 
Code: 
root@pve:~# systemd --version
systemd 247 (247.3-7+1-pmx11u1)
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +ZSTD +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=unified

I ended up mv:ing the postinst file and the dpkg --configure -a command worked. The issue seemed to be the restarting/reloading of services. I can always revert the script and rerun it, it seems atomic enough in terms of what is being done, and looking at the affected files (lvm.conf etc) the changes seem to actually have been made, so I don't think I actually missed any of the postinstall config steps.. But of course it would feel better if the script ran without error.

Can I / should I upgrade systemd somehow? EDIT: Checked https://packages.debian.org/bullseye/systemd and it seems i have the latest version
 
Last edited:
Chris said:
Also, what's the output of systemctl list-jobs while you have the hanging restart?

AFAICT from reading some more bug reports, the systemd-tty-ask-password-agent --watch is not necessarily the culprit here as it is spawned together with the systemctl but not blocking it https://github.com/systemd/systemd/issues/9507

So let's shift our focus on why the pvescheduler.service restart might fail. What's the output of systemctl status pvescheduler.service while the upgrade hangs? Please also attach the journal since boot, journalctl -b > journal.txt
Click to expand...
I checked pvescheduler yesterday and there were no apparent issues with it. Since then I rebooted, so the journal/systemctl is truncated. :(

EDIT: I moved the postinst script back and ran dpkg-reconfigure pve-manager but no issues this time..

pveversion -v since yesterday:

Bash: 
root@pve:~# pveversion -v
proxmox-ve: 7.4-1 (running kernel: 5.15.107-2-pve)
pve-manager: 7.4-3 (running version: 7.4-3/9002ab8a)
pve-kernel-5.15: 7.4-3
pve-kernel-5.4: 6.4-20
pve-kernel-5.0: 6.0-11
pve-kernel-5.15.107-2-pve: 5.15.107-2
pve-kernel-5.4.203-1-pve: 5.4.203-1
pve-kernel-5.4.195-1-pve: 5.4.195-1
pve-kernel-5.0.21-5-pve: 5.0.21-10
pve-kernel-5.0.15-1-pve: 5.0.15-1
ceph-fuse: 14.2.21-1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx4
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4-3
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.4-1
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-3
libpve-rs-perl: 0.7.6
libpve-storage-perl: 7.4-2
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
proxmox-backup-client: 2.4.2-1
proxmox-backup-file-restore: 2.4.2-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.1-1
proxmox-widget-toolkit: 3.7.0
pve-cluster: 7.3-3
pve-container: 4.4-3
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-2
pve-firewall: 4.3-2
pve-firmware: 3.6-5
pve-ha-manager: 3.6.1
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-1
qemu-server: 7.4-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.11-pve1
 
Last edited:
39grader said:
I checked pvescheduler yesterday and there were no apparent issues with it. Since then I rebooted, so the journal/systemctl is truncated. :(

EDIT: I moved the postinst script back and ran dpkg-reconfigure pve-manager but no issues this time..

pveversion -v since yesterday:

Bash: 
root@pve:~# pveversion -v
proxmox-ve: 7.4-1 (running kernel: 5.15.107-2-pve)
pve-manager: 7.4-3 (running version: 7.4-3/9002ab8a)
pve-kernel-5.15: 7.4-3
pve-kernel-5.4: 6.4-20
pve-kernel-5.0: 6.0-11
pve-kernel-5.15.107-2-pve: 5.15.107-2
pve-kernel-5.4.203-1-pve: 5.4.203-1
pve-kernel-5.4.195-1-pve: 5.4.195-1
pve-kernel-5.0.21-5-pve: 5.0.21-10
pve-kernel-5.0.15-1-pve: 5.0.15-1
ceph-fuse: 14.2.21-1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx4
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4-3
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.4-1
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-3
libpve-rs-perl: 0.7.6
libpve-storage-perl: 7.4-2
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
proxmox-backup-client: 2.4.2-1
proxmox-backup-file-restore: 2.4.2-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.1-1
proxmox-widget-toolkit: 3.7.0
pve-cluster: 7.3-3
pve-container: 4.4-3
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-2
pve-firewall: 4.3-2
pve-firmware: 3.6-5
pve-ha-manager: 3.6.1
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-1
qemu-server: 7.4-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.11-pve1
Click to expand...


EDIT:

This is status of pvescheduler after reconfigure;

Code: 
root@pve:~# systemctl status pvescheduler.service
● pvescheduler.service - Proxmox VE scheduler
     Loaded: loaded (/lib/systemd/system/pvescheduler.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-05-31 23:30:13 CEST; 12h ago
    Process: 155914 ExecReload=/usr/bin/pvescheduler restart (code=exited, status=0/SUCCESS)
   Main PID: 7855 (pvescheduler)
      Tasks: 1 (limit: 270659)
     Memory: 101.2M
        CPU: 13.795s
     CGroup: /system.slice/pvescheduler.service
             └─7855 pvescheduler

May 31 23:30:12 pve systemd[1]: Starting Proxmox VE scheduler...
May 31 23:30:13 pve pvescheduler[7855]: starting server
May 31 23:30:13 pve systemd[1]: Started Proxmox VE scheduler.
Jun 01 12:11:30 pve systemd[1]: Reloading Proxmox VE scheduler.
Jun 01 12:11:31 pve pvescheduler[155914]: send HUP to 7855
Jun 01 12:11:31 pve pvescheduler[7855]: received signal HUP
Jun 01 12:11:31 pve pvescheduler[7855]: server shutdown (restart)
Jun 01 12:11:31 pve systemd[1]: Reloaded Proxmox VE schedule

Looks like the reconfigure correctly reloaded the service. I would strace dpkg-reconfigure pvescheduler if i had the time and energy, but it looks like we're back to a working state (yay!)
 
You must log in or register to reply here.
Top Bottom