[SOLVED] Major issues after upgrading to 7.2

poxin

Well-Known Member
Jun 27, 2017
70
6
48
Was on PVE 7.1 prior to update via web gui. The standard update and dist-upgrade command ran and I rebooted the server.

Noticed upon boot-up that I can SSH to the machine, but the web interface does not work. Seems a lot of processes will not start.

Code:
root@pve:~# pveversion -v
proxmox-ve: not correctly installed (running kernel: 5.13.19-4-pve)
pve-manager: not correctly installed (running version: 7.2-3/c743d6c1)
pve-kernel-5.11: 7.0-10
pve-kernel-5.4: 6.4-5
pve-kernel-5.13.19-4-pve: 5.13.19-9
pve-kernel-5.13.19-3-pve: 5.13.19-7
pve-kernel-5.13.19-1-pve: 5.13.19-3
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.4.128-1-pve: 5.4.128-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 14.2.21-1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: not correctly installed
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: not correctly installed
libpve-apiclient-perl: 3.2-1
libpve-common-perl: not correctly installed
libpve-guest-common-perl: not correctly installed
libpve-http-server-perl: 4.1-1
libpve-storage-perl: not correctly installed
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: not correctly installed
lxcfs: not correctly installed
novnc-pve: not correctly installed
proxmox-backup-client: not correctly installed
proxmox-backup-file-restore: not correctly installed
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: not correctly installed
pve-cluster: not correctly installed
pve-container: not correctly installed
pve-docs: not correctly installed
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: not correctly installed
pve-ha-manager: not correctly installed
pve-i18n: not correctly installed
pve-qemu-kvm: not correctly installed
pve-xtermjs: 4.16.0-1
qemu-server: not correctly installed
smartmontools: not correctly installed
spiceterm: 3.2-2
swtpm: not correctly installed
vncterm: 1.7-1
zfsutils-linux: not correctly installed
 
Code:
root@pve:~# systemctl status pve*
● pve-ha-crm.service - PVE Cluster HA Resource Manager Daemon
     Loaded: loaded (/lib/systemd/system/pve-ha-crm.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:06 EDT; 5min ago
    Process: 4948 ExecStart=/usr/sbin/pve-ha-crm start (code=exited, status=2)
        CPU: 317ms

May 05 17:09:06 pve pve-ha-crm[4948]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/HA/Resources/PVEVM.pm line 19, <DATA> line 960.
May 05 17:09:06 pve pve-ha-crm[4948]: Compilation failed in require at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 21, <DATA> line 960.
May 05 17:09:06 pve pve-ha-crm[4948]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 21, <DATA> line 960.
May 05 17:09:06 pve pve-ha-crm[4948]: Compilation failed in require at /usr/share/perl5/PVE/Service/pve_ha_crm.pm line 9, <DATA> line 960.
May 05 17:09:06 pve pve-ha-crm[4948]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/pve_ha_crm.pm line 9, <DATA> line 960.
May 05 17:09:06 pve pve-ha-crm[4948]: Compilation failed in require at /usr/sbin/pve-ha-crm line 6, <DATA> line 960.
May 05 17:09:06 pve pve-ha-crm[4948]: BEGIN failed--compilation aborted at /usr/sbin/pve-ha-crm line 6, <DATA> line 960.
May 05 17:09:06 pve systemd[1]: pve-ha-crm.service: Control process exited, code=exited, status=2/INVALIDARGUMENT
May 05 17:09:06 pve systemd[1]: pve-ha-crm.service: Failed with result 'exit-code'.
May 05 17:09:06 pve systemd[1]: Failed to start PVE Cluster HA Resource Manager Daemon.

● pve-daily-update.timer - Daily PVE download activities
     Loaded: loaded (/lib/systemd/system/pve-daily-update.timer; enabled; vendor preset: enabled)
     Active: active (waiting) since Thu 2022-05-05 17:09:03 EDT; 5min ago
    Trigger: Fri 2022-05-06 04:31:48 EDT; 11h left
   Triggers: ● pve-daily-update.service

May 05 17:09:03 pve systemd[1]: Started Daily PVE download activities.

● pve-guests.service - PVE guests
     Loaded: loaded (/lib/systemd/system/pve-guests.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:09 EDT; 5min ago
    Process: 5715 ExecStartPre=/usr/share/pve-manager/helpers/pve-startall-delay (code=exited, status=0/SUCCESS)
    Process: 5717 ExecStart=/usr/bin/pvesh --nooutput create /nodes/localhost/startall (code=exited, status=255/EXCEPTION)
   Main PID: 5717 (code=exited, status=255/EXCEPTION)
        CPU: 483ms

May 05 17:09:09 pve pvesh[5717]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2/Cluster.pm line 14, <DATA> line 960.
May 05 17:09:09 pve pvesh[5717]: Compilation failed in require at /usr/share/perl5/PVE/API2.pm line 14, <DATA> line 960.
May 05 17:09:09 pve pvesh[5717]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2.pm line 14, <DATA> line 960.
May 05 17:09:09 pve pvesh[5717]: Compilation failed in require at /usr/share/perl5/PVE/CLI/pvesh.pm line 16, <DATA> line 960.
May 05 17:09:09 pve pvesh[5717]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/CLI/pvesh.pm line 16, <DATA> line 960.
May 05 17:09:09 pve pvesh[5717]: Compilation failed in require at /usr/bin/pvesh line 6, <DATA> line 960.
May 05 17:09:09 pve pvesh[5717]: BEGIN failed--compilation aborted at /usr/bin/pvesh line 6, <DATA> line 960.
May 05 17:09:09 pve systemd[1]: pve-guests.service: Main process exited, code=exited, status=255/EXCEPTION
May 05 17:09:09 pve systemd[1]: pve-guests.service: Failed with result 'exit-code'.
May 05 17:09:09 pve systemd[1]: Failed to start PVE guests.

● pvestatd.service - PVE Status Daemon
     Loaded: loaded (/lib/systemd/system/pvestatd.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:05 EDT; 5min ago
    Process: 4881 ExecStart=/usr/bin/pvestatd start (code=exited, status=2)
        CPU: 336ms

May 05 17:09:05 pve pvestatd[4881]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/QemuConfig.pm line 6, <DATA> line 960.
May 05 17:09:05 pve pvestatd[4881]: Compilation failed in require at /usr/share/perl5/PVE/QemuServer.pm line 46, <DATA> line 960.
May 05 17:09:05 pve pvestatd[4881]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/QemuServer.pm line 46, <DATA> line 960.
May 05 17:09:05 pve pvestatd[4881]: Compilation failed in require at /usr/share/perl5/PVE/Service/pvestatd.pm line 20, <DATA> line 960.
May 05 17:09:05 pve pvestatd[4881]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/pvestatd.pm line 20, <DATA> line 960.
May 05 17:09:05 pve pvestatd[4881]: Compilation failed in require at /usr/bin/pvestatd line 9, <DATA> line 960.
May 05 17:09:05 pve pvestatd[4881]: BEGIN failed--compilation aborted at /usr/bin/pvestatd line 9, <DATA> line 960.
May 05 17:09:05 pve systemd[1]: pvestatd.service: Control process exited, code=exited, status=2/INVALIDARGUMENT
May 05 17:09:05 pve systemd[1]: pvestatd.service: Failed with result 'exit-code'.
May 05 17:09:05 pve systemd[1]: Failed to start PVE Status Daemon.

● pve-storage.target - PVE Storage Target
     Loaded: loaded (/lib/systemd/system/pve-storage.target; static)
     Active: active since Thu 2022-05-05 17:09:04 EDT; 5min ago

May 05 17:09:04 pve systemd[1]: Reached target PVE Storage Target.

● pveproxy.service - PVE API Proxy Server
     Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:10 EDT; 5min ago
    Process: 5716 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
    Process: 5719 ExecStart=/usr/bin/pveproxy start (code=exited, status=255/EXCEPTION)
        CPU: 545ms

May 05 17:09:10 pve systemd[1]: pveproxy.service: Scheduled restart job, restart counter is at 5.
May 05 17:09:10 pve systemd[1]: Stopped PVE API Proxy Server.
May 05 17:09:10 pve systemd[1]: pveproxy.service: Start request repeated too quickly.
May 05 17:09:10 pve systemd[1]: pveproxy.service: Failed with result 'exit-code'.
May 05 17:09:10 pve systemd[1]: Failed to start PVE API Proxy Server.

● pvebanner.service - Proxmox VE Login Banner
     Loaded: loaded (/lib/systemd/system/pvebanner.service; enabled; vendor preset: enabled)
     Active: active (exited) since Thu 2022-05-05 17:09:03 EDT; 5min ago
    Process: 3247 ExecStart=/usr/bin/pvebanner (code=exited, status=0/SUCCESS)
   Main PID: 3247 (code=exited, status=0/SUCCESS)
        CPU: 223ms

May 05 17:09:03 pve systemd[1]: Starting Proxmox VE Login Banner...
May 05 17:09:03 pve systemd[1]: Finished Proxmox VE Login Banner.

● pvescheduler.service - Proxmox VE scheduler
     Loaded: loaded (/lib/systemd/system/pvescheduler.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:10 EDT; 5min ago
    Process: 5720 ExecStart=/usr/bin/pvescheduler start (code=exited, status=2)
        CPU: 295ms

May 05 17:09:10 pve pvescheduler[5720]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Jobs/VZDump.pm line 8, <DATA> line 960.
May 05 17:09:10 pve pvescheduler[5720]: Compilation failed in require at /usr/share/perl5/PVE/Jobs.pm line 9, <DATA> line 960.
May 05 17:09:10 pve pvescheduler[5720]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Jobs.pm line 9, <DATA> line 960.
May 05 17:09:10 pve pvescheduler[5720]: Compilation failed in require at /usr/share/perl5/PVE/Service/pvescheduler.pm line 8, <DATA> line 960.
May 05 17:09:10 pve pvescheduler[5720]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/pvescheduler.pm line 8, <DATA> line 960.
May 05 17:09:10 pve pvescheduler[5720]: Compilation failed in require at /usr/bin/pvescheduler line 6, <DATA> line 960.
May 05 17:09:10 pve pvescheduler[5720]: BEGIN failed--compilation aborted at /usr/bin/pvescheduler line 6, <DATA> line 960.
May 05 17:09:10 pve systemd[1]: pvescheduler.service: Control process exited, code=exited, status=2/INVALIDARGUMENT
May 05 17:09:10 pve systemd[1]: pvescheduler.service: Failed with result 'exit-code'.
May 05 17:09:10 pve systemd[1]: Failed to start Proxmox VE scheduler.

● pve-lxc-syscalld.service - Proxmox VE LXC Syscall Daemon
     Loaded: loaded (/lib/systemd/system/pve-lxc-syscalld.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-05-05 17:09:03 EDT; 5min ago
   Main PID: 3364 (pve-lxc-syscall)
      Tasks: 13 (limit: 76791)
     Memory: 1.2M
        CPU: 7ms
     CGroup: /system.slice/pve-lxc-syscalld.service
             └─3364 /usr/lib/x86_64-linux-gnu/pve-lxc-syscalld/pve-lxc-syscalld --system /run/pve/lxc-syscalld.sock

May 05 17:09:03 pve systemd[1]: Starting Proxmox VE LXC Syscall Daemon...
May 05 17:09:03 pve systemd[1]: Started Proxmox VE LXC Syscall Daemon.

● pve-cluster.service - The Proxmox VE cluster filesystem
     Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-05-05 17:09:05 EDT; 5min ago
    Process: 3848 ExecStart=/usr/bin/pmxcfs (code=exited, status=0/SUCCESS)
   Main PID: 3890 (pmxcfs)
      Tasks: 5 (limit: 76791)
     Memory: 10.5M
        CPU: 76ms
     CGroup: /system.slice/pve-cluster.service
             └─3890 /usr/bin/pmxcfs

May 05 17:09:04 pve systemd[1]: Starting The Proxmox VE cluster filesystem...
May 05 17:09:05 pve systemd[1]: Started The Proxmox VE cluster filesystem.

● pve-firewall.service - Proxmox VE firewall
     Loaded: loaded (/lib/systemd/system/pve-firewall.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-05-05 17:09:05 EDT; 5min ago
    Process: 4879 ExecStartPre=/usr/bin/update-alternatives --set ebtables /usr/sbin/ebtables-legacy (code=exited, status=0/SUCCESS)
    Process: 4883 ExecStartPre=/usr/bin/update-alternatives --set iptables /usr/sbin/iptables-legacy (code=exited, status=0/SUCCESS)
    Process: 4884 ExecStartPre=/usr/bin/update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy (code=exited, status=0/SUCCESS)
    Process: 4887 ExecStart=/usr/sbin/pve-firewall start (code=exited, status=0/SUCCESS)
   Main PID: 4932 (pve-firewall)
      Tasks: 1 (limit: 76791)
     Memory: 72.0M
        CPU: 1.387s
     CGroup: /system.slice/pve-firewall.service
             └─4932 pve-firewall

May 05 17:09:05 pve systemd[1]: Starting Proxmox VE firewall...
May 05 17:09:05 pve pve-firewall[4932]: starting server
May 05 17:09:05 pve systemd[1]: Started Proxmox VE firewall.

● pvenetcommit.service - Commit Proxmox VE network changes
     Loaded: loaded (/lib/systemd/system/pvenetcommit.service; enabled; vendor preset: enabled)
     Active: active (exited) since Thu 2022-05-05 17:09:03 EDT; 5min ago
    Process: 3252 ExecStartPre=/bin/rm -f /etc/openvswitch/conf.db (code=exited, status=0/SUCCESS)
    Process: 3320 ExecStart=sh -c if [ -f ${FN}.new ]; then mv ${FN}.new ${FN}; fi (code=exited, status=0/SUCCESS)
   Main PID: 3320 (code=exited, status=0/SUCCESS)
        CPU: 2ms

May 05 17:09:03 pve systemd[1]: Starting Commit Proxmox VE network changes...
May 05 17:09:03 pve systemd[1]: Finished Commit Proxmox VE network changes.

● pvedaemon.service - PVE API Daemon
     Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:08 EDT; 5min ago
    Process: 5625 ExecStart=/usr/bin/pvedaemon start (code=exited, status=255/EXCEPTION)
        CPU: 339ms

May 05 17:09:08 pve systemd[1]: Failed to start PVE API Daemon.
May 05 17:09:08 pve systemd[1]: pvedaemon.service: Start request repeated too quickly.
May 05 17:09:08 pve systemd[1]: pvedaemon.service: Failed with result 'exit-code'.
May 05 17:09:08 pve systemd[1]: Failed to start PVE API Daemon.
May 05 17:09:09 pve systemd[1]: pvedaemon.service: Start request repeated too quickly.
May 05 17:09:09 pve systemd[1]: pvedaemon.service: Failed with result 'exit-code'.
May 05 17:09:09 pve systemd[1]: Failed to start PVE API Daemon.
May 05 17:09:10 pve systemd[1]: pvedaemon.service: Start request repeated too quickly.
May 05 17:09:10 pve systemd[1]: pvedaemon.service: Failed with result 'exit-code'.
May 05 17:09:10 pve systemd[1]: Failed to start PVE API Daemon.

● pvefw-logger.service - Proxmox VE firewall logger
     Loaded: loaded (/lib/systemd/system/pvefw-logger.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-05-05 17:09:03 EDT; 5min ago
    Process: 3250 ExecStart=/usr/sbin/pvefw-logger (code=exited, status=0/SUCCESS)
   Main PID: 3299 (pvefw-logger)
      Tasks: 2 (limit: 76791)
     Memory: 820.0K
        CPU: 25ms
     CGroup: /system.slice/pvefw-logger.service
             └─3299 /usr/sbin/pvefw-logger

May 05 17:09:03 pve systemd[1]: Starting Proxmox VE firewall logger...
May 05 17:09:03 pve pvefw-logger[3299]: starting pvefw logger
May 05 17:09:03 pve systemd[1]: Started Proxmox VE firewall logger.

● pve-ha-lrm.service - PVE Local HA Resource Manager Daemon
     Loaded: loaded (/lib/systemd/system/pve-ha-lrm.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:08 EDT; 5min ago
    Process: 5629 ExecStart=/usr/sbin/pve-ha-lrm start (code=exited, status=2)
        CPU: 315ms

May 05 17:09:08 pve pve-ha-lrm[5629]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/HA/Resources/PVEVM.pm line 19, <DATA> line 960.
May 05 17:09:08 pve pve-ha-lrm[5629]: Compilation failed in require at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 21, <DATA> line 960.
May 05 17:09:08 pve pve-ha-lrm[5629]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 21, <DATA> line 960.
May 05 17:09:08 pve pve-ha-lrm[5629]: Compilation failed in require at /usr/share/perl5/PVE/Service/pve_ha_lrm.pm line 9, <DATA> line 960.
May 05 17:09:08 pve pve-ha-lrm[5629]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/pve_ha_lrm.pm line 9, <DATA> line 960.
May 05 17:09:08 pve pve-ha-lrm[5629]: Compilation failed in require at /usr/sbin/pve-ha-lrm line 6, <DATA> line 960.
May 05 17:09:08 pve pve-ha-lrm[5629]: BEGIN failed--compilation aborted at /usr/sbin/pve-ha-lrm line 6, <DATA> line 960.
May 05 17:09:08 pve systemd[1]: pve-ha-lrm.service: Control process exited, code=exited, status=2/INVALIDARGUMENT
May 05 17:09:08 pve systemd[1]: pve-ha-lrm.service: Failed with result 'exit-code'.
May 05 17:09:08 pve systemd[1]: Failed to start PVE Local HA Resource Manager Daemon.
 
Resolved:

Code:
apt-get dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 libpve-common-perl : Depends: libproxmox-rs-perl but it is not installed
 pve-manager : Depends: libproxmox-rs-perl but it is not installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

root@pve:~# apt install libproxmox-rs-perl

root@pve:~# apt-get dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  pve-kernel-5.11 pve-kernel-5.11.22-7-pve pve-kernel-5.13.19-3-pve
Use 'apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
 
Thanks, this was very helpful. On several of my servers that I regularly update, I had no issues. On another server I try to only update once every several months, this issue occurred. My guess there was a small upgrade before the 7.2 release that applied this Perl package, and it was mistakenly left out as a dependency from the 7.2 release.

They did add in some new perl-to-rust features in 7.2, so this does seem to be a 7.2 related upgrade bug.

Again, thanks for sharing how you resolved it.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!