[SOLVED] Major issues after upgrading to 7.2

P

poxin

Well-Known Member
Jun 27, 2017
73
6
48
Was on PVE 7.1 prior to update via web gui. The standard update and dist-upgrade command ran and I rebooted the server.

Noticed upon boot-up that I can SSH to the machine, but the web interface does not work. Seems a lot of processes will not start.

Code: 
root@pve:~# pveversion -v
proxmox-ve: not correctly installed (running kernel: 5.13.19-4-pve)
pve-manager: not correctly installed (running version: 7.2-3/c743d6c1)
pve-kernel-5.11: 7.0-10
pve-kernel-5.4: 6.4-5
pve-kernel-5.13.19-4-pve: 5.13.19-9
pve-kernel-5.13.19-3-pve: 5.13.19-7
pve-kernel-5.13.19-1-pve: 5.13.19-3
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.4.128-1-pve: 5.4.128-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 14.2.21-1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: not correctly installed
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: not correctly installed
libpve-apiclient-perl: 3.2-1
libpve-common-perl: not correctly installed
libpve-guest-common-perl: not correctly installed
libpve-http-server-perl: 4.1-1
libpve-storage-perl: not correctly installed
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: not correctly installed
lxcfs: not correctly installed
novnc-pve: not correctly installed
proxmox-backup-client: not correctly installed
proxmox-backup-file-restore: not correctly installed
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: not correctly installed
pve-cluster: not correctly installed
pve-container: not correctly installed
pve-docs: not correctly installed
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: not correctly installed
pve-ha-manager: not correctly installed
pve-i18n: not correctly installed
pve-qemu-kvm: not correctly installed
pve-xtermjs: 4.16.0-1
qemu-server: not correctly installed
smartmontools: not correctly installed
spiceterm: 3.2-2
swtpm: not correctly installed
vncterm: 1.7-1
zfsutils-linux: not correctly installed
 
Code: 
root@pve:~# systemctl status pve*
● pve-ha-crm.service - PVE Cluster HA Resource Manager Daemon
     Loaded: loaded (/lib/systemd/system/pve-ha-crm.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:06 EDT; 5min ago
    Process: 4948 ExecStart=/usr/sbin/pve-ha-crm start (code=exited, status=2)
        CPU: 317ms

May 05 17:09:06 pve pve-ha-crm[4948]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/HA/Resources/PVEVM.pm line 19, <DATA> line 960.
May 05 17:09:06 pve pve-ha-crm[4948]: Compilation failed in require at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 21, <DATA> line 960.
May 05 17:09:06 pve pve-ha-crm[4948]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 21, <DATA> line 960.
May 05 17:09:06 pve pve-ha-crm[4948]: Compilation failed in require at /usr/share/perl5/PVE/Service/pve_ha_crm.pm line 9, <DATA> line 960.
May 05 17:09:06 pve pve-ha-crm[4948]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/pve_ha_crm.pm line 9, <DATA> line 960.
May 05 17:09:06 pve pve-ha-crm[4948]: Compilation failed in require at /usr/sbin/pve-ha-crm line 6, <DATA> line 960.
May 05 17:09:06 pve pve-ha-crm[4948]: BEGIN failed--compilation aborted at /usr/sbin/pve-ha-crm line 6, <DATA> line 960.
May 05 17:09:06 pve systemd[1]: pve-ha-crm.service: Control process exited, code=exited, status=2/INVALIDARGUMENT
May 05 17:09:06 pve systemd[1]: pve-ha-crm.service: Failed with result 'exit-code'.
May 05 17:09:06 pve systemd[1]: Failed to start PVE Cluster HA Resource Manager Daemon.

● pve-daily-update.timer - Daily PVE download activities
     Loaded: loaded (/lib/systemd/system/pve-daily-update.timer; enabled; vendor preset: enabled)
     Active: active (waiting) since Thu 2022-05-05 17:09:03 EDT; 5min ago
    Trigger: Fri 2022-05-06 04:31:48 EDT; 11h left
   Triggers: ● pve-daily-update.service

May 05 17:09:03 pve systemd[1]: Started Daily PVE download activities.

● pve-guests.service - PVE guests
     Loaded: loaded (/lib/systemd/system/pve-guests.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:09 EDT; 5min ago
    Process: 5715 ExecStartPre=/usr/share/pve-manager/helpers/pve-startall-delay (code=exited, status=0/SUCCESS)
    Process: 5717 ExecStart=/usr/bin/pvesh --nooutput create /nodes/localhost/startall (code=exited, status=255/EXCEPTION)
   Main PID: 5717 (code=exited, status=255/EXCEPTION)
        CPU: 483ms

May 05 17:09:09 pve pvesh[5717]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2/Cluster.pm line 14, <DATA> line 960.
May 05 17:09:09 pve pvesh[5717]: Compilation failed in require at /usr/share/perl5/PVE/API2.pm line 14, <DATA> line 960.
May 05 17:09:09 pve pvesh[5717]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2.pm line 14, <DATA> line 960.
May 05 17:09:09 pve pvesh[5717]: Compilation failed in require at /usr/share/perl5/PVE/CLI/pvesh.pm line 16, <DATA> line 960.
May 05 17:09:09 pve pvesh[5717]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/CLI/pvesh.pm line 16, <DATA> line 960.
May 05 17:09:09 pve pvesh[5717]: Compilation failed in require at /usr/bin/pvesh line 6, <DATA> line 960.
May 05 17:09:09 pve pvesh[5717]: BEGIN failed--compilation aborted at /usr/bin/pvesh line 6, <DATA> line 960.
May 05 17:09:09 pve systemd[1]: pve-guests.service: Main process exited, code=exited, status=255/EXCEPTION
May 05 17:09:09 pve systemd[1]: pve-guests.service: Failed with result 'exit-code'.
May 05 17:09:09 pve systemd[1]: Failed to start PVE guests.

● pvestatd.service - PVE Status Daemon
     Loaded: loaded (/lib/systemd/system/pvestatd.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:05 EDT; 5min ago
    Process: 4881 ExecStart=/usr/bin/pvestatd start (code=exited, status=2)
        CPU: 336ms

May 05 17:09:05 pve pvestatd[4881]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/QemuConfig.pm line 6, <DATA> line 960.
May 05 17:09:05 pve pvestatd[4881]: Compilation failed in require at /usr/share/perl5/PVE/QemuServer.pm line 46, <DATA> line 960.
May 05 17:09:05 pve pvestatd[4881]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/QemuServer.pm line 46, <DATA> line 960.
May 05 17:09:05 pve pvestatd[4881]: Compilation failed in require at /usr/share/perl5/PVE/Service/pvestatd.pm line 20, <DATA> line 960.
May 05 17:09:05 pve pvestatd[4881]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/pvestatd.pm line 20, <DATA> line 960.
May 05 17:09:05 pve pvestatd[4881]: Compilation failed in require at /usr/bin/pvestatd line 9, <DATA> line 960.
May 05 17:09:05 pve pvestatd[4881]: BEGIN failed--compilation aborted at /usr/bin/pvestatd line 9, <DATA> line 960.
May 05 17:09:05 pve systemd[1]: pvestatd.service: Control process exited, code=exited, status=2/INVALIDARGUMENT
May 05 17:09:05 pve systemd[1]: pvestatd.service: Failed with result 'exit-code'.
May 05 17:09:05 pve systemd[1]: Failed to start PVE Status Daemon.

● pve-storage.target - PVE Storage Target
     Loaded: loaded (/lib/systemd/system/pve-storage.target; static)
     Active: active since Thu 2022-05-05 17:09:04 EDT; 5min ago

May 05 17:09:04 pve systemd[1]: Reached target PVE Storage Target.

● pveproxy.service - PVE API Proxy Server
     Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:10 EDT; 5min ago
    Process: 5716 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
    Process: 5719 ExecStart=/usr/bin/pveproxy start (code=exited, status=255/EXCEPTION)
        CPU: 545ms

May 05 17:09:10 pve systemd[1]: pveproxy.service: Scheduled restart job, restart counter is at 5.
May 05 17:09:10 pve systemd[1]: Stopped PVE API Proxy Server.
May 05 17:09:10 pve systemd[1]: pveproxy.service: Start request repeated too quickly.
May 05 17:09:10 pve systemd[1]: pveproxy.service: Failed with result 'exit-code'.
May 05 17:09:10 pve systemd[1]: Failed to start PVE API Proxy Server.

● pvebanner.service - Proxmox VE Login Banner
     Loaded: loaded (/lib/systemd/system/pvebanner.service; enabled; vendor preset: enabled)
     Active: active (exited) since Thu 2022-05-05 17:09:03 EDT; 5min ago
    Process: 3247 ExecStart=/usr/bin/pvebanner (code=exited, status=0/SUCCESS)
   Main PID: 3247 (code=exited, status=0/SUCCESS)
        CPU: 223ms

May 05 17:09:03 pve systemd[1]: Starting Proxmox VE Login Banner...
May 05 17:09:03 pve systemd[1]: Finished Proxmox VE Login Banner.

● pvescheduler.service - Proxmox VE scheduler
     Loaded: loaded (/lib/systemd/system/pvescheduler.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:10 EDT; 5min ago
    Process: 5720 ExecStart=/usr/bin/pvescheduler start (code=exited, status=2)
        CPU: 295ms

May 05 17:09:10 pve pvescheduler[5720]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Jobs/VZDump.pm line 8, <DATA> line 960.
May 05 17:09:10 pve pvescheduler[5720]: Compilation failed in require at /usr/share/perl5/PVE/Jobs.pm line 9, <DATA> line 960.
May 05 17:09:10 pve pvescheduler[5720]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Jobs.pm line 9, <DATA> line 960.
May 05 17:09:10 pve pvescheduler[5720]: Compilation failed in require at /usr/share/perl5/PVE/Service/pvescheduler.pm line 8, <DATA> line 960.
May 05 17:09:10 pve pvescheduler[5720]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/pvescheduler.pm line 8, <DATA> line 960.
May 05 17:09:10 pve pvescheduler[5720]: Compilation failed in require at /usr/bin/pvescheduler line 6, <DATA> line 960.
May 05 17:09:10 pve pvescheduler[5720]: BEGIN failed--compilation aborted at /usr/bin/pvescheduler line 6, <DATA> line 960.
May 05 17:09:10 pve systemd[1]: pvescheduler.service: Control process exited, code=exited, status=2/INVALIDARGUMENT
May 05 17:09:10 pve systemd[1]: pvescheduler.service: Failed with result 'exit-code'.
May 05 17:09:10 pve systemd[1]: Failed to start Proxmox VE scheduler.

● pve-lxc-syscalld.service - Proxmox VE LXC Syscall Daemon
     Loaded: loaded (/lib/systemd/system/pve-lxc-syscalld.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-05-05 17:09:03 EDT; 5min ago
   Main PID: 3364 (pve-lxc-syscall)
      Tasks: 13 (limit: 76791)
     Memory: 1.2M
        CPU: 7ms
     CGroup: /system.slice/pve-lxc-syscalld.service
             └─3364 /usr/lib/x86_64-linux-gnu/pve-lxc-syscalld/pve-lxc-syscalld --system /run/pve/lxc-syscalld.sock

May 05 17:09:03 pve systemd[1]: Starting Proxmox VE LXC Syscall Daemon...
May 05 17:09:03 pve systemd[1]: Started Proxmox VE LXC Syscall Daemon.

● pve-cluster.service - The Proxmox VE cluster filesystem
     Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-05-05 17:09:05 EDT; 5min ago
    Process: 3848 ExecStart=/usr/bin/pmxcfs (code=exited, status=0/SUCCESS)
   Main PID: 3890 (pmxcfs)
      Tasks: 5 (limit: 76791)
     Memory: 10.5M
        CPU: 76ms
     CGroup: /system.slice/pve-cluster.service
             └─3890 /usr/bin/pmxcfs

May 05 17:09:04 pve systemd[1]: Starting The Proxmox VE cluster filesystem...
May 05 17:09:05 pve systemd[1]: Started The Proxmox VE cluster filesystem.

● pve-firewall.service - Proxmox VE firewall
     Loaded: loaded (/lib/systemd/system/pve-firewall.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-05-05 17:09:05 EDT; 5min ago
    Process: 4879 ExecStartPre=/usr/bin/update-alternatives --set ebtables /usr/sbin/ebtables-legacy (code=exited, status=0/SUCCESS)
    Process: 4883 ExecStartPre=/usr/bin/update-alternatives --set iptables /usr/sbin/iptables-legacy (code=exited, status=0/SUCCESS)
    Process: 4884 ExecStartPre=/usr/bin/update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy (code=exited, status=0/SUCCESS)
    Process: 4887 ExecStart=/usr/sbin/pve-firewall start (code=exited, status=0/SUCCESS)
   Main PID: 4932 (pve-firewall)
      Tasks: 1 (limit: 76791)
     Memory: 72.0M
        CPU: 1.387s
     CGroup: /system.slice/pve-firewall.service
             └─4932 pve-firewall

May 05 17:09:05 pve systemd[1]: Starting Proxmox VE firewall...
May 05 17:09:05 pve pve-firewall[4932]: starting server
May 05 17:09:05 pve systemd[1]: Started Proxmox VE firewall.

● pvenetcommit.service - Commit Proxmox VE network changes
     Loaded: loaded (/lib/systemd/system/pvenetcommit.service; enabled; vendor preset: enabled)
     Active: active (exited) since Thu 2022-05-05 17:09:03 EDT; 5min ago
    Process: 3252 ExecStartPre=/bin/rm -f /etc/openvswitch/conf.db (code=exited, status=0/SUCCESS)
    Process: 3320 ExecStart=sh -c if [ -f ${FN}.new ]; then mv ${FN}.new ${FN}; fi (code=exited, status=0/SUCCESS)
   Main PID: 3320 (code=exited, status=0/SUCCESS)
        CPU: 2ms

May 05 17:09:03 pve systemd[1]: Starting Commit Proxmox VE network changes...
May 05 17:09:03 pve systemd[1]: Finished Commit Proxmox VE network changes.

● pvedaemon.service - PVE API Daemon
     Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:08 EDT; 5min ago
    Process: 5625 ExecStart=/usr/bin/pvedaemon start (code=exited, status=255/EXCEPTION)
        CPU: 339ms

May 05 17:09:08 pve systemd[1]: Failed to start PVE API Daemon.
May 05 17:09:08 pve systemd[1]: pvedaemon.service: Start request repeated too quickly.
May 05 17:09:08 pve systemd[1]: pvedaemon.service: Failed with result 'exit-code'.
May 05 17:09:08 pve systemd[1]: Failed to start PVE API Daemon.
May 05 17:09:09 pve systemd[1]: pvedaemon.service: Start request repeated too quickly.
May 05 17:09:09 pve systemd[1]: pvedaemon.service: Failed with result 'exit-code'.
May 05 17:09:09 pve systemd[1]: Failed to start PVE API Daemon.
May 05 17:09:10 pve systemd[1]: pvedaemon.service: Start request repeated too quickly.
May 05 17:09:10 pve systemd[1]: pvedaemon.service: Failed with result 'exit-code'.
May 05 17:09:10 pve systemd[1]: Failed to start PVE API Daemon.

● pvefw-logger.service - Proxmox VE firewall logger
     Loaded: loaded (/lib/systemd/system/pvefw-logger.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-05-05 17:09:03 EDT; 5min ago
    Process: 3250 ExecStart=/usr/sbin/pvefw-logger (code=exited, status=0/SUCCESS)
   Main PID: 3299 (pvefw-logger)
      Tasks: 2 (limit: 76791)
     Memory: 820.0K
        CPU: 25ms
     CGroup: /system.slice/pvefw-logger.service
             └─3299 /usr/sbin/pvefw-logger

May 05 17:09:03 pve systemd[1]: Starting Proxmox VE firewall logger...
May 05 17:09:03 pve pvefw-logger[3299]: starting pvefw logger
May 05 17:09:03 pve systemd[1]: Started Proxmox VE firewall logger.

● pve-ha-lrm.service - PVE Local HA Resource Manager Daemon
     Loaded: loaded (/lib/systemd/system/pve-ha-lrm.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2022-05-05 17:09:08 EDT; 5min ago
    Process: 5629 ExecStart=/usr/sbin/pve-ha-lrm start (code=exited, status=2)
        CPU: 315ms

May 05 17:09:08 pve pve-ha-lrm[5629]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/HA/Resources/PVEVM.pm line 19, <DATA> line 960.
May 05 17:09:08 pve pve-ha-lrm[5629]: Compilation failed in require at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 21, <DATA> line 960.
May 05 17:09:08 pve pve-ha-lrm[5629]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 21, <DATA> line 960.
May 05 17:09:08 pve pve-ha-lrm[5629]: Compilation failed in require at /usr/share/perl5/PVE/Service/pve_ha_lrm.pm line 9, <DATA> line 960.
May 05 17:09:08 pve pve-ha-lrm[5629]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/pve_ha_lrm.pm line 9, <DATA> line 960.
May 05 17:09:08 pve pve-ha-lrm[5629]: Compilation failed in require at /usr/sbin/pve-ha-lrm line 6, <DATA> line 960.
May 05 17:09:08 pve pve-ha-lrm[5629]: BEGIN failed--compilation aborted at /usr/sbin/pve-ha-lrm line 6, <DATA> line 960.
May 05 17:09:08 pve systemd[1]: pve-ha-lrm.service: Control process exited, code=exited, status=2/INVALIDARGUMENT
May 05 17:09:08 pve systemd[1]: pve-ha-lrm.service: Failed with result 'exit-code'.
May 05 17:09:08 pve systemd[1]: Failed to start PVE Local HA Resource Manager Daemon.
 
Resolved:

Code: 
apt-get dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 libpve-common-perl : Depends: libproxmox-rs-perl but it is not installed
 pve-manager : Depends: libproxmox-rs-perl but it is not installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

root@pve:~# apt install libproxmox-rs-perl

root@pve:~# apt-get dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  pve-kernel-5.11 pve-kernel-5.11.22-7-pve pve-kernel-5.13.19-3-pve
Use 'apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
 
  • Like
Reactions: fapsi007, ITT and Rumpelkammer
Thanks, this was very helpful. On several of my servers that I regularly update, I had no issues. On another server I try to only update once every several months, this issue occurred. My guess there was a small upgrade before the 7.2 release that applied this Perl package, and it was mistakenly left out as a dependency from the 7.2 release.

They did add in some new perl-to-rust features in 7.2, so this does seem to be a 7.2 related upgrade bug.

Again, thanks for sharing how you resolved it.
 
You must log in or register to reply here.
Top Bottom