pve* services fail to load

eugrus

Member
Aug 2, 2022
18
3
8
So I've decided to try out Proxmox on a VPS and installed it on top of a fresh Debian 9 instance by following https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Stretch

My hosting is known to work fine with nested virtualization in general: I've tried it out with kvm/qemu+virtmanager before.

The following Proxmox daemons fail to load though. Here are the diagnostics I've gathered for those services:
Code:
root@vpshost:~# systemctl status pveproxy.service >> errors
root@vpshost:~# systemctl status pve-guests.service >> errors
root@vpshost:~# systemctl status pve-ha-crm.service >> errors
root@vpshost:~# systemctl status pve-ha-lrm.service >> errors
root@vpshost:~# systemctl status smartd.service >> errors
root@vpshost:~# cat errors
● pvedaemon.service - PVE API Daemon
   Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2022-08-23 20:12:53 MSK; 48s ago
  Process: 23993 ExecStop=/usr/bin/pvedaemon stop (code=exited, status=2)
  Process: 14591 ExecStart=/usr/bin/pvedaemon start (code=exited, status=255)
 Main PID: 1083 (code=killed, signal=KILL)
      CPU: 1.243s

Aug 23 20:12:51 vpshost systemd[1]: Starting PVE API Daemon...
Aug 23 20:12:53 vpshost pvedaemon[14591]: start failed - can't put server into background - fork failed at /usr/share/perl5/PVE/Daemon.pm line 322.
Aug 23 20:12:53 vpshost pvedaemon[14591]: start failed - can't put server into background - fork failed at /usr/share/perl5/PVE/Daemon.pm line 322.
Aug 23 20:12:53 vpshost systemd[1]: pvedaemon.service: Control process exited, code=exited status=255
Aug 23 20:12:53 vpshost systemd[1]: Failed to start PVE API Daemon.
Aug 23 20:12:53 vpshost systemd[1]: pvedaemon.service: Unit entered failed state.
Aug 23 20:12:53 vpshost systemd[1]: pvedaemon.service: Failed with result 'exit-code'.
● pveproxy.service - PVE API Proxy Server
   Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sat 2022-08-20 01:25:27 MSK; 3 days ago
  Process: 1466 ExecStart=/usr/bin/pveproxy start (code=exited, status=2)
      CPU: 494ms

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
● pve-guests.service - PVE guests
   Loaded: loaded (/lib/systemd/system/pve-guests.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sat 2022-08-20 01:20:32 MSK; 3 days ago
  Process: 1179 ExecStart=/usr/bin/pvesh --nooutput create /nodes/localhost/startall (code=exited, status=2)
 Main PID: 1179 (code=exited, status=2)
      CPU: 528ms

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
● pve-ha-crm.service - PVE Cluster Ressource Manager Daemon
   Loaded: loaded (/lib/systemd/system/pve-ha-crm.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sat 2022-08-20 01:20:30 MSK; 3 days ago
  Process: 1100 ExecStart=/usr/sbin/pve-ha-crm start (code=exited, status=2)
      CPU: 465ms

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
● pve-ha-lrm.service - PVE Local HA Ressource Manager Daemon
   Loaded: loaded (/lib/systemd/system/pve-ha-lrm.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sat 2022-08-20 01:20:31 MSK; 3 days ago
  Process: 1152 ExecStart=/usr/sbin/pve-ha-lrm start (code=exited, status=2)
      CPU: 515ms

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
● smartd.service - Self Monitoring and Reporting Technology (SMART) Daemon
   Loaded: loaded (/lib/systemd/system/smartd.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sat 2022-08-20 01:20:23 MSK; 3 days ago
     Docs: man:smartd(8)
           man:smartd.conf(5)
  Process: 503 ExecStart=/usr/sbin/smartd -n $smartd_opts (code=exited, status=17)
 Main PID: 503 (code=exited, status=17)
      CPU: 18ms

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
root@vpshost:~#

Any ideas if there's a fix?
 
Ahh - sorry: here's the actual journal:

Code:
root@vpshost:~# rm errors
root@vpshost:~# systemctl status pveproxy.service >> errors
root@vpshost:~# systemctl status pvedaemon.service >> errors
root@vpshost:~# systemctl status pve-guests.service >> errors
root@vpshost:~# systemctl status pve-ha-crm.service >> errors
root@vpshost:~# systemctl status pve-ha-lrm.service >> errors
root@vpshost:~# systemctl status smartd.service >> errors
root@vpshost:~# cat errors
● pveproxy.service - PVE API Proxy Server
   Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2022-08-23 21:23:43 MSK; 4min 1s ago
  Process: 1095 ExecStart=/usr/bin/pveproxy start (code=exited, status=2)
      CPU: 452ms

Aug 23 21:23:42 vpshost pveproxy[1095]: Compilation failed in require at /usr/share/perl5/PVE/API2.pm line 13, <DATA> line 755.
Aug 23 21:23:42 vpshost pveproxy[1095]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2.pm line 13, <DATA> line 755.
Aug 23 21:23:42 vpshost pveproxy[1095]: Compilation failed in require at /usr/share/perl5/PVE/Service/pveproxy.pm line 15, <DATA> line 755.
Aug 23 21:23:42 vpshost pveproxy[1095]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/pveproxy.pm line 15, <DATA> line 755.
Aug 23 21:23:42 vpshost pveproxy[1095]: Compilation failed in require at /usr/bin/pveproxy line 11, <DATA> line 755.
Aug 23 21:23:42 vpshost pveproxy[1095]: BEGIN failed--compilation aborted at /usr/bin/pveproxy line 11, <DATA> line 755.
Aug 23 21:23:43 vpshost systemd[1]: pveproxy.service: Control process exited, code=exited status=2
Aug 23 21:23:43 vpshost systemd[1]: Failed to start PVE API Proxy Server.
Aug 23 21:23:43 vpshost systemd[1]: pveproxy.service: Unit entered failed state.
Aug 23 21:23:43 vpshost systemd[1]: pveproxy.service: Failed with result 'exit-code'.
● pvedaemon.service - PVE API Daemon
   Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2022-08-23 21:23:41 MSK; 4min 11s ago
  Process: 1025 ExecStart=/usr/bin/pvedaemon start (code=exited, status=0/SUCCESS)
 Main PID: 1085 (pvedaemon)
    Tasks: 4 (limit: 4915)
   Memory: 115.7M
      CPU: 1.453s
   CGroup: /system.slice/pvedaemon.service
           ├─1085 pvedaemon
           ├─1089 pvedaemon worker
           ├─1090 pvedaemon worker
           └─1091 pvedaemon worker

Aug 23 21:23:38 vpshost systemd[1]: Starting PVE API Daemon...
Aug 23 21:23:41 vpshost pvedaemon[1085]: starting server
Aug 23 21:23:41 vpshost pvedaemon[1085]: starting 3 worker(s)
Aug 23 21:23:41 vpshost pvedaemon[1085]: worker 1089 started
Aug 23 21:23:41 vpshost pvedaemon[1085]: worker 1090 started
Aug 23 21:23:41 vpshost pvedaemon[1085]: worker 1091 started
Aug 23 21:23:41 vpshost systemd[1]: Started PVE API Daemon.
● pve-guests.service - PVE guests
   Loaded: loaded (/lib/systemd/system/pve-guests.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2022-08-23 21:23:44 MSK; 4min 14s ago
  Process: 1167 ExecStart=/usr/bin/pvesh --nooutput create /nodes/localhost/startall (code=exited, status=2)
 Main PID: 1167 (code=exited, status=2)
      CPU: 405ms

Aug 23 21:23:44 vpshost pvesh[1167]: Compilation failed in require at /usr/share/perl5/PVE/API2.pm line 13, <DATA> line 755.
Aug 23 21:23:44 vpshost pvesh[1167]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2.pm line 13, <DATA> line 755.
Aug 23 21:23:44 vpshost pvesh[1167]: Compilation failed in require at /usr/share/perl5/PVE/CLI/pvesh.pm line 16, <DATA> line 755.
Aug 23 21:23:44 vpshost pvesh[1167]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/CLI/pvesh.pm line 16, <DATA> line 755.
Aug 23 21:23:44 vpshost pvesh[1167]: Compilation failed in require at /usr/bin/pvesh line 6, <DATA> line 755.
Aug 23 21:23:44 vpshost pvesh[1167]: BEGIN failed--compilation aborted at /usr/bin/pvesh line 6, <DATA> line 755.
Aug 23 21:23:44 vpshost systemd[1]: pve-guests.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Aug 23 21:23:44 vpshost systemd[1]: Failed to start PVE guests.
Aug 23 21:23:44 vpshost systemd[1]: pve-guests.service: Unit entered failed state.
Aug 23 21:23:44 vpshost systemd[1]: pve-guests.service: Failed with result 'exit-code'.
● pve-ha-crm.service - PVE Cluster Ressource Manager Daemon
   Loaded: loaded (/lib/systemd/system/pve-ha-crm.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2022-08-23 21:23:43 MSK; 4min 23s ago
  Process: 1094 ExecStart=/usr/sbin/pve-ha-crm start (code=exited, status=2)
      CPU: 454ms

Aug 23 21:23:42 vpshost pve-ha-crm[1094]: Compilation failed in require at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 20, <DATA> line 755.
Aug 23 21:23:42 vpshost pve-ha-crm[1094]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 20, <DATA> line 755.
Aug 23 21:23:42 vpshost pve-ha-crm[1094]: Compilation failed in require at /usr/share/perl5/PVE/Service/pve_ha_crm.pm line 9, <DATA> line 755.
Aug 23 21:23:42 vpshost pve-ha-crm[1094]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/pve_ha_crm.pm line 9, <DATA> line 755.
Aug 23 21:23:42 vpshost pve-ha-crm[1094]: Compilation failed in require at /usr/sbin/pve-ha-crm line 6, <DATA> line 755.
Aug 23 21:23:42 vpshost pve-ha-crm[1094]: BEGIN failed--compilation aborted at /usr/sbin/pve-ha-crm line 6, <DATA> line 755.
Aug 23 21:23:43 vpshost systemd[1]: pve-ha-crm.service: Control process exited, code=exited status=2
Aug 23 21:23:43 vpshost systemd[1]: Failed to start PVE Cluster Ressource Manager Daemon.
Aug 23 21:23:43 vpshost systemd[1]: pve-ha-crm.service: Unit entered failed state.
Aug 23 21:23:43 vpshost systemd[1]: pve-ha-crm.service: Failed with result 'exit-code'.
● pve-ha-lrm.service - PVE Local HA Ressource Manager Daemon
   Loaded: loaded (/lib/systemd/system/pve-ha-lrm.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2022-08-23 21:23:44 MSK; 4min 27s ago
  Process: 1129 ExecStart=/usr/sbin/pve-ha-lrm start (code=exited, status=2)
      CPU: 454ms

Aug 23 21:23:44 vpshost pve-ha-lrm[1129]: Compilation failed in require at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 20, <DATA> line 755.
Aug 23 21:23:44 vpshost pve-ha-lrm[1129]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/HA/Env/PVE2.pm line 20, <DATA> line 755.
Aug 23 21:23:44 vpshost pve-ha-lrm[1129]: Compilation failed in require at /usr/share/perl5/PVE/Service/pve_ha_lrm.pm line 9, <DATA> line 755.
Aug 23 21:23:44 vpshost pve-ha-lrm[1129]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/pve_ha_lrm.pm line 9, <DATA> line 755.
Aug 23 21:23:44 vpshost pve-ha-lrm[1129]: Compilation failed in require at /usr/sbin/pve-ha-lrm line 6, <DATA> line 755.
Aug 23 21:23:44 vpshost pve-ha-lrm[1129]: BEGIN failed--compilation aborted at /usr/sbin/pve-ha-lrm line 6, <DATA> line 755.
Aug 23 21:23:44 vpshost systemd[1]: pve-ha-lrm.service: Control process exited, code=exited status=2
Aug 23 21:23:44 vpshost systemd[1]: Failed to start PVE Local HA Ressource Manager Daemon.
Aug 23 21:23:44 vpshost systemd[1]: pve-ha-lrm.service: Unit entered failed state.
Aug 23 21:23:44 vpshost systemd[1]: pve-ha-lrm.service: Failed with result 'exit-code'.
● smartd.service - Self Monitoring and Reporting Technology (SMART) Daemon
   Loaded: loaded (/lib/systemd/system/smartd.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2022-08-23 21:23:35 MSK; 4min 43s ago
     Docs: man:smartd(8)
           man:smartd.conf(5)
  Process: 502 ExecStart=/usr/sbin/smartd -n $smartd_opts (code=exited, status=17)
 Main PID: 502 (code=exited, status=17)
      CPU: 19ms

Aug 23 21:23:35 vpshost systemd[1]: Started Self Monitoring and Reporting Technology (SMART) Daemon.
Aug 23 21:23:35 vpshost systemd[1]: smartd.service: Main process exited, code=exited, status=17/n/a
Aug 23 21:23:35 vpshost systemd[1]: smartd.service: Unit entered failed state.
Aug 23 21:23:35 vpshost systemd[1]: smartd.service: Failed with result 'exit-code'.
root@vpshost:~#

Disk and RAM usage are both at about 50% though.
 
do the files mentioned in the output look ok to you? They should be plain text perl code.
It seems like you have a corrupted installation, perhaps you lost power at some point? Or may be node was hard reset without syncing to disk.
You probably need to force reinstall all the packages.


Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox