[SOLVED] Promox not starting due to Failed to start The Proxmox VE cluster filesystem error.

[Lxndr]

Hi All,

I have an issue with my Proxmox server, I have ran out of space on the hard drives and I'm unable to connect to the server any more, I've tried removing some of the VM that where not being used in the last several months and did a reboot of the server in order to recover some space but now the web gui is not available.

I've tried to restart the pve-cluster.service via the cli but I'm getting the below error, any indication on what I need to look into to get this solved would be highly appreciated, I've seem other cases with similar errors but the solutions provided have not worked for me:

pve-cluster.service - The Proxmox VE cluster filesystem
     Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sat 2022-08-13 21:29:19 CEST; 6min ago
    Process: 1081 ExecStart=/usr/bin/pmxcfs (code=exited, status=255/EXCEPTION)
        CPU: 8ms

Aug 13 21:29:19 pve systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 5.
Aug 13 21:29:19 pve systemd[1]: Stopped The Proxmox VE cluster filesystem.
Aug 13 21:29:19 pve systemd[1]: pve-cluster.service: Start request repeated too quickly.
Aug 13 21:29:19 pve systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
Aug 13 21:29:19 pve systemd[1]: Failed to start The Proxmox VE cluster filesystem.

● pveproxy.service - PVE API Proxy Server
     Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2022-08-13 21:29:19 CEST; 6min ago
    Process: 1079 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=111)
    Process: 1080 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
   Main PID: 1083 (pveproxy)
      Tasks: 4 (limit: 76937)
     Memory: 131.0M
        CPU: 9.102s
     CGroup: /system.slice/pveproxy.service
             ├─1083 pveproxy
             ├─1389 pveproxy worker
             ├─1390 pveproxy worker
             └─1391 pveproxy worker

Aug 13 21:35:42 pve pveproxy[1387]: worker exit
Aug 13 21:35:42 pve pveproxy[1083]: worker 1387 finished
Aug 13 21:35:42 pve pveproxy[1083]: starting 1 worker(s)
Aug 13 21:35:42 pve pveproxy[1083]: worker 1390 started
Aug 13 21:35:42 pve pveproxy[1388]: worker exit
Aug 13 21:35:42 pve pveproxy[1390]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1891.
Aug 13 21:35:42 pve pveproxy[1083]: worker 1388 finished
Aug 13 21:35:42 pve pveproxy[1083]: starting 1 worker(s)
Aug 13 21:35:42 pve pveproxy[1083]: worker 1391 started
Aug 13 21:35:42 pve pveproxy[1391]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1891.

● pvedaemon.service - PVE API Daemon
     Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2022-08-13 21:29:18 CEST; 6min ago
    Process: 1065 ExecStart=/usr/bin/pvedaemon start (code=exited, status=0/SUCCESS)
   Main PID: 1073 (pvedaemon)
      Tasks: 4 (limit: 76937)
     Memory: 139.3M
        CPU: 939ms
     CGroup: /system.slice/pvedaemon.service
             ├─1073 pvedaemon
             ├─1074 pvedaemon worker
             ├─1075 pvedaemon worker
             └─1076 pvedaemon worker

Aug 13 21:29:17 pve systemd[1]: Starting PVE API Daemon...
Aug 13 21:29:18 pve pvedaemon[1073]: starting server
Aug 13 21:29:18 pve pvedaemon[1073]: starting 3 worker(s)
Aug 13 21:29:18 pve pvedaemon[1073]: worker 1074 started
Aug 13 21:29:18 pve pvedaemon[1073]: worker 1075 started
Aug 13 21:29:18 pve pvedaemon[1073]: worker 1076 started
Aug 13 21:29:18 pve systemd[1]: Started PVE API Daemon.
~
~

Thanks

Al

 

[cool]

mark

 

[Lxndr]

Hi everyone ,

So if I not mistaking, this is all because I have no free space left in the drives, what I can "safely" delete in order to get the server back running?

If check the /etc/pve directory and it's empty which I was not expecting to be honest.

 

[Lxndr]

I've managed to sort this out.

Thanks