[SOLVED] Proxmox like dead after unexpected shutdown

prox_noob

New Member
Jun 22, 2022
6
1
3
I need help pretty quick.
After power went out and my server restarted none of the VMs came back to life neither did proxmox. Can´t access webinterface only over putty and the IP. Can´t seem to find the problem since I dunno that much ´bout proxmox.
 
hi,

Can´t access webinterface only over putty and the IP. Can´t seem to find the problem since I dunno that much ´bout proxmox.
if you can SSH to your server then you can check the journals for any apparent error messages.

please run journalctl -b0 > journal.txt and attach your journal here if you need help, along with the output from pveversion -v on your server.
 
hi,


if you can SSH to your server then you can check the journals for any apparent error messages.

please run journalctl -b0 > journal.txt and attach your journal here if you need help, along with the output from pveversion -v on your server.
alright. ty for the fast answer.
its really much tho..
pveversion is at the buttom
 

Attachments

seems like pve-cluster service is not running correctly, that's why the other services aren't coming up either.

excerpt from your log:

Code:
1258 Jun 22 10:27:24 pve systemd[1]: Starting The Proxmox VE cluster filesystem...                                                                                                                                                                                                          
1259 Jun 22 10:27:24 pve pmxcfs[1087]: fuse: mountpoint is not empty                                                                                                                                                                                                                        
1260 Jun 22 10:27:24 pve pmxcfs[1087]: fuse: if you are sure this is safe, use the 'nonempty' mount option                                                                                                                                                                                  
1261 Jun 22 10:27:24 pve pmxcfs[1087]: [main] crit: fuse_mount error: File exists                                                                                                                                                                                                           
1262 Jun 22 10:27:24 pve pmxcfs[1087]: [main] notice: exit proxmox configuration filesystem (-1)                                                                                                                                                                                            
1263 Jun 22 10:27:24 pve pmxcfs[1087]: [main] crit: fuse_mount error: File exists                                                                                                                                                                                                           
1264 Jun 22 10:27:24 pve pmxcfs[1087]: [main] notice: exit proxmox configuration filesystem (-1)                                                                                                                                                                                            
1265 Jun 22 10:27:24 pve systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION                                                                                                                                                                         
1266 Jun 22 10:27:24 pve systemd[1]: pve-cluster.service: Failed with result 'exit-code'.                                                                                                                                                                                                   
1267 Jun 22 10:27:24 pve systemd[1]: Failed to start The Proxmox VE cluster filesystem.

the error mentions that the mountpoint is not empty, it's talking about the cluster filesystem that's normally mounted in /etc/pve/ directory [0].
so it seems like you might have some files on there.

you could do the following to take a backup of the directory and restart the service to see if it helps:
Code:
tar czvf /root/etc-pve.tar.gz /etc/pve # create a backup archive of the directory contents
systemctl stop pve-cluster # stop the service just in case
rm -rf /etc/pve # clean it up
systemctl start pve-cluster

if that works you should be able to restart the rest of the services

hope this helps

[0]: https://pve.proxmox.com/pve-docs/chapter-pmxcfs.html
 
seems like pve-cluster service is not running correctly, that's why the other services aren't coming up either.

excerpt from your log:

Code:
1258 Jun 22 10:27:24 pve systemd[1]: Starting The Proxmox VE cluster filesystem...                                                                                                                                                                                                         
1259 Jun 22 10:27:24 pve pmxcfs[1087]: fuse: mountpoint is not empty                                                                                                                                                                                                                       
1260 Jun 22 10:27:24 pve pmxcfs[1087]: fuse: if you are sure this is safe, use the 'nonempty' mount option                                                                                                                                                                                 
1261 Jun 22 10:27:24 pve pmxcfs[1087]: [main] crit: fuse_mount error: File exists                                                                                                                                                                                                          
1262 Jun 22 10:27:24 pve pmxcfs[1087]: [main] notice: exit proxmox configuration filesystem (-1)                                                                                                                                                                                           
1263 Jun 22 10:27:24 pve pmxcfs[1087]: [main] crit: fuse_mount error: File exists                                                                                                                                                                                                          
1264 Jun 22 10:27:24 pve pmxcfs[1087]: [main] notice: exit proxmox configuration filesystem (-1)                                                                                                                                                                                           
1265 Jun 22 10:27:24 pve systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION                                                                                                                                                                        
1266 Jun 22 10:27:24 pve systemd[1]: pve-cluster.service: Failed with result 'exit-code'.                                                                                                                                                                                                  
1267 Jun 22 10:27:24 pve systemd[1]: Failed to start The Proxmox VE cluster filesystem.

the error mentions that the mountpoint is not empty, it's talking about the cluster filesystem that's normally mounted in /etc/pve/ directory [0].
so it seems like you might have some files on there.

you could do the following to take a backup of the directory and restart the service to see if it helps:
Code:
tar czvf /root/etc-pve.tar.gz /etc/pve # create a backup archive of the directory contents
systemctl stop pve-cluster # stop the service just in case
rm -rf /etc/pve # clean it up
systemctl start pve-cluster

if that works you should be able to restart the rest of the services

hope this helps

[0]: https://pve.proxmox.com/pve-docs/chapter-pmxcfs.html
omg it worked. I love you
thanks
 
  • Like
Reactions: oguz
For us to avoid such future issues, could you please post what files were wrongly in the mountpoint before pmxcfs being started:
tar tf /root/etc-pve.tar.gz
when i paste that code and enter I only get this:

root@pve:~# tar tf /root/etc-pve.tar.gz
etc/pve/
etc/pve/pve-root-ca.pem

Idk if thats what you where looking for..
 
A hard reset of a server is never good and you will loose more or less data, especially if you got consumer SSDs and so one without a powerloss protection. So its always a good idea to put your server behind a UPS (useful ones start at 50$) that gracefully shutdowns your server on an power outage so such problems won't happen in the first place.