Proxmox Crash and 'unable to activate storage'

Apr 12, 2018
31
2
13
37
Hi,

My Proxmox (ve 5.1) server was not working a few days ago and I hard-reboot it.
It did not work and I had to use 'rescue boot' with the boot CD to fix that.
Since then, I have a storage that does not work anymore.

Here is the log file (/var/log/daemon.log) just before the crash:
Code:
Apr  9 16:33:53 proxmox pveproxy[20245]: worker exit
Apr  9 16:34:00 proxmox systemd[1]: Starting Proxmox VE replication runner...
Apr  9 16:34:02 proxmox systemd[1]: Started Proxmox VE replication runner.
Apr  9 16:35:00 proxmox systemd[1]: Starting Proxmox VE replication runner...
Apr  9 16:35:02 proxmox systemd[1]: Started Proxmox VE replication runner.
Apr  9 16:36:00 proxmox systemd[1]: Starting Proxmox VE replication runner...
Apr  9 16:36:02 proxmox systemd[1]: Started Proxmox VE replication runner.
Apr  9 16:37:00 proxmox systemd[1]: Starting Proxmox VE replication runner...
Apr  9 16:37:02 proxmox systemd[1]: Started Proxmox VE replication runner.
Apr  9 16:38:00 proxmox systemd[1]: Starting Proxmox VE replication runner...
Apr  9 16:38:02 proxmox systemd[1]: Started Proxmox VE replication runner.
Apr  9 16:39:00 proxmox systemd[1]: Starting Proxmox VE replication runner...
Apr  9 16:39:02 proxmox systemd[1]: Started Proxmox VE replication runner.
Apr  9 16:40:00 proxmox systemd[1]: Starting Proxmox VE replication runner...
Apr  9 16:40:02 proxmox systemd[1]: Started Proxmox VE replication runner.
Apr  9 16:40:09 proxmox pveproxy[18651]: worker exit
Apr  9 16:40:09 proxmox pveproxy[41371]: worker 18651 finished
Apr  9 16:40:09 proxmox pveproxy[41371]: starting 1 worker(s)
Apr  9 16:40:09 proxmox pveproxy[41371]: worker 24954 started
Apr  9 16:41:50 proxmox systemd[3494]: Time has been changed
Apr  9 16:41:50 proxmox systemd[1]: Time has been changed
Apr  9 16:41:50 proxmox systemd[1]: apt-daily.timer: Adding 2h 4min 2.175313s random time.
Apr  9 16:41:50 proxmox systemd[1]: apt-daily-upgrade.timer: Adding 9min 18.942264s random time.
Apr  9 16:41:50 proxmox systemd[1]: Starting Proxmox VE replication runner...
Apr  9 16:41:50 proxmox watchdog-mux[841]: client watchdog expired - disable watchdog updates
Apr  9 16:41:50 proxmox watchdog-mux[841]: client watchdog expired - disable watchdog updates
Apr  9 16:41:50 proxmox pve-ha-crm[1404]: loop take too long (90 seconds)
Apr  9 16:41:50 proxmox watchdog-mux[841]: exit watchdog-mux with active connections
Apr  9 16:41:51 proxmox pve-ha-lrm[1415]: loop take too

And now :
Code:
Apr 12 09:12:47 proxmox pvestatd[1505]: unable to activate storage 'local' - directory is expected to be a mount point but is not mounted: '/var/lib/vz'
Apr 12 09:12:57 proxmox pvestatd[1505]: unable to activate storage 'local' - directory is expected to be a mount point but is not mounted: '/var/lib/vz'
Apr 12 09:13:00 proxmox systemd[1]: Starting Proxmox VE replication runner...
Apr 12 09:13:02 proxmox systemd[1]: Started Proxmox VE replication runner.
Apr 12 09:13:07 proxmox pvestatd[1505]: unable to activate storage 'local' - directory is expected to be a mount point but is not mounted: '/var/lib/vz'
Apr 12 09:13:17 proxmox pvestatd[1505]: unable to activate storage 'local' - directory is expected to be a mount point but is not mounted: '/var/lib/vz'
Apr 12 09:13:27 proxmox pvestatd[1505]: unable to activate storage 'local' - directory is expected to be a mount point but is not mounted: '/var/lib/vz'
Apr 12 09:13:37 proxmox pvestatd[1505]: unable to activate storage 'local' - directory is expected to be a mount point but is not mounted: '/var/lib/vz'
Apr 12 09:13:47 proxmox pvestatd[1505]: unable to activate storage 'local' - directory is expected to be a mount point but is not mounted: '/var/lib/vz'
Apr 12 09:13:57 proxmox pvestatd[1505]: unable to activate storage 'local' - directory is expected to be a mount point but is not mounted: '/var/lib/vz'
Apr 12 09:14:00 proxmox systemd[1]: Starting Proxmox VE replication runner...
Apr 12 09:14:02 proxmox systemd[1]: Started Proxmox VE replication runner.
Apr 12 09:14:07 proxmox pvestatd[1505]: unable to activate storage 'local' - directory is expected to be a mount point but is not mounted: '/var/lib/vz'
Apr 12 09:14:17 proxmox pvestatd[1505]: unable to activate storage 'local' - directory is expected to be a mount point but is not mounted: '/var/lib/vz'
Apr 12 09:14:27 proxmox pvestatd[1505]: unable to activate storage 'local' - directory is expected to be a mount point but is not mounted: '/var/lib/vz'
Apr 12 09:14:37 proxmox pvestatd[1505]: unable to activate storage 'local' - directory is expected to be a mount point but is not mounted: '/var/lib/vz'
Apr 12 09:14:47 proxmox pvestatd[1505]: unable to activate storage 'local' - directory is expected to be a mount point but is not mounted: '/var/lib/vz'

I would like to know the cause of the crash, and how to fix the storage.
Thanks
 
Hi,

this look like a partition is missing.

check fstab and check why /var/lib/vz can't be mounted.
 
This storage is working now, yet there is still the error message in the log, weird...

What does mean "client watchdog expired - disable watchdog updates" ?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!