All Nodes/VM online, but pvestatd shows offline

wahmed

Famous Member
Oct 28, 2012
1,148
59
113
Calgary, Canada
www.symmcom.com
Sometimes last night during VM backup, one of the HDD in my NFS backup server failed which degraded the ZFS pool. Since then all my Proxmox Nodes shows offline includes all VMs in them. If i run #service pvestatd restart on nodes, it seem to show them online for little bit but then goes offline again.
on-offline.png
All the VMs working though. I am actually creating this post from one of the Windows VM that shows offline.
This is not the first time it happened and it seem to mostly related with NFS. Is there a way to prevent this? The solution i see right now is to reboot the 2nd node which is where backup was running. But it is somewhat hassle to stop/migrate all the VMs.
Thanks!
 
Are you able to create new NFS connection to the NFS server?

If this is possible you could try killing all NFS processes on the pve node related to the specific NFS connection and then unmount the NFS mount in question. Proxmox should then try to reconnect to the NFS server.