Hello everyone,
I have been using Proxmox VE for a few years now, and this is the first major issue I have had with it. Having tried most of the alternatives, I love this hypervisor, besides the problem I'm having this week.
So I updated one node of my cluster via the GUI and everything went seemingly normal. A reboot due to kernel update was requested at the end of the update, so I shut my VMs down and triggered a reboot. That is when the node went permanently offline. My recovery plan is to manually migrate the VMs, from the failed node to my working node, with SysRec, then investigate the system logs and attempt to get the node back online. If I am unable to restore the OS, I will reimage and reconfigure.
As per the title, my nodes are headless dedicated server rentals from an unmanaged service provider. The facility has crash carts available but I prefer to use that as a last resort. Currently, there is a SysRec live image loaded and I am doing my investigation and recovery from that. In the system journal, I see a normal software update followed by a normal shutdown.
The last time I had to manually migrate VMs, they were located in /etc/pve and that folder seems to be missing on this node. I have a shell session opened to my functioning node for comparison. Using df -h I can see the disk usage is more than my functioning node, so my VMs must still be on that machine, which gets us to my main questions.
Where else might they be saved?
Is /etc/pve linked elsewhere that I might not see with the filesystem mounted in SysRec?
I appreciate your time reading and any assistance provided,
Brad
I have been using Proxmox VE for a few years now, and this is the first major issue I have had with it. Having tried most of the alternatives, I love this hypervisor, besides the problem I'm having this week.
So I updated one node of my cluster via the GUI and everything went seemingly normal. A reboot due to kernel update was requested at the end of the update, so I shut my VMs down and triggered a reboot. That is when the node went permanently offline. My recovery plan is to manually migrate the VMs, from the failed node to my working node, with SysRec, then investigate the system logs and attempt to get the node back online. If I am unable to restore the OS, I will reimage and reconfigure.
As per the title, my nodes are headless dedicated server rentals from an unmanaged service provider. The facility has crash carts available but I prefer to use that as a last resort. Currently, there is a SysRec live image loaded and I am doing my investigation and recovery from that. In the system journal, I see a normal software update followed by a normal shutdown.
The last time I had to manually migrate VMs, they were located in /etc/pve and that folder seems to be missing on this node. I have a shell session opened to my functioning node for comparison. Using df -h I can see the disk usage is more than my functioning node, so my VMs must still be on that machine, which gets us to my main questions.
Where else might they be saved?
Is /etc/pve linked elsewhere that I might not see with the filesystem mounted in SysRec?
I appreciate your time reading and any assistance provided,
Brad
Last edited: