[SOLVED] Restored VM will not boot from HDD

Kenneth_H

Well-Known Member
Feb 28, 2017
30
1
48
31
Hi
As part of testing out Proxmox for our new virtualization infrastructure, we have configured an offsite server with the same Proxmox VE 4.4-13/7ea56165 as our internal 5 node cluster.
The cluster runs a daily backup to a RAID6 NFS-share on our storage server. This server then does an rsync to a folder on the remote proxmox server. This folder is then created as a backup storage location in the proxmox webGUI on the remote proxmox server.
The purpose of this configuration is to provide fast disaster recovery from backups on the internal network if a node or VM dies and then use the remote server to restore mission critical VM's if our entire location goes down due to fx. fire, loss of power, long term outage on the internet connection etc.

However our test did not succeed as expected. We did perform the local scheduled backup of all test VM's on our internal PVE cluster to the internal backup server and this was also sent to the remote server using rsync, but upon restore of the VM on the remote PVE server it was unable to boot from the harddisk. The VM restores without errors and we then correct the network configuration to match the remote environment, while keeping everything else as it was when restored. We boot the VM, but it keeps stuck with a blinking cursor when trying to boot from the hard disk.

We then downloaded the backup from the remote server, onto the same node it was normally running on, using SCP and restored it here. It then booted fine.

Our local cluster is a mix of different servers, where most of them are using Intel Xeon E5620 processors and one server has 4 Intel Xeon X7550. The remote server uses a newer Intel Xeon E3-1240Lv3 and our test VM that was running a fresh and fully updated installation of CentOS 7.3 with nothing else installed.

How do I debug the issue on the remote server when no error or warning output is shown on screen or in the task log of the web GUI?
 
Last edited:
Sure.
Here is the config file from the original system on our internal cluster:
Code:
bootdisk: scsi0
cores: 1
ide2: vm-shared:iso/CentOS-7-x86_64-Minimal-1611.iso,media=cdrom,size=680M
memory: 2048
name: kundeservice.conviator.com
net0: virtio=5A:E8:AE:B8:9F:35,bridge=vmbr1
numa: 1
ostype: l26
scsi0: vm-shared:100/vm-100-disk-1.raw,size=32G
scsihw: virtio-scsi-pci
smbios1: uuid=9eaa4181-2aa2-4e3b-882a-cdd7cc5f73a0
sockets: 1

Here is the same file after being restored and corrected to use NAT-networking on the remote server:
Code:
bootdisk: scsi0
cores: 1
ide2: none,media=cdrom
memory: 2048
name: kundeservice.conviator.com
net0: virtio=5A:E8:AE:B8:9F:35
numa: 1
ostype: l26
scsi0: local:100/vm-100-disk-1.raw,size=32G
scsihw: virtio-scsi-pci
smbios1: uuid=9eaa4181-2aa2-4e3b-882a-cdd7cc5f73a0
sockets: 1
 
Hi,
see no issue… boot the backup-VM from CD?
Do you see grub, or hang the VM before?
Has the target node Numa?

Which pve-version is running on the target?

Something special with your local storage? Perhaps try cache=writethrough


Udo
 
Hi,
I does not even show grub. It is just stuck at the screen saying "booting from hard disk".
Skærmbillede 2017-03-25 kl. 21.06.09.png
I tried booting from a CD and that still works.
I tried to disable Numa on the VM and that did not change anything. Also changed the disk use write-through. Still no change.
The remote server is running Proxmox VE 4.4-13/7ea56165, which is the same version as the internal cluster. This was done to avoid issues due to mismatching versions.
 
Hi again
We just made a new test where we installed a Windows VM that used IDE for the hard drive in Proxmox VE and did the same procedure and restored this on the remote system. The Windows installation seems to boot without any issues.
I then edited the configuration file for the Linux VM to also use IDE, as I thought that maybe the issue was related to the virtIO scsi controller, but even after changing the Linux VM to use IDE, it did still not boot.

So maybe this is something that is specific to Linux
 
Hi again
We just made a new test where we installed a Windows VM that used IDE for the hard drive in Proxmox VE and did the same procedure and restored this on the remote system. The Windows installation seems to boot without any issues.
I then edited the configuration file for the Linux VM to also use IDE, as I thought that maybe the issue was related to the virtIO scsi controller, but even after changing the Linux VM to use IDE, it did still not boot.

So maybe this is something that is specific to Linux
Hi,
perhaps somthing wreid with the grub?! Can you boot an CD on the backup-vm and do an grub-install/update-grub?

Udo
 
… or there is a serious issue with grub in a VM on proxmox.
Hi,
I think not that is an systematic issue which is proxmox related. I have moved in the last two weeks more than 40 VMs from an older system to an new cluster with backup/restore and all - execpt one - work without trouble. The only one VM, which arent boot, doesn't boot on the old server too, because auto-update was active and the new kernels fill /boot - so that the last kernel wasn't install right...

Perhaps your original VM also doesn't boot?

Udo
 
Hi
I just checked the original VM, by turning it off and then perform a cold boot. This boots fine.
I tried to then perform a new backup and compare the size against the previous backup, and it seems something was wrong in that backup when it was sent to the offsite server.

I restored the new backup on the remote system and it now works fine.

Thank you for your help and guidance on this issue
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!