Disk unbootable after qmrestore!

A

axelgenus

Guest
Hi guys,
I've used Proxmox on a production server for a month and I'm quite happy with it. I'm using the 2.6.35 kernel branch. Right now I have only one OS installed in a VM but I'm planning to add more. I set the backup feature on and tried to restore from a recent backup using qmrestore for testing purposes. Here it comes the problem: the VM was re-created successfully but the OS was unbootable!

I used virtio and the Red Hat drivers to install the OS and everything worked just fine.

Here's the vzdump log:

Code:
Nov 28 05:00:01 INFO: Starting Backup of VM 101 (qemu)
Nov 28 05:00:01 INFO: running
Nov 28 05:00:01 INFO: status = running
Nov 28 05:00:02 INFO: backup mode: snapshot
Nov 28 05:00:02 INFO: ionice priority: 7
Nov 28 05:00:02 INFO:   Logical volume "vzsnap-hera-0" created
Nov 28 05:00:02 INFO: creating archive '/mnt/backups/vzdump-qemu-101-2010_11_28-05_00_01.tgz'
Nov 28 05:00:02 INFO: adding '/mnt/backups/vzdump-qemu-101-2010_11_28-05_00_01.tmp/qemu-server.conf' to archive ('qemu-server.conf')
Nov 28 05:00:02 INFO: adding '/dev/VolProxmox/vzsnap-hera-0' to archive ('vm-disk-virtio0.raw')
Nov 28 05:31:40 INFO: Total bytes written: 14045342208 (7.06 MiB/s)
Nov 28 05:31:40 INFO: archive file size: 5.39GB
Nov 28 05:31:40 INFO: delete old backup '/mnt/backups/vzdump-qemu-101-2010_11_27-05_00_01.tgz'
Nov 28 05:31:41 INFO:   Logical volume "vzsnap-hera-0" successfully removed
Nov 28 05:31:41 INFO: Finished Backup of VM 101 (00:31:40)

I used the line "qmrestore /mnt/backups/vzdump-qemu-101-2010_11_28-05_00_01.tgz 101" to start the program and no errors were found.

I also tried to boot from a recovery CD but the partition was gone.

What did happened? :confused:
 
# pveversion -v
Code:
pve-manager: 1.6-5 (pve-manager/1.6/5261)
running kernel: 2.6.35-1-pve
proxmox-ve-2.6.35: 1.6-7
pve-kernel-2.6.35-1-pve: 2.6.35-7
qemu-server: 1.1-22
pve-firmware: 1.0-9
libpve-storage-perl: 1.0-14
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-8
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.12.5-2
ksm-control-daemon: 1.0-4
 
Is that bug reproducible (how)?
Well... I didn't really wanted to try that again (I deleted the old VM in order to restore it). I'm going to try on a different VM #.

However I simply gave the qmrestore command I specified before and accessed the Proxmox CP to start it.
 
Perhaps you can try 'qm start 101' and see what error message you get ?
It happened to me that I previously configured a VM with an iso image as CDROM (to install it), and that I removed the iso afterwards. It did not find the iso and refused to start. Once I removed the iso in the configuration, it worked.
It is perhaps not the same thing here, but the error message can be helpful...

Alain
 
This is the output of qmrestore:

Code:
# qmrestore vzdump-qemu-101-2010_11_28-05_00_01.tgz 501
INFO: restore QemuServer backup 'vzdump-qemu-101-2010_11_28-05_00_01.tgz' using ID 501
INFO: extracting 'qemu-server.conf' from archive
INFO: extracting 'vm-disk-virtio0.raw' from archive
INFO:   Rounding up size to full physical extent 120.00 GB
INFO:   Logical volume "vm-501-disk-1" created
INFO: new volume ID is 'Dischi_Virtuali:vm-501-disk-1'
INFO: restore data to '/dev/VolProxmox/vm-501-disk-1' (128849018880 bytes)
INFO: 17449+1911059 records in
INFO: 17449+1911059 records out
INFO: 128849018880 bytes (129 GB) copied, 2792.28 s, 46.1 MB/s
INFO: restore QemuServer backup 'vzdump-qemu-101-2010_11_28-05_00_01.tgz' successful

and this is the result:

screen01.JPG

Nothing changed. :(

@alain: the VM starts with no errors. The disk is simply gone over the backup/restore cycle.
 
There is something strange in your configuration. qmretore use by default the storage 'dev/VolProxmoxw'. You did not provide the option '-storage' to define another storage...
In a standard Proxmox installation, you have volume group 'pve', and logical volume 'data', so the default storage should be, I think, '/dev/pve/data/vm-501-disk-1'. How did you configure your Proxmox server ?

Alain
 
I've installed it on top of a clean Debian Linux 5.0 (fully updated). I manually created the LVM volume and called it VolProxmox.
 
In a standard Proxmox installation, the files should be in '/var/lib/vz/images/501'. Do you see your image files there ? Or do you use one logical volume per machine ?
I think it could fool qmrestore...

Did you try 'qm start 501' on the console. All you showed is a VNC screen copy, so perhaps you use the web interface to start the VM. On the console, the messages after the command qm can give you some hints.

But with such a special configuration, I fear you are on your own to debug. It is not supported, as it is not standard configuration...

Also did you use the option 'snapshot' to bakckup your machine ? With such an option, you must have free space on your volume group to store the snapshot...
 
In a standard Proxmox installation, the files should be in '/var/lib/vz/images/501'. Do you see your image files there ? Or do you use one logical volume per machine ?
I think it could fool qmrestore...
I use one logical volume per (virtual) machine.

Did you try 'qm start 501' on the console. All you showed is a VNC screen copy, so perhaps you use the web interface to start the VM. On the console, the messages after the command qm can give you some hints.
Ok, I'm going to try to start the VM from the console. Fingers crossed!

But with such a special configuration, I fear you are on your own to debug. It is not supported, as it is not standard configuration...
Yes, I know but it was the only way to use Proxmox on my ISP's machine because the solution they offer doesn't work (also Proxmox 1.6 but the system hangs during the installation). However it seems to work flawlessy except for this small defect.

Also did you use the option 'snapshot' to bakckup your machine ? With such an option, you must have free space on your volume group to store the snapshot...
Yes, I set up a snapshot backup (I use LVM just for this matter). The virtual disk is 120GB and the LVM volume is 750GB: about 500GB are free to use. Moreover I guess it would show up in the backup log if there wasn't enough space.
 
yes, you're right, if you had not enough place for your snapshot, backup should have failed...

I think you are not far from succeeding. You were able to setup VMs, your backup succeeded, qmrestore don't work. I don't think 'qm start 501' will show anything interesting. From your screen copy, the BIOS cannot access the MBR on the image file.

You should verify the files you backuped are correct. If you use raw image files, you could try this :
-copy the backup archive somewhere, and untar it.
- Mount the raw files inside. For example, try something like this :
Code:
# mount -o loop,offset=32256 /path/to/file.raw /mount/point
cd to /mount/point, you should see the files inside the first partition of the image. This prove you have access to the MBR.

Then, you should verify the files that has been restored. It is not so direct to access image files on an LVM volume, but you can use the program kpartx (apt-get install kpartx) for this purpose. See for example :
http://www.centos.org/docs/5/html/Virtualization-en-US/ch-virt-accessing-data.html

Hopes that helps...
 
You should verify the files you backuped are correct.
It seems like the backup is corrupted. I tried to do as you said:

Code:
# mount -o loop,offset=32256 vm-disk-virtio0.raw /mnt/temp
mount: you must specify the filesystem type

Code:
# mount -o loop,offset=32256 -t ntfs vm-disk-virtio0.raw /mnt/temp
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

Code:
# dmesg | tail
NTFS-fs warning (device loop0): is_boot_sector_ntfs(): Invalid boot sector checksum.
NTFS-fs error (device loop0): read_ntfs_boot_sector(): Primary boot sector is invalid.
NTFS-fs error (device loop0): read_ntfs_boot_sector(): Mount option errors=recover not used. Aborting without trying to recover.
NTFS-fs error (device loop0): ntfs_fill_super(): Not an NTFS volume.

That's really a huge problem. If I cannot trust vzdump, disaster recovery (and I guess live migration) could really be a mess!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!