Restoring from backup results in "got timeout"

colourreich

Active Member
May 11, 2013
4
3
43
I installed Proxmox from the ISO file proxmox-ve_3.0-7ce622f2-9.iso yesterday, and have been testing Windows 7 under KVM using the VirtIO Windows drivers (network, disk, balloon).
After an unexplained lock up of a virtual machine, I decided to back it up to an external USB hard drive. The backup completed successfully, but trying to restore it results in the following error:

Code:
[COLOR=#000000][FONT=tahoma]restore vma archive: zcat /usb1/backup/dump/vzdump-qemu-104-2013_05_11-11_47_45.vma.gz|vma extract -v -r /var/tmp/vzdumptmp25361.fifo - /var/tmp/vzdumptmp25361[/FONT][/COLOR][COLOR=#000000][FONT=tahoma]
[/FONT][/COLOR][COLOR=#000000][FONT=tahoma]TASK ERROR: command 'zcat /usb1/backup/dump/vzdump-qemu-104-2013_05_11-11_47_45.vma.gz|vma extract -v -r /var/tmp/vzdumptmp25361.fifo - /var/tmp/vzdumptmp25361' failed: got timeout
[/FONT][/COLOR]

The USB device to which the backup was saved is mounted using ntfs-3g. Running only the zcat /usb1/*/*/*.vma.gz command from the terminal completes without reporting any errors. df reports 780GB+ free on /var/lib/vz, 9GB+ free on /usb1, and 93GB+ free on / (root).
 
Last edited:
Hi Dietmar,

Have this really been fixed? I just deployed a proxmox VE 3.0 recently and have done aptitude -y update && aptitude -y full-upgrade . However, during the test, I do encounter the "command vma extract failed: got timeout" issue. I have 3 set of proxmox VE 3.0 and all faced the same issue.

Please advise.
 
However, during the test, I do encounter the "command vma extract failed: got timeout" issue. I have 3 set of proxmox VE 3.0 and all faced the same issue.

Please can you provide detailed instructions how to reproduce that (restore works for me).
 
Hi Dietmar,

1. We have a custom template. Example the template name is CentOS6.
2. use below cmd to restore via the proxmox host shell:

qmrestore --storage [storage location] [Custome template] [VM ID]

eg : qmrestore --storage local CentOS6 101

3. Will get below error:

command 'vma extract -v -r /var/tmp/vzdumptmp72248.fifo /CentOS6 /var/tmp/vzdumptmp72248' failed: got timeout


Note:
- The same template and the same method has been used in older proxmox version where it is working fine.
 
pve-manager: 3.0-23 (pve-manager/3.0/957f0862)
running kernel: 2.6.32-20-pve
proxmox-ve-2.6.32: 3.0-100
pve-kernel-2.6.32-20-pve: 2.6.32-100
lvm2: 2.02.95-pve3
clvm: 2.02.95-pve3
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-1
pve-cluster: 3.0-4
qemu-server: 3.0-20
pve-firmware: 1.0-22
libpve-common-perl: 3.0-4
libpve-access-control: 3.0-4
libpve-storage-perl: 3.0-8
vncterm: 1.1-4
vzctl: 4.0-1pve3
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.4-13
ksm-control-daemon: 1.1-1
 
older versions does not support VM templates.

are you sure you do not VM and containers here?
 
Apologize, apparently the term that i used here was incorrect. I didn't use template provided by proxmox. I used the clone image of the pre-configured OS.
 
Hi,

As to rectify this issue, I have setup proxmox 2.2 and proxmox 3.0 in 2 different servers. Below are my findings:

Server 1 (Proxmox 2.2)
1. Before perform aptitude -y update && aptitude -y full-upgrade
- cmd : qmrestore --storage local CentOS6 101
- able to restore the VM & WORKING FINE

2. After perform aptitude -y update && aptitude -y full-upgrade
- cmd : qmrestore --storage local CentOS6 102
- Unable to restore the VM with error of "vma extract -v -r xxxx failed: got timeout" being shown

Server 2 (Proxmox 3.0)
1. Before perform aptitude -y update && aptitude -y full-upgrade
- cmd : qmrestore --storage local CentOS6 101
- Unable to restore the VM with error of "vma extract -v -r xxxx failed: got timeout" being shown

2. After perform aptitude -y update && aptitude -y full-upgrade
- Didn't do it as the same issue already happen prior to update & upgrade

In different issue, when restoring the .lzo via proxmox GUI. It is working for both 2.2 and 3.0. However, I didn't test on restoring the .lzo via cmd.

Besides that, the used clone image of the pre-configured OS (CentOS6) has been used on the proxmox older version (2.1 & 2.2) on several set of servers. Thus, this issue has nothing to do with the cloned image. The only different is I didn't do the update & upgrade for the old servers. I believe if I do it, the same issue will happen. But I didn't dare to take that risk. Enough with above testing.

From there, I confirmed there was a bug in the latest respository. Possible on the restoring module which is using qmrestore cmd. Hope it will help you guys on rectifiying this issue.
 
I run into the same problem, trying to run:
qmrestore 112.vma 112
and got timeout Problem, heres the answer line:
command 'vma extract -v -r /var/tmp/vzdumptmp85307.fifo /backup/vm/112.vma /var/tmp/vzdumptmp85307' failed: got timeout

I found a solution (don't laugh). I renamed the file in 112.vma.lzo (although I set up no compression for the backupfile) and voila it worked.
mv 112.vma 112.vma.lzo
qmrestore 112.vma.lzo 112

restore vma archive: lzop -d -c /backup/vm/112.vma.lzo|vma extract -v -r /var/tmp/vzdumptmp85392.fifo - /var/tmp/vzdumptmp85392
CFG: size: 323 name: qemu-server.conf
DEV: dev_id=1 size: 10737418240 devname: drive-ide0
CTIME: Tue Jul 30 14:04:43 2013
...

So either when no compression is selected it is compressed with lzo or qmrestore can't properly handle uncompressed files. Here's my pveversion command output.

pve-manager: 3.0-23 (pve-manager/3.0/957f0862)
running kernel: 2.6.32-20-pve
proxmox-ve-2.6.32: 3.0-100
pve-kernel-2.6.32-20-pve: 2.6.32-100
lvm2: 2.02.95-pve3
clvm: 2.02.95-pve3
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-1
pve-cluster: 3.0-4
qemu-server: 3.0-20
pve-firmware: 1.0-22
libpve-common-perl: 3.0-4
libpve-access-control: 3.0-4
libpve-storage-perl: 3.0-8
vncterm: 1.1-4
vzctl: 4.0-1pve3
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.4-13
ksm-control-daemon: 1.1-1

Regards
Martin
 
command 'vma extract -v -r /var/tmp/vzdumptmp85307.fifo /backup/vm/112.vma /var/tmp/vzdumptmp85307' failed: got timeout

Strange, I simply cant reproduce that here. How long does it take until you get that timeout?
 
Strange, I simply cant reproduce that here. How long does it take until you get that timeout?
About 2-3 sec after start. The backup is done automatically with this command:

vzdump 101 ... --quiet 1 --mailto ... --mode snapshot --storage backup

After that I use the vzhook Script (which btw. is not delivered as described here http://pve.proxmox.com/wiki/Vzdump_manual. I got it from an ubuntu src package) which copys the backup to a backupserver and renames it. Thats all I do with that file.

Regards
Martin
 
Do you restore to a existing VM?
The VM 112 exists on an other node and is stopped. With the rename trick the restore process starts and claims at the end something like can't create vm 112 but the image is extracted. Shall I try "qmrestore 112.vma 113" and see what happens?

Regards
Martin
 
Please do - we need to find out how to reproduce this bug.
Ok lets do some test step by step.
Backup Command via Web:
> vzdump 112 --remove 0 --mode snapshot --storage backup --node vm2

after that file is renamed to 112.vma.lzo and pushed to backup server.
I try to restore this vm 112 on vm server "vm1" to new vmid 115:
> qmrestore 112.vma.lzo 115
restore vma archive: lzop -d -c /backup/vm/112.vma.lzo|vma extract -v -r /var/tmp/vzdumptmp155396.fifo - /var/tmp/vzdumptmp155396
command 'lzop -d -c /backup/vm/112.vma.lzo|vma extract -v -r /var/tmp/vzdumptmp155396.fifo - /var/tmp/vzdumptmp155396' failed: got timeout
This took about 7 secs.

Lets try this with the renamed file
>mv 112.vma.lzo 112.vma
>qmrestore 112.vma 115
restore vma archive: vma extract -v -r /var/tmp/vzdumptmp155575.fifo /backup/vm/112.vma /var/tmp/vzdumptmp155575
CFG: size: 323 name: qemu-server.conf
DEV: dev_id=1 size: 10737418240 devname: drive-ide0
CTIME: Thu Aug 1 11:35:49 2013
Formatting '/var/lib/vz/images/115/vm-115-disk-1.qcow2', fmt=qcow2 size=10737418240 encryption=off cluster_size=65536 preallocation='metadata' lazy_refcounts=off
new volume ID is 'local:115/vm-115-disk-1.qcow2'
map 'drive-ide0' to '/var/lib/vz/images/115/vm-115-disk-1.qcow2' (write zeros = 0)
progress 1% (read 107413504 bytes, duration 0 sec)
...
It works.

New try. New Backup via Webfrontend with compression, here the command:
vzdump 112 --remove 0 --mode snapshot --compress lzo --storage backup --node vm2

Restore try:
>qmrestore 112.vma.lzo 115
restore vma archive: lzop -d -c /backup/vm/112.vma.lzo|vma extract -v -r /var/tmp/vzdumptmp155653.fifo - /var/tmp/vzdumptmp155653
CFG: size: 323 name: qemu-server.conf
DEV: dev_id=1 size: 10737418240 devname: drive-ide0
CTIME: Thu Aug 1 11:47:04 2013
Formatting '/var/lib/vz/images/115/vm-115-disk-1.qcow2', fmt=qcow2 size=10737418240 encryption=off cluster_size=65536 preallocation='metadata' lazy_refcounts=off
new volume ID is 'local:115/vm-115-disk-1.qcow2'
map 'drive-ide0' to '/var/lib/vz/images/115/vm-115-disk-1.qcow2' (write zeros = 0)
progress 1% (read 107413504 bytes, duration 0 sec)
...

Rename & Restore Try:
>mv 112.vma.lzo 112.vma
>qmrestore 112.vma 115
restore vma archive: vma extract -v -r /var/tmp/vzdumptmp155701.fifo /backup/vm/112.vma /var/tmp/vzdumptmp155701
command 'vma extract -v -r /var/tmp/vzdumptmp155701.fifo /backup/vm/112.vma /var/tmp/vzdumptmp155701' failed: got timeout
Again after 7-8 secs.

This is just what it should be and everything is fine. I have no clue why it works now. I think case closed and if someone bumps here he/she can try it with renaming.

Thx for changing the wiki entry.

Regards Martin
 
Sorry, but you are doing strange things! The file is lzo compressed, and you remove the '.lzo' extension (or the other way around)?

after that file is renamed to 112.vma.lzo and pushed to backup server.
I try to restore this vm 112 on vm server "vm1" to new vmid 115:
> qmrestore 112.vma.lzo 115
restore vma archive: lzop -d -c /backup/vm/112.vma.lzo|vma extract -v -r /var/tmp/vzdumptmp155396.fifo - /var/tmp/vzdumptmp155396
command 'lzop -d -c /backup/vm/112.vma.lzo|vma extract -v -r /var/tmp/vzdumptmp155396.fifo - /var/tmp/vzdumptmp155396' failed: got timeout

Quite obvious, because the file is not correctly decompressed (yes, the error message could be better).
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!