pct restore from pipe fails

May 31, 2015
19
0
21
Heyho!

I've setup up two no hosts in a new datacenter and tried to move LXCs from the old hosts via vzdump & ssh-pipe. This failed several times and I could break the problem down to `pct restore` from pipe:

Code:
root@host:~# vzdump 99999 --stdout | pct restore --storage ssd --rootfs 1 99991 -
ERROR: file '-' does not exist
trying to acquire lock...
 OK
INFO: starting new backup job: vzdump 99999
INFO: Starting Backup of VM 99999 (lxc)
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: CT Name: test
INFO: sending archive to stdout
ERROR: Backup of VM 99999 failed - command 'set -o pipefail && tar cpf - --totals --one-file-system -p --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-file-ignored' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/var/lib/vz/dump/vzdump-lxc-99999-2019_02_21-22_34_57.tmp' ./etc/vzdump/pct.conf '--directory=/mnt/vzsnap0' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' ./' failed: got signal 13
INFO: Backup job finished with errors
job errors

I assume the issue is already in line 2: ERROR: file '-' does not exist (which probably comes from `pct restore`).

vzdump seems to work, because if I do `vzdump 99999 --stdout | cat -` I get all the contents of the LXC on my terminal.

I'm happy about any idea to this issue :)

My pveversion (should all be the newest stable):
Code:
root@host:~# pveversion --verbose
proxmox-ve: 5.3-1 (running kernel: 4.15.18-10-pve)
pve-manager: 5.3-8 (running version: 5.3-8/2929af8e)
pve-kernel-4.15: 5.3-1
pve-kernel-4.15.18-10-pve: 4.15.18-32
pve-kernel-4.15.17-1-pve: 4.15.17-9
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-3
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-43
libpve-guest-common-perl: 2.0-19
libpve-http-server-perl: 2.0-11
libpve-storage-perl: 5.0-36
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-2
lxcfs: 3.0.2-2
novnc-pve: 1.0.0-2
proxmox-widget-toolkit: 1.0-22
pve-cluster: 5.0-33
pve-container: 2.0-33
pve-docs: 5.3-1
pve-edk2-firmware: 1.20181023-1
pve-firewall: 3.0-17
pve-firmware: 2.0-6
pve-ha-manager: 2.0-6
pve-i18n: 1.0-9
pve-libspice-server1: 0.14.1-1
pve-qemu-kvm: 2.12.1-1
pve-xtermjs: 3.10.1-1
qemu-server: 5.0-45
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.12-pve1~bpo1
 
Hi, your variables seem correct though the orde might matter.
Here's the command I used for container migration:
ssh 192.168.100.231 'vzdump 115 --mode stop --stdout' | pv | pct restore --rootfs 5 115 --storage local-zfs -

Hope it helps.
 
Hi mbosma,

thanks for your reply.

I tried it again with you command but again the same error:

Code:
root@host:~# ssh old_host 'vzdump 7701 --mode stop --stdout' | pv | pct restore --rootfs 4 7701 --storage ssd -
unable to restore CT 7701 - ERROR: file '-' does not exist
Use of uninitialized value $et in string eq at /usr/share/perl5/PVE/AccessControl.pm line 638.
Use of uninitialized value $et in string eq at /usr/share/perl5/PVE/AccessControl.pm line 677.
Use of uninitialized value $et in string eq at /usr/share/perl5/PVE/AccessControl.pm line 705.
Use of uninitialized value $et in string eq at /usr/share/perl5/PVE/AccessControl.pm line 724.
Use of uninitialized value $et in string eq at /usr/share/perl5/PVE/AccessControl.pm line 757.
user config - ignore config line: :
INFO: starting new backup job: vzdump 7701 --mode stop
INFO: Starting Backup of VM 7701 (lxc)
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: CT Name: ffdnproxy
INFO: sending archive to stdout
   0 B 0:00:00 [   0 B/s] [<=>                                                                                        ]
ERROR: Backup of VM 7701 failed - command 'set -o pipefail && tar cpf - --totals --one-file-system -p --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-file-ignored' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/var/lib/vz/dump/vzdump-lxc-7701-2019_02_22-10_11_36.tmp' ./etc/vzdump/pct.conf ./etc/vzdump/pct.fw '--directory=/mnt/vzsnap0' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' ./' failed: got signal 13
INFO: Backup job finished with errors
job errors
 
At least I found the source of the `Use of uninitialized value $et in string eq`-isse: in `/etc/pve/user.cfg` there was one line just with a ":" and nothing else. I removed that line and these five error messages are gone, but the pct restore error still exists...
 
I've setup up two no hosts in a new datacenter and tried to move LXCs from the old hosts via vzdump & ssh-pipe. This failed several times and I could break the problem down to `pct restore` from pipe:

Why don't you use scp to transfer the dumps and then use pct restore on the new host instead? Is there a space problem?
 
Why don't you use scp to transfer the dumps and then use pct restore on the new host instead? Is there a space problem?

Yeah there are actually two reasons:
1. The space problem: All disks in the server (except the one with the Proxmox OS) are ZFS formatted, so they can't handle dump files and the backups are done to an external NFS storage which is noch accessible from the new host (done by a different company, it's complicated...)
2. Some of the containers (we're taking about 50-100) are quite critical when it's about uptime and I thought vzdump & restore over SSH pipe (maybe compressed) is the fastest way to get it back online on the new host without having different versions of the filesystems).
 
It seems to be tied to the latest update.
Even piping it on the host locally spits out an error.
Another host wich is on 5.3-5 works fine, but my prod host on 5.3-8 gave me the same error you have.
 
Mbosma, thanks for trying! My next step would have been a reinstall of the host, so I can skip that.
@ Proxmox-Staff: can you confirm this is a bug?
 
No problem, I use this command from somewhat regularly so getting this issue fixed also benefits me (and others).
 
Okay, I can confirm:
Downgrading pve-container to 2.0-31 helps (2.0-33 has the same issue and 2.0-32 was somehow unavailable on my system).
This seems to work without any dependency problems.

Should I create an issue on bugzilla.proxmox.com or is this done by anyone else?
 
Okay, I can confirm:
Downgrading pve-container to 2.0-31 helps (2.0-33 has the same issue and 2.0-32 was somehow unavailable on my system).
This seems to work without any dependency problems.

Should I create an issue on bugzilla.proxmox.com or is this done by anyone else?


Please create a report on bugzilla. Try to include as much as info possible (commands you used, errors you got, if the errors were different when using different versions of pve-container etc.)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!