backup fails

mir

Famous Member
Apr 14, 2012
3,585
142
133
Copenhagen, Denmark
Hi all,

Can anybody explain what is going on here?

​INFO: starting new backup job: vzdump 112 --remove 0 --mode snapshot --compress lzo --storage qnap_nfs --node esx2INFO: Starting Backup of VM 112 (openvz)
INFO: CTID 112 exist mounted running
INFO: status = running
INFO: mode failure - unable to detect lvm volume group
INFO: trying 'suspend' mode instead
INFO: backup mode: suspend
INFO: ionice priority: 7
INFO: starting first sync /mnt/pve/qnap_nfs/private/112/ to /mnt/pve/qnap_nfs/dump/vzdump-openvz-112-2013_02_10-00_58_36.tmp
INFO: Number of files: 36194
INFO: Number of files transferred: 28730
INFO: Total file size: 1403709608 bytes
INFO: Total transferred file size: 1400286737 bytes
INFO: Literal data: 1400286737 bytes
INFO: Matched data: 0 bytes
INFO: File list size: 838223
INFO: File list generation time: 0.002 seconds
INFO: File list transfer time: 0.000 seconds
INFO: Total bytes sent: 1402554767
INFO: Total bytes received: 587082
INFO: sent 1402554767 bytes received 587082 bytes 7033292.48 bytes/sec
INFO: total size is 1403709608 speedup is 1.00
INFO: first sync finished (199 seconds)
INFO: suspend vm
INFO: Setting up checkpoint...
INFO: suspend...
INFO: get context...
INFO: Checkpointing completed successfully
INFO: starting final sync /mnt/pve/qnap_nfs/private/112/ to /mnt/pve/qnap_nfs/dump/vzdump-openvz-112-2013_02_10-00_58_36.tmp
INFO: Number of files: 36196
INFO: Number of files transferred: 3
INFO: Total file size: 1403709949 bytes
INFO: Total transferred file size: 291435 bytes
INFO: Literal data: 935 bytes
INFO: Matched data: 290500 bytes
INFO: File list size: 838295
INFO: File list generation time: 0.008 seconds
INFO: File list transfer time: 0.000 seconds
INFO: Total bytes sent: 843665
INFO: Total bytes received: 5202
INFO: sent 843665 bytes received 5202 bytes 54765.61 bytes/sec
INFO: total size is 1403709949 speedup is 1653.63
INFO: final sync finished (15 seconds)
INFO: resume vm
INFO: Resuming...
INFO: vm is online again after 15 seconds
INFO: creating archive '/mnt/pve/qnap_nfs/dump/vzdump-openvz-112-2013_02_10-00_58_36.tar.lzo'
INFO: tar: ./usr/lib/python2.6/distutils/unixccompiler.pyc: file changed as we read it
INFO: tar: ./usr/lib/python2.6/encodings/mac_croatian.py: file changed as we read it
INFO: tar: ./etc/network: file changed as we read it
INFO: tar: ./var/local: file changed as we read it
INFO: tar: ./var/www: file changed as we read it
INFO: tar: ./var/log/fsck: file changed as we read it
INFO: Total bytes written: 1426298880 (1.4GiB, 16MiB/s)
ERROR: Backup of VM 112 failed - command '(cd /mnt/pve/qnap_nfs/dump/vzdump-openvz-112-2013_02_10-00_58_36.tmp;find . '(' -regex '^\.$' ')' -o '(' -type 's' -prune ')' -o -print0|sed 's/\\/\\\\/g'|tar cpf - --totals --sparse --numeric-owner --no-recursion --one-file-system --null -T -|lzop) >/mnt/pve/qnap_nfs/dump/vzdump-openvz-112-2013_02_10-00_58_36.tar.dat' failed: exit code 1
INFO: Backup job finished with errors
TASK ERROR: job errors
 
ERROR: Backup of VM 112 failed - command '(cd /mnt/pve/qnap_nfs/dump/vzdump-openvz-112-2013_02_10-00_58_36.tmp;find . '(' -regex '^\.$' ')' -o '(' -type 's' -prune ')' -o -print0|sed 's/\\/\\\\/g'|tar cpf - --totals --sparse --numeric-owner --no-recursion --one-file-system --null -T -|lzop) >/mnt/pve/qnap_nfs/dump/vzdump-openvz-112-2013_02_10-00_58_36.tar.dat' failed: exit code 1
INFO: Backup job finished with errors
TASK ERROR: job errors

Oh, I see the error now. The output redirect to "/mnt/pve/qnap_nfs" looks strange to me. Please can you post your storage config?
 
Plenty:
192.168.2.10:/vz 1,4T 891G 483G 65% /mnt/pve/qnap_nfs

I have even tried with a backup of the CT using mode stop. It also fails with almost identical error:

INFO: starting new backup job: vzdump 112 --remove 0 --mode stop --compress lzo --storage qnap_nfs --node esx1
INFO: Starting Backup of VM 112 (openvz)
INFO: CTID 112 exist mounted running
INFO: status = running
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: stopping vm
INFO: Stopping container ...
INFO: Container was stopped
INFO: Container is unmounted
INFO: creating archive '/mnt/pve/qnap_nfs/dump/vzdump-openvz-112-2013_02_10-01_37_06.tar.lzo'
INFO: tar: ./lib/xtables/libxt_tos.so: file changed as we read it
INFO: tar: ./lib/xtables/libxt_NOTRACK.so: file changed as we read it
INFO: tar: ./lib/xtables/libip6t_HL.so: file changed as we read it
INFO: Total bytes written: 1425571840 (1.4GiB, 23MiB/s)
INFO: restarting vm
INFO: Starting container ...
INFO: Container is mounted
INFO: Setting CPU units: 1000
INFO: Setting CPUs: 2
INFO: Configure veth devices: veth112.0
INFO: Adding interface veth112.0 to bridge vmbr1 on CT0 for CT112
INFO: Container start in progress...
INFO: vm is online again after 76 seconds
ERROR: Backup of VM 112 failed - command '(cd /mnt/pve/qnap_nfs/private/112;find . '(' -regex '^\.$' ')' -o '(' -type 's' -prune ')' -o -print0|sed 's/\\/\\\\/g'|tar cpf - --totals --sparse --numeric-owner --no-recursion --one-file-system --null -T -|lzop) >/mnt/pve/qnap_nfs/dump/vzdump-openvz-112-2013_02_10-01_37_06.tar.dat' failed: exit code 1
INFO: Backup job finished with errors
TASK ERROR: job errors
 
Some background:

CT is running Debian 6.04 x86 template from proxmox upgraded to 6.06 and apart from the template the only thing installed is an owncloud server.

cat /etc/pve/openvz/112.conf

ONBOOT="yes"


PHYSPAGES="0:512M"
SWAPPAGES="0:512M"
KMEMSIZE="232M:256M"
DCACHESIZE="116M:128M"
LOCKEDPAGES="256M"
PRIVVMPAGES="unlimited"
SHMPAGES="unlimited"
NUMPROC="unlimited"
VMGUARPAGES="0:unlimited"
OOMGUARPAGES="0:unlimited"
NUMTCPSOCK="unlimited"
NUMFLOCK="unlimited"
NUMPTY="unlimited"
NUMSIGINFO="unlimited"
TCPSNDBUF="unlimited"
TCPRCVBUF="unlimited"
OTHERSOCKBUF="unlimited"
DGRAMRCVBUF="unlimited"
NUMOTHERSOCK="unlimited"
NUMFILE="unlimited"
NUMIPTENT="unlimited"


# Disk quota parameters (in form of softlimit:hardlimit)
DISKSPACE="30G:33G"
DISKINODES="6000000:6600000"
QUOTATIME="0"
QUOTAUGIDLIMIT="0"


# CPU fair scheduler parameter
CPUUNITS="1000"
CPUS="2"
HOSTNAME="owncloud.datanom.net"
SEARCHDOMAIN="datanom.net"
NAMESERVER="172.16.1.201 89.233.43.71"
NETIF="ifname=eth0,mac=16:80:0E:73:F6:EB,host_ifname=veth112.0,host_mac=A6:A1:05:69:43:48,bridge=vmbr1"
VE_ROOT="/var/lib/vz/root/$VEID"
VE_PRIVATE="/mnt/pve/qnap_nfs/private/112"
OSTEMPLATE="debian-6.0-standard_6.0-4_i386.tar.gz"
DESCRIPTION="eth0%3A 172.16.1.6"
 
What is the output of

# pveversion -v
pveversion -v
pve-manager: 2.2-32 (pve-manager/2.2/3089a616)
running kernel: 2.6.32-17-pve
proxmox-ve-2.6.32: 2.2-83
pve-kernel-2.6.32-17-pve: 2.6.32-83
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.4-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.93-2
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.9-1
pve-cluster: 1.0-34
qemu-server: 2.0-72
pve-firmware: 1.0-21
libpve-common-perl: 1.0-41
libpve-access-control: 1.0-25
libpve-storage-perl: 2.0-36
vncterm: 1.0-3
vzctl: 4.0-1pve2
vzprocps: 2.0.11-2
vzquota: 3.1-1
pve-qemu-kvm: 1.3-10
ksm-control-daemon: 1.1-1
 
Do you get the same error if you use no compression (or gzip)?
Very strange. Today it works. Only difference between today and yesterday is that I have taking this CT out of HA? Another hint: Another CT which was not under HA had no problems with backup when the backup of this CT failed which was under HA.