vzdump still fails in some cases without any error

[hk@]

Hi
we have the following problem:

Aug 18 06:07:03 INFO: Starting Backup of VM 68011 (openvz)
Aug 18 06:07:03 INFO: CTID 68011 exist mounted running
Aug 18 06:07:03 INFO: status = running
Aug 18 06:07:03 INFO: backup mode: snapshot
Aug 18 06:07:03 INFO: bandwidth limit: 5120 KB/s
Aug 18 06:07:03 INFO: ionice priority: 7
Aug 18 06:07:03 INFO: creating lvm snapshot of /dev/mapper/pve-data ('/dev/pve/vzsnap-k79-0')
Aug 18 06:07:03 INFO: Logical volume "vzsnap-k79-0" created
Aug 18 06:07:04 INFO: creating archive '/mnt/pve/kXXbackup/dump/vzdump-openvz-68011-2012_08_18-06_07_03.tar.lzo'
Aug 18 06:38:41 INFO: tar: ./var/kunden/logs/adserver-access.log: File shrank by 26209727 bytes; padding with zeros
Aug 18 06:38:46 INFO: tar: ./var/kunden/logs/superopas-access.log.4.gz: Read error at byte 0, while reading 2437 bytes: Input/output error

meaning at 06:38 the lvm snapshot has been filled up and therefore the snapshot was revoked - this goes on for minutes, until tar believes it has read all files:

Aug 18 06:43:38 INFO: tar: ./home/newsletter/.aptitude/config: Read error at byte 0, while reading 83 bytes: Input/output error
Aug 18 06:43:38 INFO: tar: ./home/ftp/welcome.msg: Read error at byte 0, while reading 166 bytes: Input/output error
Aug 18 06:43:38 INFO: Total bytes written: 11502090240 (11GiB, 5.1MiB/s)
Aug 18 06:43:38 INFO: tar: Exiting with failure status due to previous errors
Aug 18 06:50:48 INFO: archive file size: 3.15GB
Aug 18 06:50:48 INFO: delete old backup '/mnt/pve/kXXbackup/dump/vzdump-openvz-68011-2012_07_28-15_26_33.tar.lzo'
Aug 18 06:50:52 INFO: Finished Backup of VM 68011 (00:43:49)




the taskview of proxmox shows it quite similar:

INFO: tar: ./home/newsletter/.bashrc: Read error at byte 0, while reading 2227 bytes: Input/output error
INFO: tar: ./home/newsletter/.aptitude/config: Read error at byte 0, while reading 83 bytes: Input/output error
INFO: tar: ./home/ftp/welcome.msg: Read error at byte 0, while reading 166 bytes: Input/output error
INFO: Total bytes written: 11502090240 (11GiB, 5.1MiB/s)
INFO: tar: Exiting with failure status due to previous errors
INFO: archive file size: 3.15GB
INFO: delete old backup '/mnt/pve/kXXbackup/dump/vzdump-openvz-68011-2012_07_28-15_26_33.tar.lzo'
INFO: Finished Backup of VM 68011 (00:43:49)
INFO: Starting Backup of VM 68012 (openvz)
INFO: CTID 68012 exist mounted running
INFO: status = running
INFO: backup mode: snapshot
INFO: bandwidth limit: 5120 KB/s
INFO: ionice priority: 7

but the end of the taskview shows this:

...

INFO: archive file size: 10.49GB
INFO: delete old backup '/mnt/pve/kXXbackup/dump/vzdump-openvz-79005-2012_07_29-06_56_44.tar.lzo'
INFO: Finished Backup of VM 79005 (01:00:57)
INFO: Starting Backup of VM 79006 (openvz)
INFO: CTID 79006 exist mounted running
INFO: status = running
INFO: backup mode: snapshot
INFO: bandwidth limit: 5120 KB/s
INFO: ionice priority: 7
INFO: creating lvm snapshot of /dev/mapper/pve-data ('/dev/pve/vzsnap-k79-0')
INFO: Logical volume "vzsnap-k79-0" created
INFO: creating archive '/mnt/pve/kXXbackup/dump/vzdump-openvz-79006-2012_08_18-14_31_51.tar.lzo'
INFO: Total bytes written: 450877440 (430MiB, 5.2MiB/s)
INFO: archive file size: 244MB
INFO: Finished Backup of VM 79006 (00:01:50)
INFO: Backup job finished successfully
TASK OK

and here our problem starts: As we have to filter the backup reporting mails for "errors" to get notifications for problems, this is never shown as the report claims everything is ok:

e-mail-subject: vzdump backup status (kXX.vss.kapper.net) : backup successful

and specifically for the failed backup it reports "ok":

68011

cXX.vss.kapper.net

OK

00:43:49

3.15GB

/mnt/pve/kXXbackup/dump/vzdump-openvz-68011-2012_08_18-06_07_03.tar.lzo

and this is bad :-(

please advise how to get this error during tar raised to the next high levels...

regards
hk

 

[mir]

How much space is available here: INFO: creating lvm snapshot of /dev/mapper/pve-data ('/dev/pve/vzsnap-k79-0')
Maybe the snapshot simply run out of free space.
What does running the command pvs show?

 

[hk@]

mir - of course the snapshot is filled up - this is not my main problem

my main problem is, that tar reports to have failed, but at the end vzdump claims to have succeeded with the backup - which is plain wrong and which should never be reported by any backup, because there will be a time when we need a real backup not missing some or many or most files...

regards
hk

 

[mir]

mir - of course the snapshot is filled up - this is not my main problem

I do not mean your snapshot but your volume group containing pve-data. if the available space for creating the snapshot is lesser than the size of the snapshot you will end up with exactly 'Read error at byte 0, while reading 2437 bytes: Input/output error' because tar tries to read after the end of the snapshot.

 

[dietmar]

Form the git log I see:

commit 7abe98f1914213c60f90c3d52d49d43a0eb4225a
Author: Dietmar Maurer <dietmar@proxmox.com>
Date: Wed Mar 21 06:42:37 2012 +0100

vzdump: do not use tar flag --ignore-failed-read

This flag is quite dangerous. For example when LVM snapshot runs full...

So what version do you use exactly?

# pveversion -v