vzdump backslash problem

kobuki

Well-Known Member
Dec 30, 2008
470
25
48
vzdump backslash problem causing skipped files

Hi. I've found a peculiar problem using vzdump that can cause skipping of backup files. Here's an earlier backup log displaying the problem:

Code:
Sep 03 23:44:18 INFO: Starting Backup of VM 103 (openvz)
Sep 03 23:44:18 INFO: CTID 103 exist mounted running
Sep 03 23:44:18 INFO: status = CTID 103 exist mounted running
Sep 03 23:44:18 INFO: backup mode: snapshot
Sep 03 23:44:18 INFO: bandwidth limit: 10240 KB/s
Sep 03 23:44:18 INFO: creating lvm snapshot of /dev/mapper/pve-data ('/dev/pve/vzsnap-vmaster-0')
Sep 03 23:44:18 INFO:   Logical volume "vzsnap-vmaster-0" created
Sep 03 23:44:19 INFO: creating archive '/backup/vzdump/vzdump-openvz-103-2010_09_03-23_44_18.tgz'
Sep 03 23:45:52 INFO: tar: ./home/user/testapp/log/test/what\a\bot.log: Warning: Cannot stat: No such file or directory
Sep 03 23:47:31 INFO: tar: ./home/user/test/what\a\bot.log: Warning: Cannot stat: No such file or directory
Sep 03 23:47:31 INFO: tar: ./home/user/test/this\\is\a\file: Warning: Cannot stat: No such file or directory
Sep 03 23:58:51 INFO: Total bytes written: 10807992320 (11GiB, 12MiB/s)
Sep 03 23:58:51 INFO: archive file size: 8.04GB
Sep 03 23:58:51 INFO: delete old backup '/backup/vzdump/vzdump-openvz-103-2010_07_28-17_51_20.tgz'
Sep 03 23:59:01 INFO:   Logical volume "vzsnap-vmaster-0" successfully removed
Sep 03 23:59:01 INFO: Finished Backup of VM 103 (00:14:43)
And here's how the files can be seen inside the VE with ls:

Code:
$ ls -la /home/user/test/
total 12
drwxr-xr-x  2 user user 4096 2010-01-12 01:09 .
drwxr-x--x 13 user user 4096 2010-09-22 11:26 ..
-rw-r--r--  1 user user    0 2010-01-12 00:31 a?¦a??a?¨
-rw-r--r--  1 user user    0 2010-01-12 00:32 this\is\a\file
-rw-r--r--  1 user user 1129 2010-01-12 01:09 what\a\bot.log

$ cd /home/user/testapp/log/test/
$ ls -la what*
-rw-rw-rw- 1 user user 1129 2007-06-18 22:46 what\a\bot.log
Could you please help with addressing this? All I can see is, when backslashes are used in the file name, the above error message is seen and the source files are skipped. I've verified they're not in the archive. On another note, I have some files with Japanese characters in their names, they are correctly backed up (that filename with question marks is an example).

Some version info:

Code:
pve-manager: 1.5-10 (pve-manager/1.5/4822)
running kernel: 2.6.24-11-pve
proxmox-ve-2.6.24: 1.5-23
pve-kernel-2.6.24-11-pve: 2.6.24-23
pve-kernel-2.6.24-8-pve: 2.6.24-16
qemu-server: 1.1-16
pve-firmware: 1.0-5
libpve-storage-perl: 1.0-13
vncterm: 0.9-2
vzctl: 3.0.23-1pve11
vzdump: 1.2-5
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.12.4-1
 
Last edited:

kobuki

Well-Known Member
Dec 30, 2008
470
25
48
Re: vzdump backslash problem causing skipped files

I guess no one experienced similar behaviour? In fact, this bug was present in all vzdump versions since 1.2. I was hoping for a fix in later versions but still none. The real problem is that files that should be backed up will always be skipped and left out from the backup. I guess this makes the problem a little more serious than one might first think. Even for simple, quick VE backups we'll need to look for better alternatives than vzdump, which is a shame.
 

tom

Proxmox Staff Member
Staff member
Aug 29, 2006
15,420
856
163
Re: vzdump backslash problem causing skipped files

how can I reproduce this?
 

kobuki

Well-Known Member
Dec 30, 2008
470
25
48
Re: vzdump backslash problem causing skipped files

how can I reproduce this?

Thanks for paying attention to this. Reproduction is very simple: create some files in an OpenVZ VE with '\' (backslash) in their name (see log in my first post) and start a backup process similar to this:

vzdump --compress --dumpdir /backup/vzdump/ --snapshot 103
 

tom

Proxmox Staff Member
Staff member
Aug 29, 2006
15,420
856
163
Re: vzdump backslash problem causing skipped files

confirmed, will dig deeper.
 

kobuki

Well-Known Member
Dec 30, 2008
470
25
48
Re: vzdump backslash problem causing skipped files

Hi Tom, have you found any time for investigating this? From what I could gather, the problem is caused by the fact that the list of files to be backed up is gathered with the unix find command, then fed to tar with the -T option and its output is not escaped for special characters, causing tar to break on file names containing them and thus skipping files. I haven't found any standardised approach to remedy this, most probably the solution will involve writing the escaping in the Perl module wrapping the backup commands. I'm not very well versed in Perl so I probably can't help with making a patch, unfortunately. Another solution might be piping the output of find through sed and do some substitution to do the required escaping.
 

tom

Proxmox Staff Member
Staff member
Aug 29, 2006
15,420
856
163
Re: vzdump backslash problem causing skipped files

thanks for sharing your findings, one of our dev´s will take a look end of this week.
 

kobuki

Well-Known Member
Dec 30, 2008
470
25
48
Re: vzdump backslash problem causing skipped files

Another quick note: bsdtar seems to have no problems using the (unescaped) output of find, and additionally it stores ACL and EA attributes when present on the filesystem, which is certainly useful. Using bsdtar might cause dependency issues, but it seems to be a robust replacement for the GNU tar currently in use. There is no support for sparse files tho.
 
Last edited:

dietmar

Proxmox Staff Member
Staff member
Apr 28, 2005
17,078
490
103
Austria
www.proxmox.com
Re: vzdump backslash problem causing skipped files

Using bsdtar might cause dependency issues, but it seems to be a robust replacement for the GNU tar currently in use. There is no support for sparse files tho.

But we really need support for sparse files.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!