VZDump exlude-path question

[binx]

First of all. It looks like this --exclude-path '/var/log/.+' excludes files in /var/log but doesn't exclude dirs.

Second. When doing a vzdump, it looks like the excludes arent really excluded until the tar is run (i.e rsync still transfers ALL data from a vps regardless of --exclude-path settings). Isn't this a bit unnecessary and also adds to the time it takes to do a backup?

To me, the reasonable thing would be to exclude these dirs/files completely from the transfer...


Thanks for a great app!

/John

 

[binx]

Damn, I guess this should've been posted in another Forum such as "Proxmox VE: Installation and Configuration". Sorry about that.

 

[dietmar]

Thanks for the bug report (added it to our bug tracker).

- Dietmar

 

[valshare]

Hello Dietmar,

is the bug fixed? I have played today with this option, but rsync didnt exclude the path as example "--exclude-path "/home/.+".

proxmox01:~# vzdump --compress --dumpdir /var/lib/vz/dump/ --exclude-path "/home/.+" 110
INFO: starting new backup job: vzdump --compress --dumpdir /var/lib/vz/dump/ --exclude-path /home/.+ 110
INFO: Starting Backup of VM 110 (openvz)
INFO: CTID 110 exist mounted running
INFO: status = CTID 110 exist mounted running
INFO: mode failure - unable to dump into snapshot (use option --dumpdir)
INFO: trying 'suspend' mode instead
INFO: backup mode: suspend
INFO: bandwidth limit: 10240 KB/s
INFO: starting first sync /var/lib/vz/private/110/ to /var/lib/vz/dump/vzdump-openvz-110-2009_10_25-13_56_42.tmp

proxmox01:/var/lib/vz/dump/vzdump-openvz-110-2009_10_25-13_56_42.tmp/home# du -sh samba/
3.4G    samba/

proxmox01:/var/lib/vz/dump# ls -lsa vzdump-openvz-110-2009_10_25-13_56_42.tmp/home/
total 16
4 drwxr-xr-x  4 root root 4096 Jul 27 21:08 .
4 drwxr-xr-x 22 root root 4096 Oct 24 20:46 ..
4 drwxr-xr-x  4 root root 4096 Sep 25 14:22 samba
proxmox01:/var/lib/vz/dump#

proxmox01:/var/lib/vz/dump# pveversion -v
pve-manager: 1.4-9 (pve-manager/1.4/4390)
qemu-server: 1.1-8
pve-kernel: 2.6.24-16
pve-qemu-kvm: 0.11.0-2
pve-firmware: 1
vncterm: 0.9-2
vzctl: 3.0.23-1pve3
vzdump: 1.2-5
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1

when the archive will be tar´ed, the directory will be removed. But isn´t there a way, that rsync will skipped the transfer from the exclude-path directly?

How can i setup the exclude-path for each openvz container in the automated backup? For example, i will exclude the directory /home for VID 1 but not for VID 2. For VID 2 i will exclude the /opt but not for VID 1. How can i setup this in the scheduled backup in proxmox?

Regards, Valle

 

[dietmar]

when the archive will be tar´ed, the directory will be removed. But isn´t there a way, that rsync will skipped the transfer from the exclude-path directly?

no, currently not.

How can i setup the exclude-path for each openvz container in the automated backup? For example, i will exclude the directory /home for VID 1 but not for VID 2. For VID 2 i will exclude the /opt but not for VID 1. How can i setup this in the scheduled backup in proxmox?

Write your own crontab, or manually edit /etc/cron.d/vzdump.

 

[teimahu]

Re: VZDump exlude-path question [WORKAROUND]

There is a workaround proposed on OpenVZ forum:
http://forum.openvz.org/index.php?t=msg&goto=36924&

User "tatawaki" suggest to modify the vzdump perl script (/usr/sbin/vzdump) :

[URL="http://forum.openvz.org/index.php?t=usrinfo&id=3186&"]tatawaki[/URL]
[B]Messages:[/B] 3
[B]Registered:[/B] December 2008  Junior Member
[B]From:[/B] *sbm.shawcable.net
ok, here's the work around:

my $rsyncopts = "--stats --numeric-ids --bwlimit=${opt_bwlimit}";

# changes
$rsyncopts = $rsyncopts." --exclude-from=/home/backups/exclude_vzdump.txt";

the txt file contains <VEID>/home/:

1094/home/

I've tested it successfully, you juste have to modify your script at line 694 and to create the file with excluded paths.
And you can still use the --all feature to backup all your VE's at the same time with a line by folder for each VE's in the same file:

500/home/
501/home/
502/home/
503/home/
510/home/

 

[cadiolis]

Is there any way this fix can be applied to the Proxmox VZDump (which seems to be re-written in perl). I'm thinking it would be useful to simply have a file on the drive which (if it exists) will be used as the exclude-from option.

Say... /etc/vz/exclude.txt which the perl VZDump checks and if it exists, would use it as the argument when doing the dump

 

[dietmar]

Say... /etc/vz/exclude.txt which the perl VZDump checks and if it exists, would use it as the argument when doing the dump

We already have a config file called /etc/vzdump.conf - I guess we could add the exclude option there. But I currently have no time to work on that. But feel free to send a patch.

 

[tito]

I have a bare metal install of pve (pveversion ->
pve-manager/1.5/4627) but i can't find the config file called /etc/vzdump.conf
Why? Is it supposed to be installed separately? Do i have to create it from scratch?

 

[tom]

I have a bare metal install of pve (pveversion ->
pve-manager/1.5/4627) but i can't find the config file called /etc/vzdump.conf
Why? Is it supposed to be installed separately? Do i have to create it from scratch?

yes, just create it:

nano /etc/vzdump.conf

 

[tito]

Where can i find a vzdum.conf example?
Thanks

 

[dietmar]

Where can i find a vzdum.conf example?
Thanks

See 'man vzdump'

 

[safar]

We already have a config file called /etc/vzdump.conf - I guess we could add the exclude option there. But I currently have no time to work on that. But feel free to send a patch.

Here is my not elegant but working patch (not so easy for non-perl programmer) :
on file /usr/share/perl5/PVE/VZDump/OpenVZ.pm
After the line

[FONT=Helvetica]my $rsyncopts = "--stats -x --numeric-ids";
[/FONT]

add

if ($opts->{'exclude-path'}){

[FONT=Helvetica]        foreach my $path (@{$opts->{'exclude-path'}}){[/FONT]
[FONT=Helvetica]            $rsyncopts .= " --exclude=$path";[/FONT]
[FONT=Helvetica]        }[/FONT]
[FONT=Helvetica]    }[/FONT]

This works. Files are excluded at rsync stage, not later on at tar stage. But I then had a problem where tar interprets exclude pattern as regex, and exclude some more (for instance tar with regex and rsync act differently on '*/home/*/*', tar with regex excludes much more), so in the end I also edited
/usr/share/perl5/PVE/VZDump.pm ; new code is

    if ($opts->{'exclude-path'}) {
        foreach my $path (@{$opts->{'exclude-path'}}) {
            # $self->find_add_exclude ('-regex', $path);                                                                                                               
        }
    }

The "$self->find_add_exclude" is commented out.

I really appreciate proxmox and OpenVz, but I'm surprised you wouldn't find the time to patch a bug that can eat all the storage space of an hypervisor.

regards

 

[dietmar]

I really appreciate proxmox and OpenVz, but I'm surprised you wouldn't find the time to patch a bug that can eat all the storage space of an hypervisor.

I do not really consider that a bug, because it only affects a few users/VMs, and storage is only used temporarily. Anyways, if someone send a correct patch I will include that. Your patch looks wrong, because it does not work with regex patterns.

 

[sunnyside]

Does someone as a tip to exclude a path except a directory (ex: exclude /var/log except /var/log/samba ) ?
Thanks for your help