New feature on PVE Console: differential backup for qemu VMs

[KP7]

I find differential backups for qemu VMs very usefull (I belive that Im not alone) and there is ready solution, please consider to acquire it into PVE again.
It could be usefull for qemu LVM groups and files.
Link to working solution:
http://ayufan.eu/projects/proxmox-ve-differential-backups/

This kind of feature is at that moment sth like step beyond other VE platform but tommorow will be just normal, for me differential buckup/deduplication can be paid add-on but should be.
It can be based on storage interface (VMw.. driver) or build like above example from open source tools.

Regards
KP7

 

[danielmash]

I've achieved my differential backups just by running xdelta3 from vzdump-hook-script.pl. In that case I don't have to modify gui and still keep it simple.

 

[KP7]

I've achieved my differential backups just by running xdelta3 from vzdump-hook-script.pl. In that case I don't have to modify gui and still keep it simple.

nice can you publish example?

Regerds
KP7

 

[danielmash]

nice can you publish example?

Regerds
KP7

Example script is in /usr/share/doc/pve-manager/examples/vzdump-hook-script.pl
cp /usr/share/doc/pve-manager/examples/vzdump-hook-script.pl /root/bin/vzdump-hook-script.pl
To enable echo "script: /root/bin/vzdump-hook-script.pl" >> /etc/vzdump.conf

My script making deltas from full backup files every time after "normal" backup but once a week just copy full into full.
Once a week all "full" and "deltas" with the same extension "group0" rotated to "group1" etc. similar to logrotate and gropupN with N>max - deleted.
Because on same storage as "normal" proxmox rotating all tar.gz but not touching my "group" and they remain here for the previous week.
xdelta3 runs in screen to let main backup process to continue. Performance as these days we have many cpu cores.
to restore just manually run xdelta3 -d -s "grooupN.full" "groupN.xdelta" "target_to_restore" then perform normal restore in proxmox.

<---cut--->
# Backup result postprocessing
    if ($phase eq 'backup-end') {

        print "Postprocessing Job 1: doing differential backup in background. Use screen -S Xdelta3 -r to control..\n";

        # Default archive type.
        my $extension = 'tar';

        # 0 and 7 = Sunday; 1 = Monday etc.
        my $full_backup_day = 6;

        # Maximum number of groups (weeks) to keep
        my $max = 7;

        if ($vmtype eq 'qemu') { $extension = 'vma'; }
        system ("

          if [ `date +%u` -eq $full_backup_day ]; then

            touch $dumpdir/0-$vmtype-$vmid-dummy.group && rename 's/group(\\d+)/sprintf \"group%d\", \$1+1/e' $dumpdir/*-$vmtype-$vmid-*.group* && mv $tarfile $dumpdir/`basename $tarfile | sed \"s/$extension/group0.full.$extension/\"` && find $dumpdir -name '*-$vmtype-$vmid-*.group$max.*' -delete;

          else

            if ls $dumpdir/*-$vmtype-$vmid-*.group0.full.$extension.*; then

              screen -S Xdelta3 -d -m xdelta3 -q -e -s `ls -1t $dumpdir/*-$vmtype-$vmid-*.group0.full.$extension.gz | head -1` $tarfile $dumpdir/`basename $tarfile | cut -d'.' -f1`.group0.xdelta.$extension.gz

            else

              mv $tarfile $dumpdir/`basename $tarfile | sed \"s/$extension/group0.full.$extension/\"`;

            fi;

          fi

                ") == 0 ||
                die "Differential backup failed. You can try to run bash script manually";


    }
<---cut--->

 

[kobuki]

Nice. Did you automate restoration too?

 

[danielmash]

Nice. Did you automate restoration too?

Not even think. Restoration is rare and mostly manual anyway.