Post backup script

[Henry]

I have a Proxmox backup that run at midnight each night.

This backs up the vms & cts to internal storage.

I would then like the backup data copied to an external hdd

Is it possible to have a script that is run when the backup completes?

 

[RomanB]

Maybe you need script for backup vms via api, and copy backup files to external storage?

 

[LnxBil]

The vzdump is able to use a hook script. Please have a look at https://pve.proxmox.com/wiki/VZDump

 

[Henry]

Thanks for this.

I have configured my backup using the GUI. I suspect these settings are kept in a config file somewhere that I new need to edit the command contained within to add the hook script.

Do you know where the defined backup config is stored?

 

[LnxBil]

All PVE-related settings are in /etc/pve, so the vdzump stuff is in /etc/pve/vzdump.cron (crontab file)

 

[Henry]

My hook script is simply:

if [ "$1" == "backup-end" ]; then
mount -L Backup /mnt/external-hdd/
cp /mnt/backup/* /mnt/external-hdd/
umount /mnt/external-hdd
fi


after the mount command I would like to add a test to first ensure the disk was mounted successfully. If true then copy the files and unmount or if false then exist with an error code so the backup is reported as failing.

Is it possible to have the script exist and flag that the process failed to Proxmox?

 

[fabian]

Yes, if the exit code of the script is non-zero the associated backup task fails at that point. You can easily test this with a hook script like:

#!/bin/bash
if [ "$1" == "backup-end" ]; then
  exit 1;
else
  exit 0;
fi

 

[Henry]

Thanks, I am testing this variation now.

#!/bin/bash

if [ "$1" == "backup-end" ]; then
  mount -L Backup /mnt/external-hdd/
  if mountpoint -q /mnt/external-hdd; then
  cp /mnt/backup/* /mnt/external-hdd/
  umount /mnt/external-hdd
  exit 0
  else
  exit 1
  fi
fi

 

[Henry]

Thanks, I am testing this variation now.

#!/bin/bash

if [ "$1" == "backup-end" ]; then
mount -L Backup /mnt/external-hdd/
if mountpoint -q /mnt/external-hdd; then
cp /mnt/backup/* /mnt/external-hdd/
umount /mnt/external-hdd
exit 0
else
exit 1
fi
fi

 

[Henry]

I have changed the hook script trigger to "job-end" from "backup-end" which runs after each backup as opposed to after the overall backup job.

The backup now fails with a "broken pipe" error. I tried to run it manually and received the following error:
ERROR: Backup of VM 1011 failed - only 1 backup(s) allowed - please consider to remove old backup files.

I understand this is because for my storage I have defined only 1 backup for each VM is to be stored.

As I want the prev backups to be overwritten each time does this setting not overwrite automatically or do I need to specify a pre backup hook script to delete the old file prior to backup?

 

[fabian]

How did you run vzdump manually? Unless you pass "-remove 0" it should automatically remove the oldest backup when hitting the maxfiles limit. Note that the removal happens after the successful backup task, not at the beginning.

 

[Henry]

I selected the VM under PVE, went to the backup tab and clicked "Backup now".

My backup storage is at capacity hence the setting of "1" however as this removes the existing backup post job, logically, then my storage would obviously need to have capacity for two backups not just one. I think this was causing the original error as the storage capacity was being exceeded.

 

[fabian]

The "Backup now" button does not remove old backups (i.e., it does pass "-remove 0") since you can remove backups on the same screen with a single click. But yes, if you have maxfiles set to 1, you need to have space for one copy of each of your backed up guests and one extra copy of your biggest guest and some extra space for the log files. In general (maxfiles = N), you need space for N copies of your backed up guests and one extra copy of your biggest guest (and space for logs).

Removing the "old" backup before creating a new one would be a disaster waiting to happen - since the "new" backup might fail and you would have no backup at all.

 

[Henry]

I have now confirmed that my backups are running and only keeping one copy as required.

What does not appear to be running is the hook script and I cannot think why:

root@pve:~# cat /etc/pve/vzdump.cron
# cluster wide vzdump cron schedule
# Automatically generated file - do not edit

PATH="/usr/sbin:/usr/bin:/sbin:/bin"

0 23 * * 1,2,3,4,5,6 root vzdump --mailnotification always --script /root/backup.sh --quiet 1 --storage backup --all 1 --compress lzo --mode snapshot --node pve --mailto henry@domain.com
root@pve:~#

root@pve:~# cat /root/backup.sh
#!/bin/bash

if [ "$1" == "job-end" ]; then
mount -L Backup /mnt/external-hdd/
if mountpoint -q /mnt/external-hdd; then
rm /mnt/external-hdd/*
cp /mnt/backup/* /mnt/external-hdd/
umount /mnt/external-hdd
exit 0
else
exit 1
fi
fi
root@pve:~#

 

[Henry]

From the backup log there is no indication of my hook script being run however it is. Can I have it included in the log?

root@pve:~# cat /etc/pve/vzdump.cron
# cluster wide vzdump cron schedule
# Automatically generated file - do not edit

PATH="/usr/sbin:/usr/bin:/sbin:/bin"

0 23 * * 1,2,3,4,5,6 root vzdump --mailto henry@domain.com --compress lzo --mailnotification always --all 1 --node pve --storage backup --quiet 1 --script /root/backup.sh --mode snapshot
root@pve:~#
root@pve:~# cat /root/backup.sh
#!/bin/bash

if [ "$1" == "job-end" ]; then
mount -L Backup /mnt/external-hdd/
if mountpoint -q /mnt/external-hdd; then
rm -r /mnt/external-hdd/dump
cp -r /mnt/backup/dump/ /mnt/external-hdd/
umount /mnt/external-hdd
exit 0
else
exit 1
fi
fi
root@pve:~#

 

[fabian]

you can use the provided example script in /usr/share/doc/pve-manager/examples/vzdump-hook-script.pl to check whether it gets executed (the script will just print the passed parameters and environment for each hook).

 

[Henry]

Hi Fabian,

I have used the "vzdump-hook-script.pl" script as suggested and can see in the log where the hooks are executed.

Unfortunately at no time is job-end executed which is the hook I need to unmount the USB hard drive.

The only hooks executed for each VM backed up in order are:
backup-start
pre-stop
pre-restart
backup-end

 

[fabian]

job-start and job-end are run at the start and end of the whole vzdump execution, while backup-start and backup-end are run at the start of each individual VM/CT backup:

INFO: starting new backup job: vzdump 101 100 --script /tmp/vzdump-hook-script.pl --mode stop --storage backups --compress lzo
INFO: HOOK: job-start
INFO: HOOK-ENV: dumpdir=/backups/dump;storeid=backups
INFO: Starting Backup of VM 100 (lxc)
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: HOOK: backup-start stop 100
INFO: HOOK-ENV: vmtype=lxc;dumpdir=/backups/dump;storeid=backups;hostname=host100;tarfile=/backups/dump/vzdump-lxc-100-2016_05_11-09_02_10.tar.lzo;logfile=/backups/dump/vzdump-lxc-100-2016_05_11-09_02_10.log
INFO: creating archive '/backups/dump/vzdump-lxc-100-2016_05_11-09_02_10.tar.lzo'
INFO: Total bytes written: 709539840 (677MiB, 18MiB/s)
INFO: archive file size: 363MB
INFO: delete old backup '/backups/dump/vzdump-lxc-100-2016_01_19-10_48_21.tar.lzo'
INFO: HOOK: backup-end stop 100
INFO: HOOK-ENV: vmtype=lxc;dumpdir=/backups/dump;storeid=backups;hostname=host100;tarfile=/backups/dump/vzdump-lxc-100-2016_05_11-09_02_10.tar.lzo;logfile=/backups/dump/vzdump-lxc-100-2016_05_11-09_02_10.log
INFO: Finished Backup of VM 100 (00:00:40)
INFO: HOOK: log-end stop 100
INFO: HOOK-ENV: vmtype=lxc;dumpdir=/backups/dump;storeid=backups;hostname=host100;tarfile=/backups/dump/vzdump-lxc-100-2016_05_11-09_02_10.tar.lzo;logfile=/backups/dump/vzdump-lxc-100-2016_05_11-09_02_10.log
INFO: Starting Backup of VM 101 (lxc)
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: HOOK: backup-start stop 101
INFO: HOOK-ENV: vmtype=lxc;dumpdir=/backups/dump;storeid=backups;hostname=host101;tarfile=/backups/dump/vzdump-lxc-101-2016_05_11-09_02_50.tar.lzo;logfile=/backups/dump/vzdump-lxc-101-2016_05_11-09_02_50.log
INFO: creating archive '/backups/dump/vzdump-lxc-101-2016_05_11-09_02_50.tar.lzo'
INFO: Total bytes written: 1836308480 (1.8GiB, 27MiB/s)
INFO: archive file size: 1.33GB
INFO: delete old backup '/backups/dump/vzdump-lxc-101-2016_05_04-11_34_29.tar'
INFO: HOOK: backup-end stop 101
INFO: HOOK-ENV: vmtype=lxc;dumpdir=/backups/dump;storeid=backups;hostname=host101;tarfile=/backups/dump/vzdump-lxc-101-2016_05_11-09_02_50.tar.lzo;logfile=/backups/dump/vzdump-lxc-101-2016_05_11-09_02_50.log
INFO: Finished Backup of VM 101 (00:01:07)
INFO: HOOK: log-end stop 101
INFO: HOOK-ENV: vmtype=lxc;dumpdir=/backups/dump;storeid=backups;hostname=host101;tarfile=/backups/dump/vzdump-lxc-101-2016_05_11-09_02_50.tar.lzo;logfile=/backups/dump/vzdump-lxc-101-2016_05_11-09_02_50.log
INFO: HOOK: job-end
INFO: HOOK-ENV: dumpdir=/backups/dump;storeid=backups
INFO: Backup job finished successfully

 

[Henry]

Hi Fabian,

job-end is what I require however I cannot get it to execute.

As stated above, when I use the example "vzdump-hook-script.pl" script I only have four hooks that are executed. job-start and job-end are not executed.

 

[fabian]

job-end is always called unless a backup job fails (hard) or is interrupted, in which case job-abort is called. can you post the complete command and output (with the example hook script) and the output of "pveversion -v"?