Put VM name into backup name

johjoh

Member
Feb 4, 2011
43
0
6
Hello,
it's possible to modify vzdump script for add to a backup name like:
vzdump-qemu-100-2016_05_20-15_10_00.vma.lzo
tha name of VM like:
vzdump-qemu-100-VMname-2016_05_20-15_10_00.vma.lzo

When a server crash or when purge old backup or when manage backup disk space it's difficult to understand what a backup refer to.
Can you implement in future release of Proxmox this feature?

Thank you
 
Dec 20, 2017
1
0
1
Vienna
I would like to have the VM name in the backup filename too. VM ID tells absolutely nothing about the backup. Moreover, using only VM ID in backups could lead to data loss.
Consider the following scenario:
I have a VM with VM ID 100, I create backups on daily basis and keep 2 versions. One day I decide to remove this VM and create a new one which happens to get the same ID as the previous one. Of course, I use different name. 2 days later when the manager asks me to recover the original VM I can only say sorry, because the backup service already deleted them. To avoid such a situation, manual configuration is needed.

Using a hook script is a nice and quick solution for the missing vm name issue. However, if I rename the backup file in the backup-end stage, the "max backup" will never be reached since VZDump module still searches for the original filename (without the name). This results in 2 things:
  • Setting the max backups value on a storage does not make any difference (it would create infinite number of backups)
  • backup rotating has to be duplicated in the hook script.
I don't mind writing my own hook script, but it would be very nice if could tell VZDump in advance what filename to use for the backup.
 

fireon

Famous Member
Oct 25, 2010
3,571
271
103
38
Austria/Graz
iteas.at
When a server crash or when purge old backup or when manage backup disk space it's difficult to understand what a backup refer to.
Why? A klick on "show config button" show you the vmname and all it's config details.

I would like to have the VM name in the backup filename too.
But yes, it would be nice too.
 

LnxBil

Famous Member
Feb 21, 2015
5,482
598
133
Germany
A few remarks on bigger installations:
* vm names are used multiple times depending on their network and usage (e.g. ns01, ldap, mail etc.)
* IDs are unique and should be grouped in pools concerning their logical environment (e.g. departments, subnets etc.) also in number groups, 1xxxx for this, 2xxxx for that.
* backups are normally archived and managed externally (not on a cluster)
* ID should not be reused unless the previous backups are taken care of after deleting a VM
 

ojaksch

Active Member
Oct 11, 2015
122
19
38
Germany/Earth
[DEPRECATED - see below]

Um...edited the script to actual version, which includes a query for maxfiles: and deleted this post instead of editing. I'm a moron.


Code:
#!/bin/bash

cd "${DUMPDIR}"
MAXFILES=$(grep "maxfiles:" /etc/vzdump.conf | tr -d " " | awk -F":" '{print $2}')

case "${1}" in
   backup-end)
       VM_FNAM_OLD=$(basename ${TARFILE})
       VM_FNAM_NEW=$(basename ${TARFILE} | awk -F "-" '{print $1"-"$2"-"$3"-'${HOSTNAME}'-"$4"-"$5}')
       VM_FNAM_DUP=$(basename ${TARFILE} | awk -F "-" '{print $1"-"$2"-"$3"-'${HOSTNAME}'"}')
       mv "${VM_FNAM_OLD}" "${VM_FNAM_NEW}"
       NUMFILES=$(ls -1 ${VM_FNAM_DUP}*.tar.* | wc -l)
       if [[ $MAXFILES =~ ^-?[0-9]+$ ]] && [ ${NUMFILES} -gt ${MAXFILES} ]; then
           ls ${VM_FNAM_DUP}*.tar.* | head -n $((${NUMFILES}-${MAXFILES})) | xargs rm
           echo "-- maxfiles set to ${MAXFILES} but ${NUMFILES} present. Deleting the older ones."
       fi
       ;;
   log-end)
       LOG_FNAM_OLD=$(basename ${LOGFILE})
       LOG_FNAM_NEW=$(basename ${LOGFILE} | awk -F "-" '{print $1"-"$2"-"$3"-'${HOSTNAME}'-"$4"-"$5}')
       LOG_FNAM_DUP=$(basename ${TARFILE} | awk -F "-" '{print $1"-"$2"-"$3"-'${HOSTNAME}'"}')
       mv "${LOG_FNAM_OLD}" "${LOG_FNAM_NEW}"
       NUMFILES=$(ls -1 ${LOG_FNAM_DUP}*.log | wc -l)
       if [[ $MAXFILES =~ ^-?[0-9]+$ ]] && [ ${NUMFILES} -gt ${MAXFILES} ]; then ls ${LOG_FNAM_DUP}*.log | head -n $((${NUMFILES}-${MAXFILES})) | xargs rm; fi
       ;;
esac
 
Last edited:

Peter123

Member
Mar 5, 2018
32
2
13
50
We are saving the dumps in the new .zst-format.

Could someone tell us how we would have to adapt the script? It didn't help to replace tar for zst.

Thanks.
 

ojaksch

Active Member
Oct 11, 2015
122
19
38
Germany/Earth
Nice to see that my script is still of use.
EDIT: But I don't see why the old code shouldn't work; PVE sends the full name of it's TAR file, therefore the code shouldn't care about the file extension.
Here is the actual version which is still working, also with zstd:
(I'm having problems with idents in the code preview. If in doubt please download, rename and use the attachment)

Code:
#!/bin/bash

# backup-start | backup-end | backup-abort | log-end | pre-stop | pre-restart | post-restart | job-start | job-end | job-abort

cd "${DUMPDIR}"
MAXFILES=$(grep "maxfiles:" /etc/vzdump.conf | tr -d " " | awk -F":" '{print $2}')

case "${1}" in
  backup-end)
VM_FNAM_OLD=$(basename ${TARFILE})
VM_FNAM_NEW=$(basename ${TARFILE} | awk -F "-" '{print $1"-"$2"-"$3"-'${HOSTNAME}'-"$4"-"$5}')
VM_FNAM_DUP=$(basename ${TARFILE} | awk -F "-" '{print $1"-"$2"-"$3"-'${HOSTNAME}'"}')
VM_TYPE=$(basename ${TARFILE} | awk -F "-" '{print $2}')
mv "${VM_FNAM_OLD}" "${VM_FNAM_NEW}"

case "${VM_TYPE}" in
lxc)
NUMFILES=$(ls -1 ${VM_FNAM_DUP}*.tar* | wc -l)
if [[ $MAXFILES =~ ^-?[0-9]+$ ]] && [ ${NUMFILES} -gt ${MAXFILES} ]; then
echo "-- maxfiles set to ${MAXFILES} but ${NUMFILES} present. Deleting the older ones."
ls ${VM_FNAM_DUP}*.tar* | head -n $((${NUMFILES}-${MAXFILES})) | xargs rm
fi
;;
qemu)
NUMFILES=$(ls -1 ${VM_FNAM_DUP}*.vma* | wc -l)
if [[ $MAXFILES =~ ^-?[0-9]+$ ]] && [ ${NUMFILES} -gt ${MAXFILES} ]; then
echo "-- maxfiles set to ${MAXFILES} but ${NUMFILES} present. Deleting the older ones."
ls ${VM_FNAM_DUP}*.vma* | head -n $((${NUMFILES}-${MAXFILES})) | xargs rm
fi
;;
esac
;;

log-end)
LOG_FNAM_OLD=$(basename ${LOGFILE})
LOG_FNAM_NEW=$(basename ${LOGFILE} | awk -F "-" '{print $1"-"$2"-"$3"-'${HOSTNAME}'-"$4"-"$5}')
LOG_FNAM_DUP=$(basename ${TARFILE} | awk -F "-" '{print $1"-"$2"-"$3"-'${HOSTNAME}'"}')
mv "${LOG_FNAM_OLD}" "${LOG_FNAM_NEW}"
NUMFILES=$(ls -1 ${LOG_FNAM_DUP}*.log | wc -l)
if [[ $MAXFILES =~ ^-?[0-9]+$ ]] && [ ${NUMFILES} -gt ${MAXFILES} ]; then ls ${LOG_FNAM_DUP}*.log | head -n $((${NUMFILES}-${MAXFILES})) | xargs rm; fi
;;

esac
 

Attachments

  • vzdump-rename.txt
    1.7 KB · Views: 23
  • Like
Reactions: John Allison

Peter123

Member
Mar 5, 2018
32
2
13
50
Speaking about simple and stupid code :)

We use rclone to upload the VMs to different cloud storages, so we have now put the following at the end of your rename script:

Code:
#!/bin/bash

if [ "$1" = "backup-end" ]; then

rclone copy /path-to-dump  cloud-storage:path-for-dumps  --config /root/.config/rclone/rclone.conf

fi

In this way, we even get a note in the confirmation email whether the upload to the cloud worked.
 

John Allison

Member
Feb 1, 2018
28
0
6
Gateshead UK
www.adlinktech.com
Fantastic little script! nice and neat. many thanks.

Quick question - do you need to restart proxmox in order for changes to take effect?
we have a few 5.4 hosts which just come up with an error after the script is applied, but the 6.2 machine seems fine.
 

ojaksch

Active Member
Oct 11, 2015
122
19
38
Germany/Earth
No outstanding news to my renaming script, but a small addition to respect the new PBS (Proxmox Backp Server).
 

Attachments

  • vzdump-rename.txt
    1.9 KB · Views: 26

Peter123

Member
Mar 5, 2018
32
2
13
50
Maybe just one small thing:

if you want to restore the renamed backup files from the GUI, Proxmox does not recognize the format and refuses to restore. So you have to manually rename the backup files each time before restoring.

According to Proxmox, the following is expected: "Currently, it has to start with vzdump- then any of qemu lxc or openvz, then a VMID number." https://forum.proxmox.com/threads/e...ormat-and-compression-type.70084/#post-324541

Could that be built into the script without much effort? Thank you again.
 

Peter123

Member
Mar 5, 2018
32
2
13
50
Apparently Proxmox has made sure with a patch that the adapted format can now be used to restore directly.

Thank you, Proxmox.
 

Peter123

Member
Mar 5, 2018
32
2
13
50
Strange. Suddenly the restore of a VM with a different name no longer worked. Happy too early.
 

Peter123

Member
Mar 5, 2018
32
2
13
50
No outstanding news to my renaming script, but a small addition to respect the new PBS (Proxmox Backp Server).
One more small thing: after the backups have been renamed, Proxmox no longer automatically deletes them after the set number (Datacenter - Storage - "max backups").

Can that be changed in the script?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!