[TUTORIAL] Retention for vzdump backups after off-site transfer – retentions

[tkn777]

I use vzdump for my Proxmox VE backups, but I deliberately keep backup creation, transport and retention as separate steps.

My workflow looks roughly like this:

vzdump
↓
vzdump hook
↓
copy/move completed backup to another location
↓
retentions
↓
apply the retention policy

For example, after the backup has been transferred to the actual backup storage, I can keep:

• 7 daily backups
• 4 weekly backups
• 12 monthly backups

with:

retentions /backup/proxmox '*.vma.zst' -d 7 -w 4 -m 12

One detail that is especially useful for backup sets is companion-file handling: If a backup has related files such as checksums, metadata or notes, retentions can remove those together with the corresponding backup instead of leaving orphaned sidecar files behind.

For example:

retentions /backup/proxmox '*.vma.zst' \
-d 7 -w 4 -m 12 \
--delete-companions "suffix:.vma.zst:.vma.zst.notes,.log

There is also a dry-run mode, so the complete retention decision can be reviewed before anything is deleted.

I originally wrote retentions because I wanted retention to be independent of both the backup software and the storage backend.

It is a small open-source Python CLI tool which works on arbitrary files or folders and supports hourly, daily, weekly, monthly, quarterly and yearly retention, as well as additional limits by age, number of files or total size. There are no dependencies. DEBs and RPMs are available.

The selection is deterministic and bucket-based, rather than simply deleting everything older than N days.

I have been using it as the retention stage behind my vzdump backup workflow and thought it might be useful for others who copy their vzdump backups to separate storage as well.

Project:
https://github.com/tkn777/retentions

Feedback is very welcome, especially from people using similar Proxmox backup workflows.

 

[tkn777]

One additional detail from my setup that may be useful:

I do not run retentions directly on the local vzdump target.

The backup is first transferred by the vzdump hook to the actual backup location, and only then is the retention policy applied there.

That keeps the three concerns separate:

• vzdump creates the backup
• the hook handles transport
• retentions handles retention

This also means the retention policy remains independent of the Proxmox host and of the transport method.