Backup retention settings

[kovaga]

Morning guys,
I have a problem with backup retention since the removal of the maxfiles options and switch to the prune groups.
Whenever the backup job is run, the system leaves the most recent backup and deletes the older ones.
I have tried setting the prune option via the "prune group qemu/XXX", but the settings are not stored and next time I open the same dialog window the settings are empty.
Are they suppose to be stored somewhere or is it an expected behavior?

I have checked the /etc/pve/vzdump.cron and it doesnt contain any of the keep-* settings mentioned in the manual.

So the question is where do you configure the backup retention?


p.s. I am running
# pveversion
pve-manager/6.3-3/eee5f901 (running kernel: 5.4.78-1-pve)

 

[fiona]

Hi,
you can either set the option for the storage (easiest to do so by editing the storage in the GUI with Datacenter > Storage > Edit) or node-specific ones in /etc/pve/vzdump.cron as you already mentioned. Simply adding a line like

prune-backups: keep-last=3,keep-daily=7

and removing the maxfiles option should work.

The prune group qemu/XXX is for pruning once manually, so those settings are not saved anywhere.

 

[kovaga]

I see, thanks Fabian.

I didn't notice that there is another tab in the "Datacenter > Storage > Edit"

 

[spongebob]

Hi @Fabian_E I can't find this on the UI. Where is the edit button?



Thanks!

 

[fiona]

Hi,

Hi @Fabian_E I can't find this on the UI. Where is the edit button?

View attachment 24324

Thanks!

first click on Datacenter, then in the middle pane, click on Storage. Now select your storage and click Edit and in the window that shows up then, there's a second tab Backup Retention.

 

[luenta]

Hi @Fabian_E,

When I set up the retention in Datacenter > Storage > Edit when is it executed? Just before the backup event? Or in a different time?

 

[Dunuin]

I think right after the backup job. So you are not missing a backup if a single VMs backup fails but you need storage for N+1 backups because you got 1 more backup than you setup (atleast for a very short time).

 

[osada]

Hi,
you can either set the option for the storage (easiest to do so by editing the storage in the GUI with Datacenter > Storage > Edit) or node-specific ones in /etc/pve/vzdump.cron as you already mentioned. Simply adding a line like

prune-backups: keep-last=3,keep-daily=7

and removing the maxfiles option should work.

The prune group qemu/XXX is for pruning once manually, so those settings are not saved anywhere.

Hi,

Does it concern the number of backups for ALL VM without exclusions by changing any values in the GUI tool, (i.e. Datacenter > Storage > Edit)? Does it mean that I will have only ONE backup for all VM if I select there 1 in "Keep last"?

How to add various retention policies for every VM in that case?

For example, I have VM with ID 111 and I want to add a retention policy to keep only 1 backup for this VM.
And I have another VM with ID 112 and I want to add a retention policy to keep only 3 backups for this VM.

In this case, I should add in /etc/pve/vzdump.cron the next strings:
vzdump 111 --prune-backups keep-last=1
vzdump 112 --prune-backups keep-last=3


In other words, If I want a different retention policy for every VM in Proxmox VE I should add it in etc/pve/vzdump.cron only?
Am I right?

After editing etc/pve/vzdump.cron should I restart any services?

 

[fiona]

Hi,

Hi,

Does it concern the number of backups for ALL VM without exclusions by changing any values in the GUI tool, (i.e. Datacenter > Storage > Edit)? Does it mean that I will have only ONE backup for all VM if I select there 1 in "Keep last"?

How to add various retention policies for every VM in that case?

For example, I have VM with ID 111 and I want to add a retention policy to keep only 1 backup for this VM.
And I have another VM with ID 112 and I want to add a retention policy to keep only 3 backups for this VM.

In this case, I should add in /etc/pve/vzdump.cron the next strings:
vzdump 111 --prune-backups keep-last=1
vzdump 112 --prune-backups keep-last=3


In other words, If I want a different retention policy for every VM in Proxmox VE I should add it in etc/pve/vzdump.cron only?
Am I right?

After editing etc/pve/vzdump.cron should I restart any services?

Yes, and there should be no need to restart any service. But note that with Proxmox VE 7.1, new backup jobs are now managed in /etc/pve/jobs.cfg and you can also configure job-specific retention settings in the UI.

 

[osada]

Hi,

Yes, and there should be no need to restart any service. But note that with Proxmox VE 7.1, new backup jobs are now managed in /etc/pve/jobs.cfg and you can also configure job-specific retention settings in the UI.

Thanks a lot!

 

[osada]

Hi,

Yes, and there should be no need to restart any service. But note that with Proxmox VE 7.1, new backup jobs are now managed in /etc/pve/jobs.cfg and you can also configure job-specific retention settings in the UI.

root@alm-srv01:~# pveversion -v
proxmox-ve: 6.4-1 (running kernel: 5.4.128-1-pve)
pve-manager: 6.4-13 (running version: 6.4-13/9f411e79)
pve-kernel-5.4: 6.4-5
pve-kernel-helper: 6.4-5
pve-kernel-5.4.128-1-pve: 5.4.128-1
pve-kernel-5.4.124-1-pve: 5.4.124-2
pve-kernel-5.4.119-1-pve: 5.4.119-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve4~bpo10
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.1.0-1
libpve-access-control: 6.4-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-3
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-3
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.12-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.6-1
pve-cluster: 6.4-1
pve-container: 3.3-6
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-4
pve-firmware: 3.2-4
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.5-pve1~bpo10+1

Currently, I have three backups of VM id 111. I'd like to add the backup rotation policy to keep only one backup for this VM in order to save the storage disk space.

But I am doing something wrong. Before editing /etc/pve/vzdump.cron this file looked like this:

less /etc/pve/vzdump.cron
# cluster wide vzdump cron schedule
# Automatically generated file - do not edit

PATH="/usr/sbin:/usr/bin:/sbin:/bin"

30 0 * * 3 root vzdump 111 --compress zstd --mailto xxx@xxx --quiet 1 --mailnotification always --storage backup --mode stop --node name_of_storage

The backup job of VM ID 111 was successfully run several times before according to this schedule.


Yesterday in the evening I added the next string to this file:
root vzdump 111 --prune-backups keep-last=1

As you see, according to the schedule the backup of VM ID 111 must be run at 00.30 AM today but it didn't happen.

In addition, the amount of backup files remains the same, i.e. three backup files.
In other words, the string "--prune-backups keep-last=1" didn't work also.

What's wrong with this configuration?

 

[fiona]

root@alm-srv01:~# pveversion -v
proxmox-ve: 6.4-1 (running kernel: 5.4.128-1-pve)
pve-manager: 6.4-13 (running version: 6.4-13/9f411e79)
pve-kernel-5.4: 6.4-5
pve-kernel-helper: 6.4-5
pve-kernel-5.4.128-1-pve: 5.4.128-1
pve-kernel-5.4.124-1-pve: 5.4.124-2
pve-kernel-5.4.119-1-pve: 5.4.119-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve4~bpo10
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.1.0-1
libpve-access-control: 6.4-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-3
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-3
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.12-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.6-1
pve-cluster: 6.4-1
pve-container: 3.3-6
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-4
pve-firmware: 3.2-4
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.5-pve1~bpo10+1

Currently, I have three backups of VM id 111. I'd like to add the backup rotation policy to keep only one backup for this VM in order to save the storage disk space.

But I am doing something wrong. Before editing /etc/pve/vzdump.cron this file looked like this:

less /etc/pve/vzdump.cron
# cluster wide vzdump cron schedule
# Automatically generated file - do not edit

PATH="/usr/sbin:/usr/bin:/sbin:/bin"

30 0 * * 3 root vzdump 111 --compress zstd --mailto xxx@xxx --quiet 1 --mailnotification always --storage backup --mode stop --node name_of_storage

The backup job of VM ID 111 was successfully run several times before according to this schedule.


Yesterday in the evening I added the next string to this file:
root vzdump 111 --prune-backups keep-last=1

As you see, according to the schedule the backup of VM ID 111 must be run at 00.30 AM today but it didn't happen.

In addition, the amount of backup files remains the same, i.e. three backup files.
In other words, the string "--prune-backups keep-last=1" didn't work also.

What's wrong with this configuration?

Please share the output of cat /etc/pve/vzdump.cron. Do you see any cron job being exectued around that time in /var/log/syslog?

 

[osada]

Please share the output of cat /etc/pve/vzdump.cron. Do you see any cron job being exectued around that time in /var/log/syslog?

Here is the file output:

less /etc/pve/vzdump.cron
# cluster wide vzdump cron schedule
# Automatically generated file - do not edit

PATH="/usr/sbin:/usr/bin:/sbin:/bin"

30 0 * * 3 root vzdump 111 --compress zstd --mailto xxx@xxxx --quiet 1 --mailnotification always --storage backup --mode stop --node xxxx

root vzdump 111 --prune-backups keep-last=1



No executed jobs regarding prune-backups exist in /var/log/syslog.


Thank you.

 

[fiona]

Here is the file output:

less /etc/pve/vzdump.cron
# cluster wide vzdump cron schedule
# Automatically generated file - do not edit

PATH="/usr/sbin:/usr/bin:/sbin:/bin"

30 0 * * 3 root vzdump 111 --compress zstd --mailto xxx@xxxx --quiet 1 --mailnotification always --storage backup --mode stop --node xxxx

root vzdump 111 --prune-backups keep-last=1

There is no schedule for the second command so I'd guess cron wasn't able to parse the file (also, the second command does not match the settings of the first one, so it would create a backup at the default storage). I think you might simply want to add --prune-backups keep-last=1 to the first command and remove the second command altogether. The single command will do both, create a new backup and then prune.

No executed jobs regarding prune-backups exist in /var/log/syslog.


Thank you.

 

[osada]

T

There is no schedule for the second command so I'd guess cron wasn't able to parse the file (also, the second command does not match the settings of the first one, so it would create a backup at the default storage). I think you might simply want to add --prune-backups keep-last=1 to the first command and remove the second command altogether. The single command will do both, create a new backup and then prune.

@fabian thank you for your fast reply.

In other words, I should edit /etc/pve/vzdump.cron like this?

30 0 * * 3 root vzdump 111 --compress zstd --mailto xxx@xxxx --quiet 1 --mailnotification always --storage backup --mode stop --node xxxx --prune-backups keep-last=1

Upd.

@Fabian_E thank you for your help. It works now!
The backup job was executed by schedule and after completing it there is only one backup file now.