Automatically delete old backups?

[Faris Raouf]

I wonder if there's some confusion here?

When you create an automated backup job (Cluster > Backups), it will create backups until the "max backups" set for the backup location is reached. At that point, it will allow one more backup, and then will delete the oldest backup.

But when you manually run a backup via the GUI (or issue a command at the cli), it will not permit you to create more backups than is permitted by the "max backups" setting for the backup location you are backing up to. It simply refuses, saying max backups has been reached for this location.

From the looks of things, from the CLI but not the GUI, you can add "remove 1" to the command line to see behaviour similar to the automated backup tasks (although I've not tested this).

@CarolBrown is this what you mean? You are suggesting a tick box in the manual backup via the GUI that effectively adds "remove 1" to delete the oldest backup after making a new backup, when max backups for the backup location has been reached?

 

[CarolBrown]

I wonder if there's some confusion here?

When you create an automated backup job (Cluster > Backups), it will create backups until the "max backups" set for the backup location is reached. At that point, it will allow one more backup, and then will delete the oldest backup.

But when you manually run a backup via the GUI (or issue a command at the cli), it will not permit you to create more backups than is permitted by the "max backups" setting for the backup location you are backing up to. It simply refuses, saying max backups has been reached for this location.

From the looks of things, from the CLI but not the GUI, you can add "remove 1" to the command line to see behaviour similar to the automated backup tasks (although I've not tested this).

@CarolBrown is this what you mean? You are suggesting a tick box in the manual backup via the GUI that effectively adds "remove 1" to delete the oldest backup after making a new backup, when max backups for the backup location has been reached?

No, once I created an automated backup this task doesn't removes old ones when proxmox raises to their max amount. I think it would be any better to add a checkbox in backup GUI called "Remove old backups" that will be add key "--remove 1" inside a task in vzdump.cron

 

[CarolBrown]

it is default since a very long time

also there is no feature difference between versions all features are always available

I have made a screenshot. Then where is it?

 

[dcsapak]

I have made a screenshot. Then where is it?

there is no checkbox for this, it is automatically added

 

[CarolBrown]

there is no checkbox for this, it is automatically added

Thanks! I Will test.

 

[sleepingcat]

Hi all I am new to Proxmox and I have been testing it for a couple of months before we take it into the production server room, however we have made backups and they all work very well but we still delete the old stuff manually because it reaches a limit (10 backups and stops) I have checked /etc/vzdump.conf and /etc/cron.d/vzdump and etc/pve/vzdump, and no mention of --remove 1, I see the above examples and we have tried them be they never work, we are using Proxmox 5.3 can anyone point us in the right direction and is there just one file to alter or do we alter them all.
Just for a quick note the wiki states that auto delete is set as default so what have we done wrong. thanks inadvance chaps.

 

[Dominic]

we are using Proxmox 5.3 can anyone point us in the right direction

Upgrading to the latest version is a start.

is there just one file to alter or do we alter them all.

Try inserting a comment into either /etc/cron.d/vzdump or /etc/pve/vzdump. Your changes will be forwarded to the other file.

Just for a quick note the wiki states that auto delete is set as default

Which is stated by the man pages as well because that's what happens. vzdump 100 is equivalent to vzdump 100 --remove 1.

Backup jobs in Datacenter->Backup do not add the remove option. Thus, the default --remove 1 is applied. Pressing "Backup now" in a node's Backup view however adds --remove 0 to the command. Consequently, pressing the button when the backup limit is reached leads to an error. I suggest consulting the task log at the bottom of the GUI to observe this behavior.

 

[sleepingcat]

Hi Dominic,
Thank you for your reply. We have tried to insert --remove 1 in the /etc/cron.d/vzdump file using nano but it appears it wont allow the file to be modified because when we log out, then later login no changes have been made, also the etc/pve/vzdump file is empty no text.

We have used the built in nano editor. Our test server is still doing regular saves but we still have to remove old backups, any other ideas, and can you explain why the information we read tells us that remove 1 is the default value but in the test model we dont seem to have that default.
Thank you.

 

[Dominic]

when we log out, then later login no changes have been made, also the etc/pve/vzdump file is empty no text.

Changes should of course survive a logout + login under normal circumstances. Is /etc/pve/vzdump the only file in /etc/pve that appears to be empty?

What is the output of

ls -lah /etc/cron.d && ls -lah /etc/pve

?

 

[sleepingcat]

Last login: Wed Jul 3 12:04:59 2019 from 192.168.1.5

root@pve:~# ls -lah /etc/cron.d && ls -lah /etc/pve

total 44K

drwxr-xr-x 2 root root 4.0K Jul 3 09:55 .

drwxr-xr-x 92 root root 4.0K Jul 2 10:08 ..

-rw-r--r-- 1 root root 102 May 3 2015 .placeholder

-rw-r--r-- 1 root root 75 Jul 12 2018 pveupdate

lrwxrwxrwx 1 root root 20 Jul 12 2018 vzdump -> /etc/pve/vzdump.cron

-rw-r----- 1 root www-data 292 Jul 3 09:07 vzdump.save

-rw-r----- 1 root www-data 292 Jul 3 09:16 vzdump.save.1

-rw-r----- 1 root www-data 292 Jul 3 09:19 vzdump.save.2

-rw-r----- 1 root www-data 292 Jul 3 09:22 vzdump.save.3

-rw-r----- 1 root www-data 292 Jul 3 09:25 vzdump.save.4

-rw-r----- 1 root www-data 292 Jul 3 09:29 vzdump.save.5

-rw-r--r-- 1 root root 214 Nov 6 2017 zfsutils-linux

total 7.5K

drwxr-xr-x 2 root www-data 0 Jan 1 1970 .

drwxr-xr-x 92 root root 4.0K Jul 2 10:08 ..

-rw-r----- 1 root www-data 451 Jul 12 2018 authkey.pub

-r--r----- 1 root www-data 8.8K Jan 1 1970 .clusterlog

-rw-r----- 1 root www-data 16 Jul 12 2018 datacenter.cfg

-rw-r----- 1 root www-data 2 Jan 1 1970 .debug

lrwxr-xr-x 1 root www-data 0 Jan 1 1970 local -> nodes/pve

lrwxr-xr-x 1 root www-data 0 Jan 1 1970 lxc -> nodes/pve/lxc

-r--r----- 1 root www-data 36 Jan 1 1970 .members

drwxr-xr-x 2 root www-data 0 Jul 12 2018 nodes

lrwxr-xr-x 1 root www-data 0 Jan 1 1970 openvz -> nodes/pve/openvz

drwx------ 2 root www-data 0 Jul 12 2018 priv

-rw-r----- 1 root www-data 2.1K Jul 12 2018 pve-root-ca.pem

-rw-r----- 1 root www-data 1.7K Jul 12 2018 pve-www.key

lrwxr-xr-x 1 root www-data 0 Jan 1 1970 qemu-server -> nodes/pve/qemu-server

-r--r----- 1 root www-data 669 Jan 1 1970 .rrd

-rw-r----- 1 root www-data 262 May 17 19:06 storage.cfg

-rw-r----- 1 root www-data 46 Jul 12 2018 user.cfg

-r--r----- 1 root www-data 404 Jan 1 1970 .version

-r--r----- 1 root www-data 142 Jan 1 1970 .vmlist

-rw-r----- 1 root www-data 292 Jul 3 09:52 vzdump.cron

root@pve:~# Last login: Wed Jul 3 12:04:59 2019 from 192.168.1.5

an 1 1970 .clusterlog

-rw-r----- 1 root www-data 16 Jul 12 2018 datacenter.cfg

-rw-r----- 1 root www-data 2 Jan 1 1970 .debug

lrwxr-xr-x 1 root www-data 0 Jan 1 1970 local -> nodes/pve

lrwxr-xr-x 1 root www-data 0 Jan 1 1970 lxc -> nodes/pve/lxc

-r--r----- 1 root www-data 36 Jan 1 1970 .members

drwxr-xr-x 2 root www-data 0 Jul 12 2018 nodes

lrwxr-xr-x 1 root www-data 0 Jan 1 1970 openvz -> nodes/pve/openvz

drwx------ 2 root www-data 0 Jul 12 2018 priv

-bash: Last: command not found

-rw-r----- 1 root www-data 2.1K Jul 12 2018 pve-root-ca.pem

-rw-r----- 1 root www-data 1.7K Jul 12 2018 pve-www.key

lrwxr-xr-x 1 root www-data 0 Jan 1 1970 qemu-server -> nodes/pve/qemu-server

-r--r----- 1 root www-data 669 Jan 1 1970 .rrd

-rw-r----- 1 root www-data 262 May 17 19:06 storage.cfg

-rw-r----- 1 root www-data 46 Jul 12 2018 user.cfg

-r--r----- 1 root www-data 404 Jan 1 1970 .version

-r--r----- 1 root www-data 142 Jan 1 1970 .vmlist

-rw-r----- 1 root www-data 292 Jul 3 09:52 vroot@pve:~# root@pve:~# ls -lah /etc/cron.d && ls -lah /etc/pve

zdump.cron

root@pve:~#-bash: root@pve:~#: command not found

root@pve:~# total 44K

-bash: total: command not found

root@pve:~# drwxr-xr-x 2 root root 4.0K Jul 3 09:55 .

-bash: drwxr-xr-x: command not found

root@pve:~# drwxr-xr-x 92 root root 4.0K Jul 2 10:08 ..

-bash: drwxr-xr-x: command not found

root@pve:~# -rw-r--r-- 1 root root 102 May 3 2015 .placeholder

-bash: -rw-r--r--: command not found

root@pve:~# -rw-r--r-- 1 root root 75 Jul 12 2018 pveupdate

-bash: -rw-r--r--: command not found

root@pve:~# lrwxrwxrwx 1 root root 20 Jul 12 2018 vzdump -> /etc/pve/vzdump.cron

-bash: lrwxrwxrwx: command not found

root@pve:~# -rw-r----- 1 root www-data 292 Jul 3 09:07 vzdump.save

-bash: -rw-r-----: command not found

root@pve:~# -rw-r----- 1 root www-data 292 Jul 3 09:16 vzdump.save.1

-bash: -rw-r-----: command not found

root@pve:~# -rw-r----- 1 root www-data 292 Jul 3 09:19 vzdump.save.2

-bash: -rw-r-----: command not found

root@pve:~# -rw-r----- 1 root www-data 292 Jul 3 09:22 vzdump.save.3

-bash: -rw-r-----: command not found

root@pve:~# -rw-r----- 1 root www-data 292 Jul 3 09:25 vzdump.save.4

-bash: -rw-r-----: command not found

root@pve:~# -rw-r----- 1 root www-data 292 Jul 3 09:29 vzdump.save.5

-bash: -rw-r-----: command not found

root@pve:~# -rw-r--r-- 1 root root 214 Nov 6 2017 zfsutils-linux

-bash: -rw-r--r--: command not found

root@pve:~# total 7.5K

-bash: total: command not found

root@pve:~# drwxr-xr-x 2 root www-data 0 Jan 1 1970 .

-bash: drwxr-xr-x: command not found

root@pve:~# drwxr-xr-x 92 root root 4.0K Jul 2 10:08 ..

-bash: drwxr-xr-x: command not found

root@pve:~# -rw-r----- 1 root www-data 451 Jul 12 2018 authkey.pub

-bash: -rw-r-----: command not found

root@pve:~# -r--r----- 1 root www-data 8.8K Jan 1 1970 .clusterlog

-bash: -r--r-----: command not found

root@pve:~# -rw-r----- 1 root www-data 16 Jul 12 2018 datacenter.cfg

-bash: -rw-r-----: command not found

root@pve:~# -rw-r----- 1 root www-data 2 Jan 1 1970 .debug

-bash: -rw-r-----: command not found

root@pve:~# lrwxr-xr-x 1 root www-data 0 Jan 1 1970 local -> nodes/pve

-bash: nodes/pve: No such file or directory

root@pve:~# lrwxr-xr-x 1 root www-data 0 Jan 1 1970 lxc -> nodes/pve/lxc

-bash: nodes/pve/lxc: No such file or directory

root@pve:~# -r--r----- 1 root www-data 36 Jan 1 1970 .members

-bash: -r--r-----: command not found

root@pve:~# drwxr-xr-x 2 root www-data 0 Jul 12 2018 nodes

-bash: drwxr-xr-x: command not found

root@pve:~# lrwxr-xr-x 1 root www-data 0 Jan 1 1970 openvz -> nodes/pve/openvz

-bash: nodes/pve/openvz: No such file or directory

root@pve:~# drwx------ 2 root www-data 0 Jul 12 2018 priv

-bash: drwx------: command not found

root@pve:~# -rw-r----- 1 root www-data 2.1K Jul 12 2018 pve-root-ca.pem

-bash: -rw-r-----: command not found

root@pve:~# -rw-r----- 1 root www-data 1.7K Jul 12 2018 pve-www.key

-bash: -rw-r-----: command not found

root@pve:~# lrwxr-xr-x 1 root www-data 0 Jan 1 1970 qemu-server -> nodes/pve/qemu-server

-bash: nodes/pve/qemu-server: No such file or directory

root@pve:~# -r--r----- 1 root www-data 669 Jan 1 1970 .rrd

-bash: -r--r-----: command not found

root@pve:~# -rw-r----- 1 root www-data 262 May 17 19:06 storage.cfg

-bash: -rw-r-----: command not found

root@pve:~# -rw-r----- 1 root www-data 46 Jul 12 2018 user.cfg

-bash: -rw-r-----: command not found

root@pve:~# -r--r----- 1 root www-data 404 Jan 1 1970 .version

-bash: -r--r-----: command not found

root@pve:~# -r--r----- 1 root www-data 142 Jan 1 1970 .vmlist

-bash: -r--r-----: command not found

root@pve:~# -rw-r----- 1 root www-data 292 Jul 3 09:52 vzdump.cron

-bash: -rw-r-----: command not found

root@pve:~# root@pve:~#

 

[sleepingcat]

Hi Dominic,
This is the output now because what we have done is installed another Proxmox on another server and started to compare the files, and now copied the /etc/cron.d/vzdump and the /etc/pve/vzdump.cron,,,, strange thing now is on the new Proxmox we can edit the files, but still cant edit the first test version of proxmox, that's why we just copied the files over. so now we have both machines doing backups and deleting , we have it set to 10 on both. So both are now working OK and still the problem of not allowing edits with nano on the first Proxmox.
Still we have the question of why the default of "--remove 1" is not on the machines, is this a bug ? anyway no panic these are for testing before we make the move over to Proxmox fully.

 

[prometheous]

the number of backups to keep is a configuration setting on the storage definition. check "Datacenter/Storage/YOUR_BACKUP_STORAGE: max backups"

Yes, but they are not deleting the oldest after they reach the max. I am having the same issue. It's got to be a missing line in /etc/vzdump.conf

 

[prometheous]

it is default since a very long time

also there is no feature difference between versions all features are always available

No, it's not. You can check max number of files, but it does not automatically delete the oldest backup once the max is reached.

 

[prometheous]

heres a cron script it deletes automatic change the /media part to where it stores the dump

0 18 * * * find /media/external/proxmox -mindepth 1 -mtime +2 -name "*.vma.lzo" -delete
0 18 * * * find /media/external/proxmox -mindepth 1 -mtime +2 -name "*.log" -delete

Where should we add these lines to delete old backups automatically?

 

[Dunuin]

My backups are working fine. I set "max backups" to 3, created a weekly backup task and the oldest backup is automatically deleted.
I'm running PVE6.4 on top of a debian.

 

[killmasta93]

Where should we add these lines to delete old backups automatically?

in the cronjob

run
cronjob -e

 

[Anguskon]

There is actually a way to do this in the GUI (Running 6.3-2), also keep in mind I have a separate NAS (over SMB/CIFS) that I use for my scheduled backups of VMs/Containers as well.

If you go to the actual storage device, and click backups on that storage device (the NAS in my case) the prune option is available where you can select the keep-daily, keep-weekly, etc. I'm unsure if that saved the information or not or if it was just a manual one-time thing that I will have to continuously do every few weeks or so.

 

[KBlixt]

@Dominic
What am I doing wrong here? why doesn't it send a "--remove 1" flag when it detects that the storage is at its backup-limit?
when the update-job goes through the scheduler then the update complete just fine, but when I try to do a manual backup It just refuses to? Do i have to do a manual delete before performing a backup?
I'm on 6.4-13.

 

[danielo515]

No, it's not. You can check max number of files, but it does not automatically delete the oldest backup once the max is reached.

Yes, that is exactly like that for me. I am using 6.2. As someone said on a previous comment, that not being default it's a joke

 

[janvv]

yeah your right i just realized that i guess no need for that cron job anymore

I think I can use this cron job for deleting orphan backup files. I noticed that the pruning feature does not clean up backup files of VM's that do not exist anymore.