Proxmox scheduled backup not running anymore

[Luki20]

Hello guys,

I have some custom vzdump.cron commands to run my VM backups. After I had reinstalled my Proxmox server some days ago and copied the following commands into /etc/pve/vzdump.cron, my scheduled backup do not run anymore.

When I create the backups via the web interface, the backups run without any problem.

Is there anything wrong? I don't know why they don't run:

# cluster wide vzdump cron schedule
# Automatically generated file - do not edit

PATH="/usr/sbin:/usr/bin:/sbin:/bin"

0 0 * * *           root vzdump 103 --compress lzo --remove 1 --storage storage --mode snapshot --mailnotification failure --mailto ***@domain.com --maxfiles 2 --quiet 1
0 12 * * *           root vzdump 101 --mode snapshot --storage storage --remove 1 --compress lzo --maxfiles 2 --mailto ***@domain.com --mailnotification failure --quiet 1
0 1 * * *           root vzdump 102 --maxfiles 1 --mailto ***@domain.com --mailnotification failure --quiet 1 --mode snapshot --storage storage --remove 1 --compress lzo
0 20 * * *           root vzdump 105 --quiet 1 --maxfiles 1 --mailto ***@domain.com --mailnotification failure --mode snapshot --storage storage --remove 1 --compress lzo

Are there any logs I can check here? Thanks for your help!

 

[Luki20]

I tried to remove the --quiet 1 from my commands and now it's running again.

But why did this run some years without problem and now not more? Additionally, it causes more I/O to my host disk? I always backup to an external storage, mounted as "storage" above. How is that possible?

 

[fiona]

Hi,
what is the output of pveversion -v and cat /etc/pve/storage.cfg? On the host where the problem occured, what does ls -l /etc/cron.d/vzdump and cat /etc/vzdump.conf show?

Were there any task logs (in the UI select the node and then Task History) at the time when the jobs were supposed to be run?

 

[Luki20]

Hi, thanks for your reply. Here are the outputs:

pveversion -v

proxmox-ve: 6.3-1 (running kernel: 5.4.103-1-pve)
pve-manager: 6.3-6 (running version: 6.3-6/2184247e)
pve-kernel-5.4: 6.3-7
pve-kernel-helper: 6.3-7
pve-kernel-5.4.103-1-pve: 5.4.103-1
pve-kernel-5.4.78-2-pve: 5.4.78-2
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.0-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.0.7
libproxmox-backup-qemu0: 1.0.3-1
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.3-5
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.1-1
libpve-storage-perl: 6.3-7
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.10-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.4-6
pve-cluster: 6.2-1
pve-container: 3.3-4
pve-docs: 6.3-1
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.2-2
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.2.0-3
pve-xtermjs: 4.7.0-3
qemu-server: 6.3-8
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2

cat /etc/pve/storage.cfg

dir: local
        path /var/lib/vz
        content rootdir,images
        maxfiles 0
        shared 0

dir: storagehdd
        path /storagehdd
        content rootdir,backup,images
        maxfiles 100
        shared 0

dir: storagehdd2
        path /storagehdd2
        content snippets,vztmpl,iso,rootdir,backup,images
        maxfiles 100
        shared 0

dir: storage
        path /storage
        content backup,images
        maxfiles 100
        shared 0

ls -l /etc/cron.d/vzdump

lrwxrwxrwx 1 root root 20 Jan 25 20:41 /etc/cron.d/vzdump -> /etc/pve/vzdump.cron

cat /etc/vzdump.conf

# vzdump default settings

#tmpdir: DIR
#dumpdir: DIR
#storage: STORAGE_ID
#mode: snapshot|suspend|stop
#bwlimit: KBPS
#ionice: PRI
#lockwait: MINUTES
#stopwait: MINUTES
#size: MB
#stdexcludes: BOOLEAN
#mailto: ADDRESSLIST
#maxfiles: N
#script: FILENAME
#exclude-path: PATHLIST
#pigz: N

Were there any task logs (in the UI select the node and then Task History) at the time when the jobs were supposed to be run?

No, nothing. That's the most interesting. Also, no errors in the task logs.

 

[fiona]

I don't see anything strange with the configuration...

Does it work when you manually run one of the backup commands with --quiet 1 (check the task log to see the output/log of course)? Is there anything regarding cron or vzdump in the syslog from the time it wasn't working?

 

[Luki20]

When I run the backup commands manually they're running without any problem.

Is there anything regarding cron or vzdump in the syslog from the time it wasn't working?

No, nothing.

 

[fiona]

Is the server part of a cluster or stand-alone?

A last thing you could test is adding back a --quiet 1 to the command in the cron file and see if it still works afterwards.

 

[Luki20]

It's a stand-alone one. Still does not work as it should.

We moved to Proxmox Backup Server now, where we don't need that additional commands and does everything else. Solved the problem for us but there needs something to be investigated, as our solution did work, but then some day not more.

 

[fiona]

It's a stand-alone one. Still does not work as it should.

You mean it does not work when you add back the --quiet 1?

We moved to Proxmox Backup Server now, where we don't need that additional commands and does everything else. Solved the problem for us but there needs something to be investigated, as our solution did work, but then some day not more.

I'm still unable to reproduce this here. It's very strange that there were no messages from cron in the syslog. Does it work with and without --quiet 1 for PBS or is there the same problem?

 

[Luki20]

You mean it does not work when you add back the --quiet 1?

Yes, didn't work.

I'm still unable to reproduce this here. It's very strange that there were no messages from cron in the syslog. Does it work with and without --quiet 1 for PBS or is there the same problem?

I didn't try that for PBS because I use the prune functions of the server directly, so I don't need "maxfiles" and so on.

 

[fiona]

But the --quiet 1 is set for the PBS job and it does work?