Backup don't fail with a zero bytes VM .conf

[DiSa]

Hello,
this is my first post on this great forum.

For a unknown reason (but i will investigate...) one of my VM has a zero bytes .conf file.
I have a PVE 4.4 cluster and I've scheduled backup for my VM every evening: the backup send a mail to my gmail address where are automatically applied Labels based on the presence of some word in the text of the mail, in particolar 'error'...

Yesterday I've opened the last received mail where NO ERRORS were present:

VMID NAME STATUS TIME SIZE FILENAME

101 VM 101 OK 00:00:01 0KB /var/lib/vz/dump/vzdump-qemu-101-2017_07_29-19_30_02.vma.gz

TOTAL 00:00:01 0KB


Detailed backup logs:

vzdump 101 --quiet 1 --mode snapshot --node pve3-tsc --compress gzip --mailnotification always --storage local --mailto XXXmymailXXX


101: lug 29 19:30:02 INFO: Starting Backup of VM 101 (qemu)
101: lug 29 19:30:02 INFO: status = running
101: lug 29 19:30:03 INFO: update VM 101: -lock backup
101: lug 29 19:30:03 INFO: backup mode: snapshot
101: lug 29 19:30:03 INFO: bandwidth limit: 90000 KB/s
101: lug 29 19:30:03 INFO: ionice priority: 7
101: lug 29 19:30:03 INFO: creating archive '/var/lib/vz/dump/vzdump-qemu-101-2017_07_29-19_30_02.vma.gz'
101: lug 29 19:30:03 INFO: backup contains no disks
101: lug 29 19:30:03 INFO: starting template backup
101: lug 29 19:30:03 INFO: /usr/bin/vma create -v -c /var/lib/vz/dump/vzdump-qemu-101-2017_07_29-19_30_02.tmp/qemu-server.conf execigz -p 16>/var/lib/vz/dump/vzdump-qemu-101-2017_07_29-19_30_02.vma.dat
101: lug 29 19:30:03 INFO: archive file size: 0KB
101: lug 29 19:30:03 INFO: delete old backup '/var/lib/vz/dump/vzdump-qemu-101-2017_07_22-19_30_02.vma.gz'
101: lug 29 19:30:03 INFO: Finished Backup of VM 101 (00:00:01)

I think that this is a dangerous situation where the backup is silently failing... Ok, the real problem is the 101.conf file with a zero byte, but the VM is online and working and I've had no indication of a (multiple) 'unusable' backup... Yes, older backup are being deleted 'correctly', and now I've no working backup available.

I'm ok with a VM with no disk as reported in the log (I could have a VM to test Live CD...) but I think that some info should be present (eg. CPU, RAM, etc...) in the 101.conf file, that the .conf file can't be zero or even that the backup can't be zero without the backup procedure raising an error (or at least some warning...).

If this situation happens it is really easy to think that everything is working and the backup are done correctly: some additional check could easily show the problem.

Bye,
DiSa.

 

[t.lamprecht]

Hi, I stumbled upon your bug report only now and would like to get more information about this.

I think that this is a dangerous situation where the backup is silently failing... Ok, the real problem is the 101.conf file with a zero byte, but the VM is online and working and I've had no indication of a (multiple) 'unusable' backup... Yes, older backup are being deleted 'correctly', and now I've no working backup available.

Not ideal, but the backup log says:

101: lug 29 19:30:03 INFO: backup contains no disks
101: lug 29 19:30:03 INFO: starting template backup

Your VM has no disk configured, which then triggers a "template" backup. In general I agree that this should be a warning, if the VM is not marked as template via the config.

But is the VM in crime (VMID 101) a template? I guess not? Can you post its config:

qm config 101

 

[DiSa]

Hi t.lamprecht,

yes, in the backup we can find two "INFO", but maybe it's important to add a new WARNING/ERROR line when the backup script checks if the <vmid>.conf file has zero byte size, just to present the anomaly and so stopping overwriting older backups with unusable empty backup: Isn't better an old valid backup that a newer and empty one? ;-)

The VM 101 was not a template (never used that...) but a Windows Server active and working: for some reason the 101.conf file has been emptied probably when I've had some replica problem within the cluster... Now I've upgraded everything to v5.0 so I can't reproduce the main problem, but you can manually empty a <vmid>.conf file of a working (testing) VM ad see by yourself that no error or warning is raised in this particular case...

Bye,
Riccardo.

 

[t.lamprecht]

yes, in the backup we can find two "INFO", but maybe it's important to add a new WARNING/ERROR line when the backup script checks if the <vmid>.conf file has zero byte size, just to present the anomaly and so stopping overwriting older backups with unusable empty backup: Isn't better an old valid backup that a newer and empty one? ;-)

In general I agree that this should be a warning, if the VM is not marked as template via the config.

As stated I agree with you, and currently transform a few "info" messages to "Warn" ones, where I think its warranted.

The VM 101 was not a template (never used that...) but a Windows Server active and working: for some reason the 101.conf file has been emptied probably when I've had some replica problem within the cluster... Now I've upgraded everything to v5.0 so I can't reproduce the main problem, but you can manually empty a <vmid>.conf file of a working (testing) VM ad see by yourself that no error or warning is raised in this particular case...

Ok, so this is not a bug where disks where dropped, "just" the result of an empty VM config, and that can only be the result of manual deleting or tinkering with the config file itself.

when I've had some replica problem within

That really should not, or rather cannot, happen without manual intervention where the config was deleted per hand.

Maybe it would not be to bad to show a warning in the WebUIs VM entry if a VM had an empty config (in addition to the warnings on backups).

 

[DiSa]

Hi t.lamprecht,

yes, the problem was only relative of an empty VM config.

Good for some WARNING (or ERROR) instead INFO: don't you think the backup should also fail in those cases?

Also, relative to my main problem with the cluster, I'm sure i didn't delete (or even modify) the content of <VMID.cfg by hand, and I'm the only one with access to Proxmox. With version 4.4 I've had strange and intermittent problems, always resolved rebooting the host: the main effect was that any command like "qm list" or "qm <something>" hangs the shell... But now with the upgrade to 5.0 no more strange problem.

Thank you for your support.

Bye,
Riccardo.