Backup stays not the same

[LnxBil]

Hi,

If I backup an offline VM (no compression) twice, the backup files are not identical. For a 1.8 GB VM (Jessie) the difference is only 313 KB, but why is there a difference at all?

$ ls -l
-rw-r--r-- 1 4294967294 wheel 1557838848 Sep 2 18:06 vzdump-qemu-3019-2015_09_02-18_04_41.vma
-rw-r--r-- 1 4294967294 wheel 1557838848 Sep 2 20:10 vzdump-qemu-3019-2015_09_02-20_09_12.vma

$ time cmp -l vzdump-qemu-3019-2015_09_02-18_04_41.vma vzdump-qemu-3019-2015_09_02-20_09_12.vma > cmp_3019_18_04_41-20_09_12

$ ls -l cmp_3019_18_04_41-20_09_12
-rw-r--r-- 1 root wheel 331529 Sep 2 21:44 cmp_3019_18_04_41-20_09_12


Any clues?

Best,
Andreas

 

[t.lamprecht]

Things like other time stamp in the files make the difference, and it's much much less then the 313KB you get, you see the real difference in bytes when you don't use the -l options.

With the -l option the byte number gets also printed, that makes a huuuge difference, for example:

$ time cmp -l vzdump-qemu-200-2015_09_03-07_38_24.vma vzdump-qemu-200-2015_09_03-07_40_59.vma > test.cmp
$ /export/ha-test/dump# ls -lh
total 21G
-rw-r--r-- 1 root users 662K Sep  3 07:53 test.cmp
[...]

$ time cmp vzdump-qemu-200-2015_09_03-07_38_24.vma vzdump-qemu-200-2015_09_03-07_40_59.vma > test.cmp
$ root@nasi:/export/ha-test/dump# ls -lh
total 21G
-rw-r--r-- 1 root users  103 Sep  3 07:50 test.cmp
[...]

So it's a fake change of 662 kilobyte vs the real and understandable change of 103 BYTES, order of magnitudes less

 

[LnxBil]

Hi Thomas,

Thank you for the explanation but it is wrong. cmp without an argument only states where the first difference occurs and therefore this yields your small file. I know that my presented file size is a little exaggerated due to -l, but only by a factor of ca. 20. The command cmp -l states each changed byte, so each line is one changed byte. Therefore the lines in the file are the changed bytes, so we have this:

$ wc -l cmp_3019_18_04_41-20_09_12
18073 cmp_3019_18_04_41-20_09_12

So, why is there a change of 18073 Bytes? This is much more than a timestamp.

Best,
Andreas

 

[t.lamprecht]

Ok sorry, you are right, cmp is not one of my favorites.

As we're Open Source you can check the code, to see if we do some malicious things

And as it's difficult to find things, as someone wou isn't into the project or even a developer, here is the essential part:
https://git.proxmox.com/?p=qemu-ser...b6eaf4cc32159803e2b94547e8e3d7d6;hb=HEAD#l224

You see we use KVM/qemu Live backup, which 'starts' the VM in an freezed state to backup everything, look at https://pve.proxmox.com/wiki/Backup_and_Restore#Backup
I could imagine that some blocks are located at different positions, or meta information's differ, as the produced files HAVE the same size.

It's a bit strange, asking the qemu people (on their mailing list) would give you an exacter answer.