How to improve backup

[claudiomiguelmuller]

Hello,
I wanted to take a course.

I have a VM with CentOS.

the backup it is 8gb.

I copied some large files into the CentOs, then the backup to 15GB.

Now, these deleted files from CentOS, but the backup continues to 15GB.

what can be done to lower this backup and return to something around 8gb?

 

[LnxBil]

Wiping the disk with e.g. dd:

dd if=/dev/zero of=/zero bs=1M; sync; sync; sync; rm -f /zero

 

[LnxBil]

The general problem is, that deleting the file will not delete the contents on disk, only remove the pointers to the data. The backup mechanism is scanning which blocks on storage are filled, so there is always something there except you really overwrite it with zeros.

This problem is common to all virtualization products, whereas there exist solutions of e.g. vmware that scan the actual file allocation table for used blocks and only back up those, but this only applies to mainstream OS like Windows Server, etc.

 

[robhost]

If you have SSD storage or any other block storage (ie. RBD) which supports TRIM, you can activate "discard" in PVE and then shrink the used space with "fstrim" from inside the VM. We're doing this on a regular base on all our VMs.

 

[claudiomiguelmuller]

Wiping the disk with e.g. dd:

dd if=/dev/zero of=/zero bs=1M; sync; sync; sync; rm -f /zero

it may be harmful for something?

 

[robhost]

it may be harmful for something?

It can cause lot if IO on your host and your virtual disk will be full for a few moments, but that should be no problem.

 

[claudiomiguelmuller]

Ok,
I am testing on a test VM, and then run it in production

and it is taking too long

 

[claudiomiguelmuller]

if it exists in the Linux mount to another Linux
it will process this place too?

 

[LnxBil]

No, you have to wipe every "real" mountpoint by yourself.

The best way is to use a storage backend with trim as @robhost explained, but it is easier than said.

 

[claudiomiguelmuller]

i have:

-bash-3.00$ df -h
Sist. Arq. Tam Usad Disp Uso% Montado em
/dev/vda1 29G 4,1G 24G 15% /
none 2,5G 0 2,5G 0% /dev/shm
/dev/vda3 263G 217G 34G 87% /home


in this case I do DD 3 times?

 

[LnxBil]

two times ... always use

df -h -x tmpfs -x devtmpfs

 

[claudiomiguelmuller]

dd if=/dev/zero of=/zero bs=1M; sync; sync; sync; rm -f /zero

and

dd if=/dev/zero of=/home/zero bs=1M; sync; sync; sync; rm -f /home/zero

?

 

[LnxBil]

That'll work.

But just try to get TRIM to work. Much, much better and easier to integrate into hundreds of machines.

 

[claudiomiguelmuller]

for now I run the DVD on each partition.
but what is TRIM software or hardware?

 

[LnxBil]

TRIM is both.

Your storage backend has to have TRIM support (e.g. SSD in hardware, or e.g. ZFS in software), then this information will be propagated to your VM and you can then run fstrim inside Linux to signalize free blocks to the underlying storage system and the storage system itself will then mark the blocks as free. This is the same than the dd command, but much more efficient and much faster.

 

[claudiomiguelmuller]

Hi,
Thank you for your help,
after DD & SYNC & RM or SDelete

VM 1 (centos): backup before 18gb, after 5gb
VM 2 (centos-web): backup before 10gb, after 9gb
VM 3 (LinuxMint-printers): backup before 10gb, after 5gb
VM 4 (Win7-web-printers): backup before 44gb, after 11gb
VM 5 (postfix): backup before 37gb, after 15gb

119gb to 45gb.

 

[LnxBil]

Yes, that's really good. I also run TRIM on all machines supporting it and dd'ing everything else.

For linux, you can also replace rm with wipe, which can overwrite with zeros automatically. That's good with logroate.