Backing up vm's with RAW disks, size?

[tomstephens89]

HI guys,

I look after a 16 node VE 3.4 cluster who's primary storage is presented in the form of iSCSI backed LVM with MPIO devices configured. Obviously Proxmox only allows for RAW image format on this type of storage.

So I have a VM that has a large disk, about 900GB, only 200 of which is used. However when using the built in backup service, the finished LZO compressed backup file is still over 700GB!

Is this just a by-product of RAW disk images or am I doing something wrong? I like using the proxmox built in backup because its quick and easy for full image style backups, but this backup after compression should be much much smaller.

Can anyone help me out?

I have attached a couple of screenshots showing actual VM disk usage vs the size of the backup.

 

[WhiteStarEOF]

What does it say when you do a du -h on the files?

 

[Erk]

You probably have to zero out the empty space within the VM before it will compress well.

http://forum.proxmox.com/threads/8976-Too-large-size-of-a-backup-copy-Why

 

[tomstephens89]

You probably have to zero out the empty space within the VM before it will compress well.

http://forum.proxmox.com/threads/8976-Too-large-size-of-a-backup-copy-Why

Forgive my ignorance but what exactly is this doing? From my knowledge, reading it looks like its using /dev/zero to write 0's continuously to a file until disk space is used up, then we delete said file. Is this all it takes to zero out the space?

Learn something new every day!

dd if=/dev/zero of=zero.small.file bs=1024 count=102400
dd if=/dev/zero of=zero.file bs=1024
rm zero.small.file
sync ; sleep 60 ; sync
rm zero.file


in a windows vm you can use sdelete:

sdelete.exe -z c::

 

[blackpaw]

Forgive my ignorance but what exactly is this doing? From my knowledge, reading it looks like its using /dev/zero to write 0's continuously to a file until disk space is used up, then we delete said file. Is this all it takes to zero out the space?

Exactly that, it ensures the empty space on the file system is filed with contiguous zeros, which makes for efficient compression.

I did this today with a VM hosted on a ZFS file system with compression enabled - you could see the vm file disk usage shrink as it progressed.

 

[tomstephens89]

Exactly that, it ensures the empty space on the file system is filed with contiguous zeros, which makes for efficient compression.

I did this today with a VM hosted on a ZFS file system with compression enabled - you could see the vm file disk usage shrink as it progressed.

Ok well I just done that on my 'problem' VM and its filled about 700GB with zero's which I have now deleted. Backup is running now, we'll see if it helps. Hopefully my backups of this VM won't be 800GB anymore!

 

[tomstephens89]

Good news! So my VM that has 275GB real usage as reported by df -h used to result in an 800GB backup archive. After zero'ing the free space, the backup archive is only 245GB.

Result!

 

[flakomix]

Mil gracias me salvaste por que no sabia como eliminar todo ese espacio vacio.