Limiting size of qcow2 image

[stephankn]

Hello,

I am running Proxmox 4.1 and a KVM which I configured to use 200GB hard disk as a qcow2 file. Was created with 4.0 a while ago.

I now had the situation that the image file grew much larger than the image size resulting in an out of diskspace on my host.

# qemu-img info vm-101-disk-1.qcow2
image: vm-101-disk-1.qcow2
file format: qcow2
virtual size: 200G (214748364800 bytes)
disk size: 219G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Is it possible to limit the size an image can grow to? I thought the size I specified when creating the KVM (the vitual size) was supposed to be it. Obviously this was not the case...

So is it possible to limit disk image sizes? And how to do so?

Stephan

 

[ghusson]

Is there any snapshots in you qcow2 image ? The size increase ca come from this. An other reason could be the data fragmentation when using "thin provisioning" in qcow2 but, it is less likely to be that.

 

[stephankn]

no, not using snapshots. Not even unintentional as they would show up on the qemu-img info output.

I was able to shrink the image by using the "convert" command. Guest uses roughly half the capacity, doing frequent writes and erases.

Disk size went down to 196G but is up to 198G already. So if this can't be fixed I would end up with a multi-hour downtime just to "repair & shrink" the image file every two or three months..

image: vm-101-disk-1.qcow2
file format: qcow2
virtual size: 200G (214748364800 bytes)
disk size: 198G
cluster_size: 65536
Format specific information:
  compat: 1.1
  lazy refcounts: false
  refcount bits: 16
  corrupt: false

What exactly do you mean with thin-provisioning? The "discard" option is not set. I expected the file to only grow until disk size reaches configured virtual size. Is this a general misunderstanding of qcow2?

Would I have been better advised using some other image format than the default suggested qcow2?

 

[ghusson]

about thin provisionning : http://www.stratoscale.com/blog/storage/advantages-and-disadvantages-of-thin-provisioning/
tool around that : http://manpages.ubuntu.com/manpages/precise/man1/virt-sparsify.1.html
I think you are faced to a bug in qcow2. This has already happened.
If you do not want snapshots/suspend, use RAW format, this is thick provisioning and you will be sure to have a fixed size VM disk.
Anyway, you could see if your bug has been found and report it eventually ?

 

[ghusson]

I just falled on that : https://pve.proxmox.com/wiki/Shrink_Qcow2_Disk_Files

 

[stephankn]

Thanks @ghusson for sharing the links. I had found the article in the wiki before and used the convert command to get the qcow2 file disk size below the virtual size again.

What that article misses to mention is whether it is considered normal that an image's disk size can grew significantly larger than the configured virtual size (without snapshots).

Given the possibility of thin provisioning and sparse files it is understandable that an image file of a virtual size 200G containing 50G in the guest can be larger than those 50G. My problem is that there seems to be no limit on the disk size backing it. And having it grow beyond 200G
(in this example) sounds weird. There is no need for extra blocks when all virtual blocks are already mapped. I would accept a few megabytes extra for a mapping table, but in my case we're talking about 22G extra.

As my image is on a SSD providing storage for a database I wanted to use most of the SSD as this storage space is quite limited. So having no real upper limit for that file will (and did) lead to problems due to out of disk space. I solve this in this specific case by switching it to raw format. That drive won't need snapshots and raw might give a slight increase of performance as a bonus.

Still for other KVM images backing operating systems an ever-growing disk image file is a problem.

I did some research and think that by passing through the "discard" option it might work better as unused blocks might be reclaimed. Does anyone have experience on how well this works and what (if any significant) performance impact to expect?

Regarding the ever-growing qcow2 bug: How would that be best reported upstream? I'm using Proxmox and not native qemu. Will they accept a bug report or first blame Proxmox? They ask for a lot of details I can't provide as I have Proxmox interfacing with qemu: http://wiki.qemu.org/Contribute/ReportABug

 

[ghusson]

I read some things about discard : this can lead to performance decreases on disk. It is advised to do the discard job in a crontab.

Regarding the bug, I really don't know. Your question is legitimate but I can't help you. Maybe create a new thread, this can be usefull for other members in the future to know what to do ?

 

[max1486]

I have ran fstrim -av from inside the guest VM as suggested but the actual qcow2 file is still 400GB larger than it should be, any suggestions?

root@media:~# fstrim -av
/: 448 GiB (480963624960 bytes) trimmed

571G Nov 6 18:45 vm-102-disk-0.qcow2

 

[raVen1983]

@Max1484: Do you have any update?