[SOLVED] Question about raw and thin provisioning

[Valerio Pachera]

In my last post I've been testing the capability of qcow2 to shrink when data gets removed from the guest.
As discussed before, the guest need to have use 'SCSI' as Bus/Scsi (virtio-scsi) and the 'dicard' option checked.

Reading qm_virtual_machines_settings documentation page I see

the raw disk image is a bit-to-bit image of a hard disk, similar to what you would get when executing the dd command on a block device in Linux. This format do not support thin provisioning or snapshotting by itself, requiring cooperation from the storage layer for these tasks

So I tried to convert my qcow2 disk to raw format and play again the trick of adding and removing data from the guest.
I expected the raw file not to deallocate the space but it did!

Host
du -sh vm-107-disk-2.raw
1,1G vm-107-disk-2.raw

Guest
dd if=/dev/urandom of=c bs=1M count=1024

Host
du -sh vm-107-disk-2.raw
1,1G vm-107-disk-2.raw

Guest
rm c
sync

Host
du -sh vm-107-disk-2.raw
1,1G vm-107-disk-2.raw

The raw file is located in /var/lib/vz/images/107 (storage local).
Filesystem is ext4 and the local volume is THICK lvm.

  --- Logical volume ---
  LV Path                /dev/pve/root
  LV Name                root
  VG Name                pve
  LV UUID                JiqjHL-Sm4D-iA1k-5hqG-YzgX-3gpI-wSVkWS
  LV Write Access        read/write
  LV Creation host, time vmserver, 2017-06-21 12:27:09 +0200
  LV Status              available
  # open                 1
  LV Size                9,31 GiB
  Current LE             2384
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:0

So, or the documentation in wrong or I'm missing something.

 

[manu]

The documentation you quote is about snapshotting and thin provisioning.
Discarding unuseed blocks is a different topic.

Also your test is detailed enough because in all you tests we see 1,1 GB size on host

 

[wbumiller]

this format does not support thin provisioning or snapshotting by itself, requiring cooperation from the storage layer for these tasks

Note the bold part. What this means is simply that raw files don't contain any metadata, which really just means that it's not up to the format itself to do any thin provisioning. However, ext4, zfs or wherever else the file is stored on may very well be capable of representing holes in raw files efficiently. Hence, discarding parts of a raw file allows the file system to also deallocate those parts if it can.

 

[Valerio Pachera]

Correct me if I'm wrong but 'discard' is strictly related to thin provisioning.
There's no meaning in discard option without thin provisioning.

From wikipedia

Thin provisioning, in a shared-storage environment, provides a method for optimizing utilization of available storage. It relies on on-demand allocation of blocks of data versus the traditional method of allocating all the blocks in advance

The topic title could be renamed "Question about raw and discard".
But I still find confusing the sentence "This format do not support thin provisioning" referred to raw format because
qemu-img create -f raw file.raw
do not allocate blocks and can be larger than the physical storage size.
Nonetheless blocks can be discarded!
In other words, I see no difference between qcow2 and raw files from this point of view.
By opposite, it's very clear that raw file can't be snapshotted.

Another consideration/question may be: if a storage supports thing provisioning, does it always support discard?

 

[wbumiller]

Another consideration/question may be: if a storage supports thing provisioning, does it always support discard?

Not necessarily, but off the top of my head I don't know any storage that wouldn't support it one way or another.
But note that 'discard' and 'thin provisioning' only partially overlap. When the guest tells qemu to discard a block, qemu needs to figure out what to do: If you're giving the guest a block device it'll have to send a discard operation (BLKDISCARD ioctl(2)), if it's a raw file it has to tell the operating system to "punch a hole" into the file (fallocate(2)'s FALLOC_FL_PUNCH_HOLE), if it's a qcow2 file it has to write metadata to remember that there's a now-unused block that can be re-used later on. Whatever happens next is then up to the block device driver or the file system in the first 2 cases. If the block device is a physical disk, the discard operation obviously doesn't thin-provision the drive, whereas if it's a virtual block device provided by lvm-thin it'll deallocate space, triggering thin-provisioning, and LVM can then also be configured to pass the discard operation further down the chain to the next block layer, and so it continues...

 

[Valerio Pachera]

Cool explanation! Thank you very much.
Next Saturday I'll talk about about Proxmox at the Linux Day, specifically about thin-provisioning and discard.
Your help is really appreciated!