Storages documentation correction

[Tacioandrade]

Hi everyone, I was reading about Proxmox Storages and what each one supports to understand the tool more deeply and found the following content https://pve.proxmox.com/wiki/Storage#_storage_types:

"A number of storages, and the Qemu image format qcow2, support thin provisioning. With thin provisioning activated, only the blocks that the guest system actually use will be written to the storage."

As I always use RAW type disks due to performance gain, in a client that had installed Proxmox in the cloud and had only "local" storage that is dir type, I went to check the used disk and it seemed to support the option of thin provisioning.

I thought something was strange and confirmed it locally with an NFS type storage and the same happened, even using 500GB RAW virtual disks, in the storage the file created was 0k and then it grew as I was installing the system.

I would like to ask administrators if possible to correct the documentation and update which disks support thin provisioning or not, or whether they all support them today.

Also, I would like to know if there is any way to do static provisioning for the disks avoiding risks of storage stuffing.


Thanks in advance to all.

 

[LnxBil]

First, storage is extremely complicated if you go into details and the PVE documentation is "as is" correct for 99,9% of all cases.

Also, thin-provisioning does also NEED support in the guest OS otherwise, it is only thin provisioned in the beginnen and thick-provisioned in the end. This is a big problem with file-based storage backends - all of them. You will not those problems e.g. with ZFS.

...had only "local" storage that is dir type, ...

Unfortunately, this does not say much. If you created that directory on a ZFS, it would have native thin-provisioning of type lazy-zeroed thin-provisioning and the raw file would not use up much space.

I would like to ask administrators if possible to correct the documentation and update which disks support thin provisioning or not, or whether they all support them today.

The small raw file is IMHO a sparse file and not thin provisioning. To the untrained eye, this looks like thin-provisioning, but it isn't. It's also not faster, if you use RAW for its speed. The speed argument does only apply to thick-provisioned storage. It is also not totally true if you use raw files, you need to use raw disks e.g. with LVM. You'll gain then best performance if you have you VM data as a big chunk of consecutive data, everything else suffers from fragmentation issues.

You also mentioned NFS, which can support thin-provisioning, but on the storage end and this is not detectable on the PVE side.

 

[Tacioandrade]

I understood, I didn't know it was so complex. As I see it using dir-like arrays, it would automatically allocate space on the hard drive, much like VMWare and Hyper-V do, but I found it strange when I found in a soyoustart.com installation that I only use local storage ( type dir) can also be used with dynamic provisioning.

While continuing as often as possible using lvm-thin, this discovery has given me a better insight into what I can do with Proxmox.

 

[LnxBil]

can also be used with dynamic provisioning.

I also have no name for that file except a sparse file, if it is a sparse file.

Can you please create a new VM like you described with a raw file and post a ls -l a du and a du --apparent-size of that raw file?

 

[Tacioandrade]

I also have no name for that file except a sparse file, if it is a sparse file.

Can you please create a new VM like you described with a raw file and post a ls -l a du and a du --apparent-size of that raw file?

Follows the print with the disk size created in Proxmox:


Using the ls command actually shows 32GB, but using the du -hs command, which I usually use to know the size of a file, tells the size of the disk rather than the actual size.



I didn't quite understand why the commands were different in size, but using "df -h", the system tells me a value that would only be possible with dynamic disks.

 

[LnxBil]

I didn't quite understand why the commands were different in size, but using "df -h", the system tells me a value that would only be possible with dynamic disks.

Yep, that's a sparse file:
https://en.wikipedia.org/wiki/Sparse_file