ESXi to Proxmox Migration with ZFS thin provisioning

[A_L_E_X]

Hey,

I'm considering a migration from my current ESXi setup with Freenas providing ZFS and datastore for VMs to Proxmox. Haven't had any issues with this setup, but would like to give a try on Proxmox and LXCs.

In the current setup I'm using thin provisioning for VMs (vmdk), initially the max allowed size of VM disk space is large - few machines are > 100GB , one 1TB. (actual used space is less than 600GB in total for all VMs).

Have another machine with proxmox and ZFS setup, so I'm trying to convert vmdk files to a proxmox KVM machines, which is working fine, but I could not make qemu-img or ZFS to ignore "unallocated space".

When using :
qemu-img convert -f vmdk -O raw <vmdk_file_location> /dev/zpool/VM/VM-disk convertion requires VM space to be equal to 100GB+ and actually allocates that space, no matter that it's thin provisioned and actual used space within VM is much less.

Tried with -S 4k (also with 1k) but nothing changed, conversion still consumes whole space.

If I make a conversion to qcow2 the output file is with correct size (same as allocated in ESXi VM, ignoring remaining free space residing in VM), but when I assign the file to VM it does not boot or even recognize it during boot process. It's simple not in the list and boot options are 1)CD 2) Network Boot 3)Legacy Rom .

Tried : qemu-img convert -f qcow2 -O raw /location_to_qcow2 /dev/zvol/storage/vm-XXX-disk1 . did not work either, as machine does not boot.

Made a zvol myself with sparse enabled , modified already existing zvols with zfs set refreservation=none tank .. no luck.

Apparently I'm missing a vital step in the process and would be appreciated if someone might give some ideas.
My Proxmox machine has 80GB DDR3 ECC and 3TB ZFS mirror with ZIL SSD.

The ultimate goal is to migrate to Proxmox ZFS current thin provisioned VMs hosted on ESXi.
Approximate given disk size to ESXi VMs: 4.5TB
Actual used size by all VMs : 0.6TB

Thank you for your cooperation.

 

[LnxBil]

qemu-img convert -f vmdk -O raw <vmdk_file_location> /dev/zpool/VM/VM-disk convertion requires VM space to be equal to 100GB+ and actually allocates that space, no matter that it's thin provisioned and actual used space within VM is much less.

Thin provisioning does not allocate the space beforehand, but if you write something that is not 0, you will use the space even if you do not see files or data there. To have the "thin provisioning" flag enabled in the storage is normally enough. You also found out about refreservation, so this is also great.

What you can do is to enable discard for the disk and do a trim inside of your VM, so that free space is really freed. Alternatively, you can also overwrite all free space with zeros (e.g. with dd on unix/linux or sdelete on Windows).

Please provide the output of one converted volume like this:

zfs get all /dev/zvol/storage/vm-XXX-disk1

 

[A_L_E_X]

Hey,

Thank you for the hint. Actually managed to perform a migration in the same day I started the post.
Used a bit lazy approach and instead of converting to a different format directly imported vmdk to VM.

qm importdisk VM_ID vmdk_location <storage ID/name> --format vmdk

Later on used

zfs set refreservation=none <tank>

and it worked fine.

Didn't had a time to perform a bit more performance tests and make a direct comparison with ESXi.
I/O definitely improved , but it's more related to the way I have done previous setup and ESXi NFS handling (sync_write .. on top of ZFS ..)

Hopefully during the weekend might find time for some perf tests on same VM configs.
Btw, might be helpful - after importing the disk had some issues with vnc console. It was solved by changing display adapter to vmware compatible from default.[/code]

 

[LnxBil]

Perfect that you could solve your problems. Most problems I encountered are more licensing centric.