Size problem with qm importovf

[ivan.audero]

Good morning everyone! First of all, thanks for your wonderful product and your precious support!

We are trying to migrate VMs from vmWare to Proxmox, and after a bit of tweaking, I succeded. The procedure I used is the following (it may be useful to other users):

- export ovf from vmWare. This gives me a .ovf file (the description of vm) and one .vmdk file (the disk of the vm)
- In Proxmox, I create a new storage pointing to the folder in which I saved the .vmdk file. The name of the storage will be the one to use in the following command.
- From command line, I launch the command: qm importovf /path/to/ovf/file.ovf name_of_imported_storage

After this procedure, I set up the network and everything works fine.

What I noticed is that during the import, the server creates a subfolder image which contains the hard drive in raw format. No matter if I convert the vmdk in qcow2, in this folder there will always be a raw file, which is HUGE with respect the original vmdk/qcow2 (below you have some self-explanatory screenshots. In this case the resulting file is just 4x bigger, but there are cases in which in much much bigger).

I tried to delete the vmdk file, and I just noticed the vm started correctly, so I suspect it is using the raw file created during conversion.

You may imagine that this is not optimal, since I end up with 10 times (or even more) the storage I used in vmWare.

So, here's my questions:

- is there a way to avoid the creation and the use of this incredibly heavy raw file?
- can I just import the vmdk, and use that one for my .ovf imported vm?
- if it is not possible to use the vmdk, can I just convert it to qcow2 and use that one, WITHOUT the creation of raw file?


Thank you very much!

 

[Dominic]

Good morning!

I think there has been a misunderstanding about how importovf works. Did you try to follow the guide on the wiki? If yes, then apparently I should make the instructions there clearer.

The problem starts here:

I create a new storage pointing to the folder in which I saved the .vmdk file

importovf takes for the <manifest> parameter (that is the .ovf file) any path. That means the folder where your exported vmdk & ovf is placed only have to be accessible from the command line interface on your Proxmox VE host. You do not need to create a storage that is listed in the GUI of your Proxmox VE host for this task. For example home (~) would do, even though it is not a Proxmox VE storage by default. What must be such a storage, however, is the <storage> parameter.

Proxmox VE will create a copy (& potentially convert) of the .vmdk files at the <manifest> and place that copy in said Proxmox VE storage. Depending on the type of your target storage you can also use the --format parameter to choose the type of the file (that is the copy) that Proxmox VE creates.

Consequently,

- From command line, I launch the command: qm importovf /path/to/ovf/file.ovf name_of_imported_storage

using name_of_imported_storage is very likely not what you want. Usually you have one (or multiple) storages where all (or at least a couple of) your virtual machines are stored and not one storage for each imported virtual machine.

What I noticed is that during the import, the server creates a subfolder image which contains the hard drive in raw format.

This is supposed to happen. Proxmox VE supports multiple types of content for storage and creates such a subfolder for each content type.

No matter if I convert the vmdk in qcow2, in this folder there will always be a raw file

The target format does not depend on the source format. Use the --format parameter of importovf instead.

I tried to delete the vmdk file, and I just noticed the vm started correctly, so I suspect it is using the raw file created during conversion.

As indicated above, this assumption is correct. You can see what files are used by your virtual machine by looking at the Hardware overview of your virtual machine in the web interface or alternatively typing qm config <vmid>. You will see something like local:vm-100-disk-0. This is the volume id where local indicates the Proxmox VE storage. With this information you can run pvesm path local:vm-100-disk-0 to get the real path of the volume on your disk.

- is there a way to avoid the creation and the use of this incredibly heavy raw file?

Using qm importovf (...) --format qcow2 parameter or importing the virtual machine to any other storage type that supports thin provisioning.

- can I just import the vmdk, and use that one for my .ovf imported vm?

Theoretically you don't have to use any import command. However, I suggest using target format qcow2 as it gives you thin provisioning (=> small size) as well as good integration into the open source environment.

- if it is not possible to use the vmdk, can I just convert it to qcow2 and use that one, WITHOUT the creation of raw file?

Using --format qcow2 will not even intermediately place a raw file on your disk.

 

[ivan.audero]

I don't know how to thank you, Dominic! Your guide is incredibly helpful!

Anyway, I still do not understand which value I should assign to the parameter storage when I import the ovf vm. The folder of the .ovf and .vmdk is accessible from command line, and indeed the first part of the command is quite clear to me: qm importovf /path/to/ovf/file.ovf . I am sure I am missing something from your reply, so I try to make you an example:

the "real" storage is a NAS in which I export .ovf and .vmdk files. The NAS is mounted on the Proxmox host (e.g. /mnt/nas), and so I can run the command qm importovf /mnt/nas/file.ovf. But what do I have to use as storage parameter? I tried to use a path, but I understood it is not what it wanted It seemed to me that it was necessary to give a storage id, and that was the reason I was creating the storage before the import.

After that, your explanation of the --format parameter is absolutely clear. I'm just missing the intermediate step.


Thank you again for your patience.

Good morning!

I think there has been a misunderstanding about how importovf works. Did you try to follow the guide on the wiki? If yes, then apparently I should make the instructions there clearer.

The problem starts here:

importovf takes for the <manifest> parameter (that is the .ovf file) any path. That means the folder where your exported vmdk & ovf is placed only have to be accessible from the command line interface on your Proxmox VE host. You do not need to create a storage that is listed in the GUI of your Proxmox VE host for this task. For example home (~) would do, even though it is not a Proxmox VE storage by default. What must be such a storage, however, is the <storage> parameter.

Proxmox VE will create a copy (& potentially convert) of the .vmdk files at the <manifest> and place that copy in said Proxmox VE storage. Depending on the type of your target storage you can also use the --format parameter to choose the type of the file (that is the copy) that Proxmox VE creates.

Consequently,


using name_of_imported_storage is very likely not what you want. Usually you have one (or multiple) storages where all (or at least a couple of) your virtual machines are stored and not one storage for each imported virtual machine.


This is supposed to happen. Proxmox VE supports multiple types of content for storage and creates such a subfolder for each content type.


The target format does not depend on the source format. Use the --format parameter of importovf instead.


As indicated above, this assumption is correct. You can see what files are used by your virtual machine by looking at the Hardware overview of your virtual machine in the web interface or alternatively typing qm config <vmid>. You will see something like local:vm-100-disk-0. This is the volume id where local indicates the Proxmox VE storage. With this information you can run pvesm path local:vm-100-disk-0 to get the real path of the volume on your disk.


Using qm importovf (...) --format qcow2 parameter or importing the virtual machine to any other storage type that supports thin provisioning.


Theoretically you don't have to use any import command. However, I suggest using target format qcow2 as it gives you thin provisioning (=> small size) as well as good integration into the open source environment.



Using --format qcow2 will not even intermediately place a raw file on your disk.

 

[Dominic]

Anyway, I still do not understand which value I should assign to the parameter storage when I import the ovf vm.

What you need is the <storage_id> from the storage configuration documentation. If you run pvesm status then the Name column shows that <storage_id> and if you go to Datacenter → Storage then ID is the column that shows it.

You will have to create a storage if you have not yet done so.

The NAS is mounted on the Proxmox host

Did you mount it using Proxmxo VE? If yes, then it should already be displayed as NFS storage. If you mounted it manually then you can

• unmount it and mount it using Proxmox VE or
• create a directory storage on top of it, that is Datacenter → Storage → Add with (looking at your example) /mnt/nas as path.

 

[ivan.audero]

Good morning Dominic! Thank you for your kind reply!

So, as I stated in the first post, the creation of a storage (as directory) is necessary for the import process. Using the --format parameter there is no more a raw disk, but a qcow2 disk, and that's what I wanted. After that, I will delete the vmdk and everything will work fine. Thank you very much!

The qcow2 It is anyway bigger than the vmdk (8GB vs 2GB), but that's probably due to the details of the format.


Thank you for your precious support!


Ivan

What you need is the <storage_id> from the storage configuration documentation. If you run pvesm status then the Name column shows that <storage_id> and if you go to Datacenter → Storage then ID is the column that shows it.

You will have to create a storage if you have not yet done so.


Did you mount it using Proxmxo VE? If yes, then it should already be displayed as NFS storage. If you mounted it manually then you can

• unmount it and mount it using Proxmox VE or
• create a directory storage on top of it, that is Datacenter → Storage → Add with (looking at your example) /mnt/nas as path.

 

[ivan.audero]

Last question: every time I import a VM, I have to disable the option "KVM hardware virtualization" in order to be able to boot it. Can I just disable this option during the import process?

 

[Dominic]

Can I just disable this option during the import process?

Unfortunately, there is no such option (yet).

Usually this should not be a problem as having KVM enabled is generally a good idea. What processor does your host have and what CPU type do you use for your virtual machine (qm config <vm_id>)?

 

[ivan.audero]

It is a Dell PowerEdge R310, with 4 x Intel(R) Core(TM) i3 CPU 540 @ 3.07GHz.

Here's instead the qm config

bootdisk: scsi0
cores: 2
kvm: 0
memory: 1024
name: ---
net0: e1000=46:BE:15:67:9E:68,bridge=vmbr0,firewall=1
scsi0: ---:102/vm-102-disk-0.qcow2
smbios1: uuid=5ba3f562-491d-4d67-8eb3-7f19301cc265
vmgenid: 69722233-35af-4361-bad3-f1398910f68b

Unfortunately, there is no such option (yet).

Usually this should not be a problem as having KVM enabled is generally a good idea. What processor does your host have and what CPU type do you use for your virtual machine (qm config <vm_id>)?

 

[Dominic]

With that CPU you should be able to enable KVM. Make sure that VT-x is enabled, for example using kvm-ok. You can also try to use cpu type host.

 

[ivan.audero]

I'll check that out as soon as possible.

Thank you very much for your help!

With that CPU you should be able to enable KVM. Make sure that VT-x is enabled, for example using kvm-ok. You can also try to use cpu type host.