vmdk disks import failure with zeroinit

O

oldguy64

New Member
Proxmox Subscriber
Oct 30, 2024
3
0
1
Hi folks,

I'm strugling with VM migration from vmware.

Disks importation using the ui import wizard or the qm import features fails systematically :

Code: 
qm importdisk 10000 /mnt/pve/source_storage/myvm/myvm_1.vmdk dest_storage -format qcow2
importing disk '/mnt/pve/source_storage/myvm/myvm_1.vmdk' to VM 10000 ...
Formatting '/mnt/pve/dest_storage/images/10000/vm-10000-disk-1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
transferred 0.0 B of 1.0 KiB (0.00%)
qemu-img: output file is smaller than input file
copy failed: command '/usr/bin/qemu-img convert -p -n -f vmdk -O qcow2 /mnt/pve/source_storage/myvm/myvm_1.vmdk zeroinit:/mnt/pve/dest_storage/images/10000/vm-10000-disk-1.qcow2' failed: exit code 1


When I try to import it by hand, with differents options (without zeroinit) :

Code: 
qemu-img convert -p -c -f vmdk -O qcow2 /mnt/pve/source_storage/myvm/myvm_1.vmdk /mnt/pve/dest_storage/images/10000/vm-10000-disk-1.qcow2
    (100.00/100%)
It works...

Storages source and destination are NFS. Source hypervisor ESXi 8.0.2 and destination PVE 9.0.10.
vmdk and -flat.vmdk are in the same location.

As I was suspecting something weird with the originals vmdk file I have tryed to rebuild it (https://knowledge.broadcom.com/external/article/321422).
And...

Code: 
qm disk import 10000 /mnt/pve/dest_storage/import/myvm.vmdk dest_storage --format qcow2
importing disk '/mnt/pve/dest_storage/import/myvm.vmdk' to VM 10000 ...
Formatting '/mnt/pve/dest_storage/images/10000/vm-10000-disk-2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=15032385536 lazy_refcounts=off refcount_bits=16
transferred 0.0 B of 14.0 GiB (0.00%)
transferred 143.4 MiB of 14.0 GiB (1.00%)
transferred 288.2 MiB of 14.0 GiB (2.01%)
transferred 431.5 MiB of 14.0 GiB (3.01%)
...

transferred 14.0 GiB of 14.0 GiB (100.00%)
unused0: successfully imported disk 'dest_storage:10000/vm-10000-disk-2.qcow2'

After comparing the vmdk files (and try and die a several times) I have found out that the 'version' parameter in the vmdk file must be 1,2 or 3 (my original vmdk is version=5) to allow importing with zeroinit.

Higher version number lead qm import to fail with the qemu-img error :
Code: 
qemu-img: output file is smaller than input file

For now I don't know if the content of imported disk is usable (I hope so).

As I would have several hundreds VM to migrate (if I achieve to make proxmox the next corporate hypervisor) it will be easier if I don't have to check vmdk version for each disk...

I can import my disks with qm import disk but not with gui nor ansible module. In those case it ends with :
Code: 
failed to update VM 10000: /mnt/pve/dest_storage/import/my_vm.vmdk: multiple children or extents are not allowed!

Any help would be appreciated :)
 
Last edited:
Hi,
Proxmox VE will allocate the target disk first with the same size as the source. It seems like the real issue is that the size was wrongly detected:
Code: 
Formatting '/mnt/pve/dest_storage/images/10000/vm-10000-disk-1.qcow2' ... size=1024 ...

What does qemu-img info /path/to/problematic.vmdk --output json show?

Could you try the manual invocation with a zeroinit target like
Code: 
qemu-img convert -p -c -f vmdk -O qcow2 /mnt/pve/source_storage/myvm/myvm_1.vmdk zeroinit:/mnt/pve/dest_storage/images/10000/vm-10000-disk-1.qcow2
to verify that it's not actually about zeroinit?

So the recreated image has a smaller version than the original? If you can, please provide a small/compressed example image where the size is not queried correctly and the import fails.
 
Hi Fiona,
Many thanks for your help :)

The ouptut of qemu-img info is... puzzling :
Code: 
{
    "children": [
        {
            "name": "file",
            "info": {
                "children": [
                ],
                "virtual-size": 1024,
                "filename": "/mnt/pve/source_storage/myvm/myvm.vmdk",
                "format": "file",
                "actual-size": 4096,
                "format-specific": {
                    "type": "file",
                    "data": {
                    }
                },
                "dirty-flag": false
            }
        }
    ],
    "virtual-size": 1024,
    "filename": "/mnt/pve/source_storage/myvm/myvm.vmdk",
    "format": "raw",
    "actual-size": 4096,
    "dirty-flag": false
}
It does not even reference the myvm-flat.vmdk.
If I do the same after changing version number in the vmdk the answer is very different.
Code: 
{
    "children": [
        {
            "name": "extents.0",
            "info": {
                "children": [
                ],
                "virtual-size": 15032385536,
                "filename": "/mnt/pve/source_storage/myvm/myvm-flat.vmdk",
                "format": "file",
                "actual-size": 9261490176,
                "format-specific": {
                    "type": "file",
                    "data": {
                    }
                },
                "dirty-flag": false
            }
        },
        {
            "name": "file",
            "info": {
                "children": [
                ],
                "virtual-size": 1024,
                "filename": "/mnt/pve/source_storage/myvm/myvm.vmdk",
                "format": "file",
                "actual-size": 4096,
                "format-specific": {
                    "type": "file",
                    "data": {
                    }
                },
                "dirty-flag": false
            }
        }
    ],
    "virtual-size": 15032385536,
    "filename": "/mnt/pve/source_storage/myvm/myvm.vmdk",
    "format": "vmdk",
    "actual-size": 9261494272,
    "format-specific": {
        "type": "vmdk",
        "data": {
            "cid": 2537300191,
            "parent-cid": 4294967295,
            "create-type": "vmfs",
            "extents": [
                {
                    "virtual-size": 15032385536,
                    "filename": "/mnt/pve/source_storage/myvm/myvm-flat.vmdk",
                    "format": "VMFS"
                }
            ]
        }
    },
    "dirty-flag": false
}
With your exact command qemu-img return:
Code: 
qemu-img convert -p -c -f vmdk -O qcow2 /mnt/pve/source_storage/myvm/myvm.vmdk zeroinit:/mnt/pve/dest_storage/images/10000/vm-10000-disk-4.qcow2
qemu-img: Protocol driver 'zeroinit' does not support image creation
When I create the qcow2 file with the same parameters as given by qm disk import, I can reproduce the failure

Code: 
root@mypve:/mnt/pve/dest_storage/import# qemu-img create -f qcow2 -o cluster_size=65536 -o extended_l2=off -o preallocation=metadata -o compression_type=zlib -o size=1k -o lazy_refcounts=off -o refcount_bits=16 /mnt/pve/dest_storage/images/10000/vm-10000-disk-4.qcow2
Formatting '/mnt/pve/dest_storage/images/10000/vm-10000-disk-4.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
ounts=off -o refcount_bits=16 /mnt/pve/dest_storage/images/10000/vm-10000-disk-4.qcow2
Formatting '/mnt/pve/dest_storage/images/10000/vm-10000-disk-4.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16

root@mypve:/mnt/pve/dest_storage/import# qemu-img convert -p -n -f vmdk -O qcow2 /mnt/pve/source_storage/myvm/myvm.vmdk zeroinit:/mnt/pve/dest_storage/images/10000/vm-10000-disk-4.qcow2
qemu-img: output file is smaller than input file
I guess it won't be easy to fit 14Go in 1kb qcow2 file...
So you are right zeroinit is not directly involved.
If I preallocate the qcow2 file with the known file size
Code: 
root@mypve:/mnt/pve/dest_storage/import# qemu-img create -f qcow2 -o cluster_size=65536 -o extended_l2=off -o preallocation=metadata -o compression_type=zlib -o size=14G -o lazy_refcou
nts=off -o refcount_bits=16 /mnt/pve/dest_storage/images/10000/vm-10000-disk-4.qcow2
Formatting '/mnt/pve/dest_storage/images/10000/vm-10000-disk-4.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=15032385536 lazy_refcounts=off refcount_bits=16

root@mypve:/mnt/pve/dest_storage/import# qemu-img convert -p -n -f vmdk -O qcow2 /mnt/pve/source_storage/myvm/myvm.vmdk zeroinit:/mnt/pve/dest_storage/images/10000/vm-10000-disk-4.qcow2
    (100.00/100%)
So the problem seems to be the incorrect evaluation of the vmdk descriptor by qemu-img when the version parameter is set to more than 3.
This leads to make buitltin tools (ui import or ansible module) unusable...

Edit: Fiona, how can I send you large files ? Even compressed the file is above 300M
 
Last edited:
oldguy64 said:
Edit: Fiona, how can I send you large files ? Even compressed the file is above 300M
Click to expand...
It might be enough to dump the first 1 MiB of the image for the header (and then compress that).

How exactly was the original vmdk image exposing the issue created, e.g. with which version/variant of VMWare, what command?
 
Thanks again Fiona,

You'll find the elements in attachment.
As I don't know the way to dump the first Meg of the disk I did :
Code: 
dd if=myvm_2-flat.vmdk of=disk_extract bs=1M count=1
Hope it will be usable...
I have also put the vmdk descriptor. As those informations are coming from a corporate environment they have been anonymized.

The VM creation is based on templates deployment. This VM (as numerous others) is an oldy and have probably been created under esxi 6.x or may be older.
This VM is passed through a lot of upgrades...

Regards
 

Attachments

You must log in or register to reply here.
Top Bottom
Back