How to migrate 500 VMs from VMWare to Proxmox, with as little downtime as possible

[selund]

I've seen a lot people doing the migration using scp, ovftool etc. However, there are a much faster way to do this. In my opinion it's also easier than most other methods. Also the downtime can be made to be as short as a single reboot.

Only thing you need is a nfs storage that you can share between proxmox and vmware. To make sure that the two systems don't interfere with each other create two shares on the same filesystem.

Use a filesystem mounted on /export/nfs/migrate, Create two folders /export/nfs/migrate/{vmware,pve}. Create two nfs exports /export/nfs/migrate/vmware and /export/nfs/migrate/pve . And mount the two different mounts on the corresponding hypervisor.

Create a new vm in proxmox with the same amount of cpu, memory, nics and disks as the source. Also remember to select the same type of bios mode. For windows you'll want to use VMWare PVSCSI in most of the cases. For Linux you can usually use VirtIO SCSI, but you might need to make sure the virtio driver is included in the initrd image before you migrate. Always use vmdk format for the disks, we'll take care of converting them later.
When the vm is created, don't start the vm, but log into your nas with ssh, and go into /export/nfs/migrate/pve/images/<VMID>
In this directory you'll find at least one vmdk file. The naming of this files are vm-<VMID>-disk-<DISKID>.vmdk. Delete these files.

Now in vmware, remove any snapshots on the vm, and migrate the vm to the nfs storage. Make sure that your backup software don't create any snapshots of the vm while you work on it.

When the migration to the nfs storage is completed, go back to your nas shell. In the same folder as before we'll create some links.
First let's link all flat and ctl vmdk files into the <VMID> folder ln ../../../vmware/<vmname>/*{ctl,flat}.vmdk .
Second let's link all vmdk config files in to the corresponding <DISKID> ln ../../../vmware/<vmname>/<firstdisk>.vmdk vm-<VMID>-disk-<DISKID>.vmdk

At this point we're ready to switch the vm over, and can power the vm down in VMWare. When the vm is off in VMWare, powert the vm on in Proxmox.

Make sure the vm is running ok in Proxmox. Network interfaces might get new id's, so you might need to do some changes to network setup, but that should be all you need to do inside a vm.

If the vm is running ok, let's move off the migration nas and at the same time migrate the disks from vmdk to eithe raw or qcow depending on the end storage. The migration is done by moving the disks. For some storage you can select format, if so, use qcow2, if you use zfs, raw will be your only choice. Select Delete source.

After the disks have been migrated we need to remove the /export/nfs/migrate/pve/images/<VMID> folder from the nas since we've linked in the ctl and flat files wihch proxmox won't migrate.

For windows systems you probably will want to migrate from the pvscsi controller to a virtio controller. This https://forum.proxmox.com/threads/how-do-i-convert-an-ide-disk-to-virtio.46394/#post-418862 forum post outlines a method that can be used.

 

[hevisko]

it's not that we can't afford any downtime at all, but 10/12 hours for a single VM is too much. If we can narrow this down to 1 / 2 hours per VM max, that would be great! All migrations will be done nightly within maintenance windows.

8hours downtime, is a 99.9% uptime over a year.
If your clients are paying you that type of monies for 99.9% uptime, then VMWare's pricing shouldn't be a reason to move/migrate.... else, a 99.5% uptime is like more than a day a year of downtime??

Sorry, I don't have sympathy in those cases.

Personally (if I were so less fortunate to be stuck with VMWare) I'll tell the clients: "You have a choice, we times 10 your invoice, or you have to suck up a 2days expected down time"

 

[JP29]

I can't speak directly for vSAN datastores but I imagine they are still symlinked on each ESXi host in /vmfs/volumes and from there the individual VM directories are organized in the same way as a local VMFS datastore.

On my standalone ESXi hosts, I would mount the VMFS datastores on my PVE host(s) using SSHFS. You can create a snapshot of your VMs in vSphere and do qemu-img convert on the now read-only parent VMDK into raw, qcow2, etc, or qm importdisk function can convert directly to RBD.

So I assume you will clone the parent VMDK to qcow2 on some other storage. Create a PVE VM of equivalent specs to your vSphere VM, copy the SMBIOS UUID if needed, power it up, and give it a temporary IP address that is different from the prod vSphere VM.

Remove vmware tools, install qemu-ga and get all of your paravirtual hardware working and synchronize the prod data one final time at the file/db/application layer as needed. If the qemu-img executes during a time of negligible activity, then you can skip this.

Then shut down the vSphere VM and fix the ip address of the PVE VM taking its place.

edit: I see you already have Ceph setup. So the command would be qm importdisk $newVMID $vmdkPath $poolName
and that will automatically create vm-ID-disk-0, disk-1, etc....

I have a PowerShell script that will go through an ESXi host and parse the VMX files, create an equivalent PVE VM, and then convert each VMDK to RBD. No need for any intermediate virtual disk file storage. Convert and Import is combined in one step.

Hey Alyarb, Is there a way you could share script, I'm looking to automate this but can't make it work through PowerShell

Thanks

 

[avnkhanh]

I am storing vmware esxi virtual disks on NFS of a Synology NAS
I have mapped NFS to Proxmox, please guide me on the solution so that Proxmox can run those vms.