Move VM to another node with different storage type

[harrylein]

My current nodes have LVM partitions. Since ZFS is required for replication, I set up a new node with a ZFS partition and created a storage 'local-zfs'.

When I try to migrate VM from an LVM node to the new ZFS node,

I get the following error messages:
2024-07-24 15:46:21 starting migration of VM 104 to node 'ESX1' (192.168.200.100)
2024-07-24 15:46:21 found local disk 'local-lvm:vm-104-disk-0' (attached)
2024-07-24 15:46:21 found local disk 'local-lvm:vm-104-disk-1' (attached)
2024-07-24 15:46:21 found generated disk 'local-lvm:vm-104-disk-2' (in current VM config)
2024-07-24 15:46:21 copying local disk images
2024-07-24 15:46:21 ERROR: storage migration for 'local-lvm:vm-104-disk-2' to storage 'local-zfs' failed - cannot migrate from storage type 'lvmthin' to 'zfspool'
2024-07-24 15:46:21 aborting phase 1 - cleanup resources
2024-07-24 15:46:21 ERROR: migration aborted (duration 00:00:00): storage migration for 'local-lvm:vm-104-disk-2' to storage 'local-zfs' failed - cannot migrate from storage type 'lvmthin' to 'zfspool'
TASK ERROR: migration aborted

How can I get the VM from an LVM node to a ZFS node?

 

[fiona]

Hi,
the issue is about the generated disk (either TPM or cloudinit). Other disks are migrated via QEMU's drive-mirror (for live-migration), but generated disks are migrated via the storage layer for technical reasons, and there are some limitations about which storage types are compatible. Please try moving that disk first to a different storage type, EDIT: misremembered, see below IIRC a directory type storage should work.

 

[harrylein]

I moved the tpm to a dir storage.

Unfortunately, I get a similar result:
2024-07-25 13:00:27 starting migration of VM 100 to node 'ESX1' (192.168.200.100)
2024-07-25 13:00:27 found local disk 'local-lvm:vm-100-disk-0' (attached)
2024-07-25 13:00:27 found local disk 'local-lvm:vm-100-disk-1' (attached)
2024-07-25 13:00:27 found generated disk 'tpm:100/vm-100-disk-0.raw' (in current VM config)
2024-07-25 13:00:27 copying local disk images
2024-07-25 13:00:27 ERROR: storage migration for 'tpm:100/vm-100-disk-0.raw' to storage 'local-zfs' failed - cannot migrate from storage type 'dir' to 'zfspool'
2024-07-25 13:00:27 aborting phase 1 - cleanup resources
2024-07-25 13:00:27 ERROR: migration aborted (duration 00:00:00): storage migration for 'tpm:100/vm-100-disk-0.raw' to storage 'local-zfs' failed - cannot migrate from storage type 'dir' to 'zfspool'
TASK ERROR: migration aborted

 

[fiona]

Oh, sorry. Right, for ZFS only ZFS is currently supported unfortunately. What you can do if you also set up the tpm storage on the target, is migrate via CLI with --online --with-local-disks --targetstorage local-zfs,tpm:tpm and move the TPM disk back to ZFS after the migration.

 

[truhponen]

Hi,

I struggled with this same issue yesterday. The difference is that I have only LXCs.

I noticed that I can use Proxmox Backup Server for migration... so, I have been backing up resources on LVM-node and restoring them on ZFS-node. So far everything has been going smoothly.

 

[harrylein]

I removed the TPM devices from some guests.
One of them was properly moved.
But two not (see attached complete protocols).

Here is the crucial part of the log:
...
all 'mirror' jobs are ready
2024-07-25 15:17:05 efidisk0: start migration to nbd:unix:/run/qemu-server/100_nbd.migrate:exportname=drive-efidisk0
drive mirror is starting for drive-efidisk0
drive-efidisk0: transferred 0.0 B of 528.0 KiB (0.00%) in 0s
drive-efidisk0: transferred 528.0 KiB of 528.0 KiB (100.00%) in 1s, ready
all 'mirror' jobs are ready
2024-07-25 15:17:06 starting online/live migration on unix:/run/qemu-server/100.migrate
2024-07-25 15:17:06 set migration capabilities
2024-07-25 15:17:06 migration downtime limit: 100 ms
2024-07-25 15:17:06 migration cachesize: 512.0 MiB
2024-07-25 15:17:06 set migration parameters
2024-07-25 15:17:06 start migrate command to unix:/run/qemu-server/100.migrate
2024-07-25 15:17:07 migration status error: failed - Unable to write to socket: Broken pipe
2024-07-25 15:17:07 ERROR: online migrate failure - aborting
2024-07-25 15:17:07 aborting phase 2 - cleanup resources
2024-07-25 15:17:07 migrate_cancel
drive-efidisk0: Cancelling block job
drive-scsi0: Cancelling block job
WARN: drive-efidisk0: Input/output error (io-status: ok)
WARN: drive-scsi0: Input/output error (io-status: ok)
drive-efidisk0: Done.
drive-scsi0: Done.
2024-07-25 15:17:13 ERROR: migration finished with problems (duration 00:19:06)
TASK ERROR: migration problems

 

[harrylein]

...
I noticed that I can use Proxmox Backup Server for migration... so, I have been backing up resources on LVM-node and restoring them on ZFS-node. So far everything has been going smoothly.

A similar procedure also helped in my case.
I don't have a backup server, but I use the backup from Proxmox VE to a shared NFS volume. I therefore restored the guests backed up by the Proxmox VE backup to the desired (other) storage.
-> Works as expected. Thanks for this tip!

@fiona: It would still be good to know why the migration stops for some guests shortly before completion.

 

[fiona]

all 'mirror' jobs are ready
2024-07-25 15:17:06 starting online/live migration on unix:/run/qemu-server/100.migrate
2024-07-25 15:17:06 set migration capabilities
2024-07-25 15:17:06 migration downtime limit: 100 ms
2024-07-25 15:17:06 migration cachesize: 512.0 MiB
2024-07-25 15:17:06 set migration parameters
2024-07-25 15:17:06 start migrate command to unix:/run/qemu-server/100.migrate
2024-07-25 15:17:07 migration status error: failed - Unable to write to socket: Broken pipe

The error happens for the migration of the VM's RAM and state. In particular writing to the socket fails, which might point to a network-related issue. Please share the output of pveversion -v from both source and target as well as the VM configuration qm config 100. Is there anything in the system journal on source or target around the time the issue happens?

Can you try using the insecure migration setting (Datacenter > Options > Migration Settings in the UI) to check if it the issue is related to using Unix sockets or not (should not be used when you don't trust the local network!)?

 

[harrylein]

I was able to migrate all the "problem" guests by restoring the data backup.

In the meantime, I have also recreated the old nodes with zfs.
Unfortunately, I no longer have the original configurations.