Cannot Migrate VM to another Node

[repa]

Hi,

we try to migrate a VM to another node and it fails with the following error:

2020-03-06 11:39:58 copying disk images
volume pve/vm-106-disk-1 already exists
command 'dd 'if=/dev/pve/vm-106-disk-1' 'bs=64k'' failed: got signal 13
send/receive failed, cleaning up snapshot(s)..
2020-03-06 11:39:59 ERROR: Failed to sync data - command 'set -o pipefail && pvesm export local-lvm:vm-106-disk-1 raw+size - -with-snapshots 0 | /usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=hostkey' root@x.x.x.x -- pvesm import local-lvm:vm-106-disk-1 raw+size - -with-snapshots 0' failed: exit code 255
2020-03-06 11:39:59 aborting phase 1 - cleanup resources
2020-03-06 11:39:59 ERROR: found stale volume copy 'local-lvm:vm-106-disk-1' on node 'hostkey'
2020-03-06 11:39:59 ERROR: migration aborted (duration 00:00:01): Failed to sync data - command 'set -o pipefail && pvesm export local-lvm:vm-106-disk-1 raw+size - -with-snapshots 0 | /usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=hostkey' root@x.x.x.x -- pvesm import local-lvm:vm-106-disk-1 raw+size - -with-snapshots 0' failed: exit code 255

Any idea?

 

[fabian]

volume pve/vm-106-disk-1 already exists

sounds like you have a stale volume from a previous failed migration on the target node.

 

[repa]

Thanks for your fast answer!

how can i savely drop that?

 

[fabian]

you can remove it on the target node by selecting the 'local-lvm' storage underneath the target node, then selecting content, then selecting the volume, then clicking the remove button.

 

[repa]

Thanks alot.

We're currently doing a bulk migration of VM's. Most of the VM's are migrated, but we have some VM's which cannot be migrated. All are in local-lvm on the source node and we try to migrate with the following command:

qm migrate 106 <dsthost> -online --with-local-disks --targetstorage cephpool1

We can see, that the problem vm's are creating a disk image in "local-lvm" on the destination host, instead of cephpool1.

any idea for that?

 

[fabian]

Thanks alot.

We're currently doing a bulk migration of VM's. Most of the VM's are migrated, but we have some VM's which cannot be migrated. All are in local-lvm on the source node and we try to migrate with the following command:

qm migrate 106 <dsthost> -online --with-local-disks --targetstorage cephpool1

We can see, that the problem vm's are creating a disk image in "local-lvm" on the destination host, instead of cephpool1.

any idea for that?

please post the full migration log (in 'code' tags).

 

[repa]

I figured out, that the two "problem vm" have two disk images, disk-0 and disk-1 in local-lvm.

in the hardware tab of the vm, i can only see disk-0 assigned.

when i try to delete disk-1, i get an error that the vm with the id <> exisits.

a qm rescan --vmid... does not help, the disk-1 does not show up.

how can i delete the disk-1 ?

 

[fabian]

please post the relevant information (e.g., storage config, VM config, output of 'qm rescan', migration logs, 'pveversion -v').

 

[repa]

we currently have some 5.4 Hosts with local storage and some 6.1 Hosts with CEPH. We want to migrate the vm's from 5.4 to the new hosts.

The source host has local storage (local-lvm), VM Config:


Local-LVM shows the two disks:


All VM's which we can migrate has only one disk attached and no ghost disks (disk-1). So i think, when we can get rid of this disk-1 the migration should work?

 qm rescan --vmid 106
rescan volumes...
rbd error: rbd: couldn't connect to the cluster!

 

[fabian]

rescan aborts if a storage throws an error - sounds like you have a storage configured that does not work (or does not work on all nodes?).. if you fix that, rescan should work, and you should be able to remove the orphaned disk via the VM config panel.

 

[repa]

I moved the affected VMs using backup/restore, now their migrated.

thanks a lot for your help fabian!