HA Cluster migration issues

[tuxis]

I have a HA cluster running, with Ceph and all, and I have rebooted one of the nodes this week. We now want to migrate the HA-VM's back to the original server, but that fails without a clear error.

I can say:

root@proxmox01:~# qm migrate 600 proxmox03 -online
Executing HA migrate for VM 600 to node proxmox03

I then see kvm starting on node proxmox03, but then something goes wrong after that and migration fails:

task started by HA resource agent
Nov 25 10:58:05 starting migration of VM 600 to node 'proxmox03' (10.1.1.3)
Nov 25 10:58:05 copying disk images
Nov 25 10:58:05 starting VM 600 on remote node 'proxmox03'
Nov 25 10:58:06 starting ssh migration tunnel
Nov 25 10:58:07 starting online/live migration on localhost:60000
Nov 25 10:58:07 migrate_set_speed: 8589934592
Nov 25 10:58:07 migrate_set_downtime: 0.1
Nov 25 10:58:09 ERROR: online migrate failure - aborting
Nov 25 10:58:09 aborting phase 2 - cleanup resources
Nov 25 10:58:09 migrate_cancel
Nov 25 10:58:10 ERROR: migration finished with problems (duration 00:00:05)
TASK ERROR: migration problems

Disabling HA for this VM does not help. Offline migration works without any issues.

Also, migration between proxmox02 and proxmox03 seems to work fine, but as soon as proxmox01 is involved, it doesn't.

proxmox01 has been up the longest and is probably running the oldest version of the software (since upgrading doesn't restart all services). Is there a service involved in this process that might cause this to fail?

 

[spirit]

do you have same pve-qemu-kvm package version on node1 && node3 ?

(node3 can't have a lower major qemu version than node1)

 

[tuxis]

root@proxmox01:~# /usr/bin/kvm --version
QEMU emulator version 2.5.1 pve-qemu-kvm_2.5-14, Copyright (c) 2003-2008 Fabrice Bellard
root@proxmox02:~# /usr/bin/kvm --version
QEMU emulator version 2.5.1 pve-qemu-kvm_2.5-14, Copyright (c) 2003-2008 Fabrice Bellard
root@proxmox03:~# /usr/bin/kvm --version
QEMU emulator version 2.5.1 pve-qemu-kvm_2.5-14, Copyright (c) 2003-2008 Fabrice Bellard

Any hints on how to get more logging out of migration is welcome as well..