Online migration fails with spice

[udo]

Hi,
just tested live migration with spice (with usb + sound redirection) and the live migration fails:

Mar 13 17:34:55 starting migration of VM 201 to node 'proxclt-a' (192.168.100.31)
Mar 13 17:34:55 copying disk images
Mar 13 17:34:55 starting VM 201 on remote node 'proxclt-a'
Mar 13 17:34:58 starting ssh migration tunnel
Mar 13 17:34:58 starting online/live migration on localhost:60000
Mar 13 17:34:58 migrate_set_speed: 8589934592
Mar 13 17:34:58 migrate_set_downtime: 0.1
Mar 13 17:34:58 spice client_migrate_info
Mar 13 17:35:00 migration status: active (transferred 240012942, remaining 599130112), total 1208827904)
Mar 13 17:35:00 migration xbzrle cachesize: 67108864 transferred 0 pages 0 cachemiss 0 overflow 0
Mar 13 17:35:02 migration status: active (transferred 469436649, remaining 360103936), total 1208827904)
Mar 13 17:35:02 migration xbzrle cachesize: 67108864 transferred 0 pages 0 cachemiss 0 overflow 0
Mar 13 17:35:04 migration status: active (transferred 700271757, remaining 52764672), total 1208827904)
Mar 13 17:35:04 migration xbzrle cachesize: 67108864 transferred 0 pages 0 cachemiss 0 overflow 0
query migrate failed: VM 201 not running

query migrate failed: VM 201 not running

query migrate failed: VM 201 not running

query migrate failed: VM 201 not running

query migrate failed: VM 201 not running

query migrate failed: VM 201 not running

Mar 13 17:35:11 ERROR: online migrate failure - too many query migrate failures - aborting
Mar 13 17:35:11 aborting phase 2 - cleanup resources
Mar 13 17:35:11 migrate_cancel
Mar 13 17:35:11 migrate_cancel error: VM 201 not running
Mar 13 17:35:13 ERROR: migration finished with problems (duration 00:00:18)
TASK ERROR: migration problems

after that the VM is switched off on the source-node.

VM-config

args: -device AC97,addr=0x18
bootdisk: virtio0
cores: 2
ide2: none,media=cdrom
memory: 1024
name: mint-client
net0: virtio=3A:62:63:30:39:65,bridge=vmbr0,tag=102
numa: 0
ostype: l26
smbios1: uuid=292b2f88-18d1-48a4-a272-429383cc587e
sockets: 1
usb0: spice
usb1: spice
usb2: spice
vga: qxl
virtio0: drbd1:vm-201-disk-1,size=17179869184

pve is on the same version on both nodes:

root@proxclt-a:~# pveversion -v
proxmox-ve: 4.1-39 (running kernel: 4.2.8-1-pve)
pve-manager: 4.1-15 (running version: 4.1-15/8cd55b52)
pve-kernel-4.2.6-1-pve: 4.2.6-36
pve-kernel-4.2.8-1-pve: 4.2.8-39
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 1.0-1
pve-cluster: 4.0-33
qemu-server: 4.0-62
pve-firmware: 1.1-7
libpve-common-perl: 4.0-49
libpve-access-control: 4.0-11
libpve-storage-perl: 4.0-42
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.5-9
pve-container: 1.0-46
pve-firewall: 2.0-18
pve-ha-manager: 1.0-24
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.5-7
lxcfs: 2.0.0-pve1
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5-pve7~jessie
drbdmanage: 0.91-1
openvswitch-switch: 2.3.2-2

If I switch the display to default the migration work well.

Udo

 

[udo]

the same happens without sound + usb-redirection

 

[spirit]

mmm, seem to be bug. (qemu or spice)

Qemu is waiting for spice client state transferred at the end of migration.
Does it work without sound and usb redirection ?

 

[udo]

Does it work without sound and usb redirection ?

Hi Spirit,
no - "display: qxl" is enough to fail live migration.

Udo

 

[spirit]

does it work with qemu 2.4 ?

 

[udo]

does it work with qemu 2.4 ?

Hi,
I have qemu 2.5 (pve4.x enterprise + nosubscription) and qemu 2.2 (pve3.4 enterprise) only...

Can try to install pve-qemu-kvm_2.4-18_amd64.deb on a test system (downgrade).


Udo

 

[udo]

does it work with qemu 2.4 ?

Hi Spirit,
qemu2.4 fails too

Mar 14 17:11:53 starting migration of VM 103 to node 'proxtest2' (172.20.2.132)
Mar 14 17:11:53 copying disk images
Mar 14 17:11:53 starting VM 103 on remote node 'proxtest2'
Mar 14 17:11:54 starting ssh migration tunnel
Mar 14 17:11:55 starting online/live migration on localhost:60000
Mar 14 17:11:55 migrate_set_speed: 8589934592
Mar 14 17:11:55 migrate_set_downtime: 0.1
Mar 14 17:11:55 spice client_migrate_info
Mar 14 17:11:57 migration status: active (transferred 239457768, remaining 738373632), total 2282569728)
Mar 14 17:11:57 migration xbzrle cachesize: 134217728 transferred 0 pages 0 cachemiss 0 overflow 0
Mar 14 17:11:59 migration status: active (transferred 475166569, remaining 497823744), total 2282569728)
Mar 14 17:11:59 migration xbzrle cachesize: 134217728 transferred 0 pages 0 cachemiss 0 overflow 0
Mar 14 17:12:01 migration status: active (transferred 709482990, remaining 257306624), total 2282569728)
Mar 14 17:12:01 migration xbzrle cachesize: 134217728 transferred 0 pages 0 cachemiss 0 overflow 0
query migrate failed: VM 103 qmp command 'query-migrate' failed - client closed connection

query migrate failed: VM 103 not running

query migrate failed: VM 103 not running

query migrate failed: VM 103 not running

query migrate failed: VM 103 not running

query migrate failed: VM 103 not running

Mar 14 17:12:19 ERROR: online migrate failure - too many query migrate failures - aborting
Mar 14 17:12:19 aborting phase 2 - cleanup resources
Mar 14 17:12:19 migrate_cancel
Mar 14 17:12:19 migrate_cancel error: VM 103 not running
Mar 14 17:12:20 ERROR: migration finished with problems (duration 00:00:27)
TASK ERROR: migration problems

Udo