KVM hot migration thinks NFS image is local

[eswood]

HI folks,

We have a 2 node PVE 2.2 cluster (yes, we are looking in to upgrading to 3.1 soon). We have both local storage and NFS storage options available for images. I recently manually moved a KVM from local to NFS so that we would have the benefit of live migration at our disposal. I did this with the following steps.

1) stop the VM
2) copy the image to the NFS storage
3) edit the /etc/pve/nodes/hostname/qemu-server/VID.conf file to reflect the new location
4) start the VM

The move went well. Then I tried to test the live migration. It failed, saying that the VM uses local disks, as follows:

Oct 11 16:32:54 starting migration of VM 115 to node 'newtarget' (10.11.12.13)
Oct 11 16:32:54 copying disk images
Oct 11 16:32:55 ERROR: Failed to sync data - can't do online migration - VM uses local disks
Oct 11 16:32:55 aborting phase 1 - cleanup resources
Oct 11 16:32:55 ERROR: migration aborted (duration 00:00:01): Failed to sync data - can't do online migration - VM uses local disks
TASK ERROR: migration aborted

Is there any easy answer that would explain why? Is there a service that may be caching info about the VM that needs restarting? Did I move the host to the NFS storage incorrectly? Any tips would be greatly appreciated. If any more info would be helpful in troubleshooting, just let me know. Thanks in advance.

Regards,
Scott

 

[screenie]

is the nfs storage marked as 'shared' ?

 

[eswood]

On the "Storage" tab is shows as "Type NFS" and when that is selected, it does show up as an "NFS share". It was mounted via the proxmox interface, not manually and is mounted at /mnt/pve/sharename, so as far as I can tell, Proxmox should know that it's not local.

 

[christophe]

A CD-ROM reader? Local iso storage?

Christophe.

 

[eswood]

A CD-ROM reader? Local iso storage?

Christophe.

Hey Christophe,

Thanks for the suggestion (and a belated thanks to screenie). There was a cd device (ide2), but nothing was mounted in it. I removed the device, just for good measure, to no avail. Still no joy. Any other ideas, folks?

Scott

 

[tom]

post the following:

cat /etc/pve/storage.cfg

qm config 115

 

[eswood]

Hey Tom,

Here's the output of the commands run on the host that currently houses the VM. All that I have changed inthe output is the hostname, vmname, and the VM's IP Address in the output of "qm config 115". Thanks for investigating.

servername:~# cat /etc/pve/storage.cfgdir: local
path /var/lib/vz
content images,iso,vztmpl,rootdir


nfs: backups
path /mnt/pve/backups
server disk-array
export /exports/backups
content backup
maxfiles 10


nfs: sharedVMs
path /mnt/pve/sharedVMs
server disk-array
export /exports/VMs
options vers=3
content images
maxfiles 1


servername:~# qm config 115
acpi: 1
boot: cd
bootdisk: ide0
cores: 2
cpuunits: 1000
description: 10.11.12.13%0A
freeze: 0
ide0: sharedVMs:115/vm-115-disk-1.raw
kvm: 1
memory: 8192
name: vmname
net0: rtl8139=46:2E:40:35:59:59,bridge=vmbr0
onboot: 1
ostype: other
sockets: 1

Regards,
Scott

 

[tom]

looks ok.

is there still content in /var/lib/vz/images/115 ?
remove the 115 directory.

but as you run a quite old version, you are probably hit by an old bug.
=> plan the move to latest stable.

 

[eswood]

Well, after deleting the /var/lib/vz/images/115 folder, I still wasn't able to do an online migration from servername no newtarget. I tried an offline migration and that was successful. Then an online migration BACK from newtarget to servername succeeded. A subsequent online migration from servername to newtarget failed again.

We'll look at doing the 2.2 -> 2.3 -> 3.x upgrade on our servers and see if it addresses our issues. Thanks to everyone for their suggestions.

Regards,
Scott

 

[eswood]

Just to update, upgrade to 3.1 is done. KVM VMs on NFS storage happily migrate now. Thanks!