[SOLVED] cannot migrate from storage type 'lvmthin' to 'zfspool'

[chuedmw]

I have one pve cluster, nodes : pve, pve-2, pve-3.
pve storage: local(dir), local-lvm(LVM-Thin), local-lvm2(LVM)
pve-2 storage: local(dir), local-lvm(LVM-Thin), local-zfs(zfs)

PS: migrate VM from pve to pve-2 and change the 'Target storage' to 'local-lvm' will success!

I migrate a VM from pve to pve-2, and change the 'Target storage' to 'local-zfs', then failure.



Error as follow:

Header
Proxmox
Virtual Environment 7.2-4
Search
Virtual Machine 101 (cloud-init-1-pve-1) on node 'pve'
Server View
Logs
()
2022-06-01 14:13:52 use dedicated network address for sending migration traffic (10.0.0.12)
2022-06-01 14:13:52 starting migration of VM 101 to node 'pve-2' (10.0.0.12)
2022-06-01 14:13:52 found generated disk 'local-lvm:vm-101-cloudinit' (in current VM config)
2022-06-01 14:13:52 found local disk 'local-lvm:vm-101-disk-0' (in current VM config)
2022-06-01 14:13:52 found local disk 'local-lvm:vm-101-disk-1' (in current VM config)
2022-06-01 14:13:52 copying local disk images
2022-06-01 14:13:52 ERROR: storage migration for 'local-lvm:vm-101-cloudinit' to storage 'local-zfs' failed - cannot migrate from storage type 'lvmthin' to 'zfspool'
2022-06-01 14:13:52 aborting phase 1 - cleanup resources
2022-06-01 14:13:52 ERROR: migration aborted (duration 00:00:01): storage migration for 'local-lvm:vm-101-cloudinit' to storage 'local-zfs' failed - cannot migrate from storage type 'lvmthin' to 'zfspool'
TASK ERROR: migration aborted

storage.cfg

dir: local
        path /var/lib/vz
        content iso,vztmpl,backup

lvmthin: local-lvm
        thinpool data
        vgname pve
        content rootdir,images

lvm: local-lvm2
        vgname local-lvm2
        content rootdir,images
        nodes pve
        shared 0

lvmthin: pve-3-sdb
        thinpool pve-3-sdb
        vgname pve-3-sdb
        content images,rootdir
        nodes pve-3

zfspool: local-zfs
        pool local-zfs
        content images,rootdir
        mountpoint /local-zfs
        nodes pve-2

VM config is 101.conf

agent: 1
boot: order=scsi0;ide0;net0
cipassword: ***
cores: 1
cpu: host
hotplug: disk,network,usb
ide0: local-lvm:vm-101-cloudinit,media=cdrom,size=4M
ipconfig0: ip=dhcp,ip6=dhcp
memory: 1024
name: cloud-init-1-pve-1
net0: virtio=66:****:4F,bridge=vmbr0,firewall=1
numa: 1
ostype: l26
protection: 1
scsi0: local-lvm:vm-101-disk-0,size=8396M,ssd=1
scsi1: local-lvm:vm-101-disk-1,size=1G
scsihw: virtio-scsi-pci
smbios1: uuid=90e27a02-***dae9
sockets: 1
sshkeys: ssh-rsa***
vmgenid: 90f73837-***8866

 

[shrdlicka]

Hi,
I played around a bit with this. The problem is the cloudinit drive that isn't moved that easily. You can remove it and restart the VM so it isn't used, then migration should work.

There is also a bug ticket which seems related:
https://bugzilla.proxmox.com/show_bug.cgi?id=3644

What I tried is move the cloudinit drive via CLI to a local ZFS storage

#qm move-disk <vmid> <disk> [<storage>] [OPTIONS]
qm move-disk 119 ide0 zfs

But this still needed a restart of VM so it doesn't use the old cloud drive on lvm.

 

[chuedmw]

Thx shrdlicka reply .

I remove VM 101 cloud init drive, and restart the VM.
Then it can migrate from pve to pve-2 with 'Target storage' 'local-zfs'

Will, My pve-2 node have a storage local-lvm, I can migrate my VM form pve to pve-2 with 'Target storage' 'local-lvm'.
And then I move the big disk from 'local-lvm' to 'local-zfs'

 

[chuedmw]

Hi,
I played around a bit with this. The problem is the cloudinit drive that isn't moved that easily. You can remove it and restart the VM so it isn't used, then migration should work.

There is also a bug ticket which seems related:
https://bugzilla.proxmox.com/show_bug.cgi?id=3644

What I tried is move the cloudinit drive via CLI to a local ZFS storage

#qm move-disk <vmid> <disk> [<storage>] [OPTIONS]
qm move-disk 119 ide0 zfs

But this still needed a restart of VM so it doesn't use the old cloud drive on lvm.

Hi, I found a thread as follow, now I have a work around to avoid restart VM

https://forum.proxmox.com/threads/online-migration-fails-with-cloud-init.65198/#post-294385

1. in shell, using command to remove cloud init drive, this will not restart VM, Make sure you know what you're doing

qm set 101 --ide2 none,media=cdrom

101 is my VM ID, ide2 is cloud init drive, make sure your cloud init dirve type is ide, other type not test, result example as follow



in web gui, the VM not restart and remove the cloud init drive



2. in web gui, you can migrate it now, and you can ping the VM ip


3. When migrate success, in web gui, you can add cloud init drive again.

 

[kajikoffo]

Hi,
just came accross this earlier as trying to decomission old node and moving VMs to a new node (both joined to cluster for live migration purpose). Cloud-init wasn't my case.

I found out I had trouble with VM having two VHDs. All of others had no problem, only this one did. Both nodes also had common storage mounted from NAS for backups, so I tried to move both VHDs to NAS and then migrate - no success, 'lvm-thin to zfs' error became 'storage [local_storage] doesn't exist on [node2]'.

So I did it 'old offline way', turned VM off to have consistent data, dumped VM to NAS with vzdump [VMid] --mode snapshot --storage [NAS] and restored on node2 under new VMid (could dismount cluster first and use original one) with qmrestore [image_path] [VMid] --storage [local_storage_on_node2]